We are opening a network file on Windows using the CreateFile API. According to MSDN, the flags FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH should bypass local caching and write directly to disk. However, we observe that the file still opens with an opportunistic lock. Could you advise if there are additional attributes or flags that can be used to prevent this?
these flags are usually used when you want to be sure that a disk write has been committed to persistent storage. When you are writing to a data base file or something where half complete writes will corrupt data, the file system cache is useless, and you want the SCSI FUA (force unit access) bit set (and equivalents in more modern SAS, NVMe etc.).
Oplocks are used with network shares. The lan manager client indicates that they would like the server to lock a particular file, and the server agrees that it will try. And will also try to notify you when it breaks that promise.
There isn’t really any overlap between these concepts. If you explain more about what you are trying to do, it is likely that you will get better advise.
Thank you for your reply @Scott_Noone_OSR . We are not using the CopyFile.
We are using the CreateFile API to open and create files on a network server. The server, which uses a Linux-based OES file system, is the destination for these files.
When we open or create a large number of small files (e.g., 5,000 files with sizes less than 100 KB), we are observing a significant performance impact. We believe this is caused by the overhead of CreateFile requesting an opportunistic lock (oplock) from the server for each file.
To mitigate this, we have already tried using the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH attributes in our CreateFile calls, as suggested in the MSDN documentation. However, our testing shows that CreateFile is still requesting oplocks.
Could you please advise on how to disable the oplock request from CreateFile?
Thank you for your reply @MBond2 . I am sharing the details step by step
We are using the CreateFile API to open and create files on a network server (network share). The server, which uses a Linux-based OES file system, is the destination for these files.
When we open or create a large number of small files (e.g., 5,000 files with sizes less than 100 KB), we are observing a significant performance impact. We believe this is caused by the overhead of CreateFile requesting an opportunistic lock (oplock) from the server for each file.
To mitigate this, we have already tried using the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH attributes in our CreateFile calls, as suggested in the MSDN documentation. However, our testing shows (WireShark lan traces) that CreateFile is still requesting oplocks.
Could you please advise on how to disable the oplock request from CreateFile?
okay you are using a network share. For sure the flags FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH will have no effect.
oplocks don’t usually have a serious performance effect, and 5000 HANDLEs isn’t usually very many. But I know far more about Windows file servers than non-Windows ones. How are you concluding that oplocks represent a performance problem?
AFAIK there isn’t a standard way to prevent the network client from requesting an oplock from the server. The server doesn’t have to grant one, but the client will request one.
How are you concluding that oplocks represent a performance problem?
===>When the Windows operating system requests an oplock, the network server first broadcasts to all machines on the network. This broadcast is causing a performance impact. Granting the oplock itself does not affect performance, but the broadcast message introduces delay.
There’s nothing in the create that says “don’t give me an oplock”. If you ask for data access then the SMB client driver is going to request one as part of the open.
Do you see the same results of network activity if you use a Windows server for the share? Sounds weird that the server is going all over the network telling people about oplocks for each open. In Windows the arbitration of the oplock is handled by the underlying file system and not by just broadcasting oplock requests everywhere. Now, if you have some other machine on the network simultaneously opening the files for data access then yeah, you’re going to get traffic.
It sounds like you need to look at why the server does that. And if there are options to control that behaviour.
I know far more about Windows than any Linux, but on Windows the server doesn’t broadcast anything when an oplock is requested. Nor would you expect it to since the lock is a contract between the server and the one client requesting it
Thank you very much for your reply. We have a Linux file system and are currently investigating the reason for the broadcast message being sent while opening the file for write. We found that if we set the access right as ‘DENY_WRITE’ while opening the file, then the broadcast message is not sent. Thank you again for your reply.