Should a layered FS minifilter take over oplocks and file locks completely?

NTFSD
1

Hello,
I have been pondering over a possible implementation of a layered file system mini filter.
I have a basic question in this context.
During the implementation of a layered FS minifilter (which could be a compression or a virtualization minifilter), should the minifilter completely take over control of the oplocks and file locks? And not forward those requests to the lower file object?
Should i worry about other minifilters sitting below me that can cause problem if I dont forward these requests to the lower file object and instead manage them totally at my layer on the Shadow file object?

Thanks,
Tushar

2

If you own the cache, it is logical for you to own the oplock. Depending upon the specifics of your mini-filter, you could pass along the oplock to the lower level file system.

Note that there are also interactions with byte range locks and oplocks.

Finally, I would caution you: oplock implementation changed substantially in Windows 7 and has further changes in Windows 8.

Tony
OSR

3

Thanks Tony for the reply.
Yes, my minifilter is going to own the cache (i dont see any other way in which i can achieve the compression functionality seamlessly).

Can you give an example as to when I can just pass along the oplock FSCTL and IRP_MJ_LOCK_CONTROL to the lower file system (using the lower file object)? And when I cannot?

Does that mean that I need to implement a version specific thing if I had to support XP, 2K3, Vista, 7 and 8?

Thanks,
Tushar

4

Sure: how about a directory oplock. In addition, you may find your filter dealing with some files that you are not compressing, in which case you would probably pass those down to the underlying FSD as well.

Yes.

Tony
OSR

5

I think implementing oplocks at different layers (handling some in a filter and letting others through, to the file system) is actually pretty complicated. The problem is that directory renames will walk through the subtree of the directory and look at the oplocks for any open files (see FatSetRenameInfo in the FastFat sample). So in order to preserve file system semantics you must break all the oplocks for all the files that you implement oplocks for in your filter before you let the directory rename through (just like the file system does) and moreover you need to synchronize this with the rename in the underlying file system. It’s not impossible but I found it very complicated.

If you actually intend to implement the namespace in your filter (so that you handle directory renames in some cases yourself) then you’re in even more trouble because you can’t look at the oplock state and break oplocks in the filesystem from a filter so you can’t implement the same semantics (and so you might be better off taking over oplocks for all the files in your namespace, irrespective of whether you virtualize them or not).

In my oppinion oplocks link file IO with the namespace in very complicated ways so once you start virtualizing them you’ll soon end up with something at least as complex as a file system…

Thanks,
Alex.

6

I do agree with Alex - it is very complex. My approach in the past has always been to refuse to grant oplocks as the default implementation and then add in only what was absolutely necessary. After all, oplocks are an optimization mechanism and getting them wrong leads to potentially ugly problems (because they manifest as cache incoherency issues). For many years this approach has worked fine because only the SMB server really used oplocks and it handled failures (including not being granted an oplock) gracefully.

Unfortunately, applications are now using them. They are proving to be intolerant of refusing to grant an oplock, which leads to application compatibility errors. For example, one of my customers has found that if a filter rejects oplocks on Windows 8, the tiles displayed by Metro are blank in some instances.

This is forcing us into a position of not only supporting oplocks but supporting essentially ALL oplocks - including oplocks on directories (new in Windows 8).

Here is the code in the FAT fsctl handler for the oplock request in Windows 7:

//
// We only permit oplock requests on files.
//

if ( FatDecodeFileObject( IrpSp->FileObject,
&Vcb,
&Fcb,
&Ccb ) != UserFileOpen ) {

FatCompleteRequest( IrpContext, Irp, STATUS_INVALID_PARAMETER );
DebugTrace(-1, Dbg, “FatOplockRequest -> STATUS_INVALID_PARAMETER\n”, 0);
return STATUS_INVALID_PARAMETER;
}

And here is the same routine in the last preview release for Windows 8:

//
// We permit oplock requests on files and directories.
//

if ((TypeOfOpen != UserFileOpen) &&
(TypeOfOpen != UserDirectoryOpen)) {

FatCompleteRequest( IrpContext, Irp, STATUS_INVALID_PARAMETER );
DebugTrace(-1, Dbg, “FatOplockRequest -> STATUS_INVALID_PARAMETER\n”, 0);
return STATUS_INVALID_PARAMETER;
}

This is even beyond the rename issue (note the FAT rename sample is quite complex and is related to how FAT deals with batch oplocks.)

Tony
OSR

7

Thanks Alex and Tony.
To be honest, I was planning to simply return some error like STATUS_NOT_SUPPORTED or STATUS_NOT_IMPLEMENTED for the Oplock FSCTL and not worry about them in the first step.

Byte range locks still need to be handled; which should be fine given that oplocks can be ignored in the first step. How about double layer of locking there? It will be like manipulating the lock in ShadowFileObject and then passing the request to the lower layers using the “lower/ original file object”… What’s your opinion?

I havent had a look at Windows 8, but if you say oplock getting used by applications… Well, I hope these are not one of those that I will have to support.

8

A good strategy to start out.

Byte range locks are different than oplocks. For several types of oplocks, taking out a byte range lock BREAKS the oplock (this forces a “back to the server” model so that the byte range locks can be properly enforced).

If you think you can avoid supporting Metro, I wish you luck… I don’t see this as being optional unless someone changes this for RTM.

Tony
OSR

9

It is unfortunate then. Maybe it’s time I start researching windows 8 too.
If metro apps are the the new things that are going to be too popular then I have to rethink.
Managing the cache from mini filter seems hard enough; oplocks seem to make it a nightmare…

10

+1. But if it’s mandatory then it needs to get done…

Sent from my iPhone

On Jun 28, 2012, at 12:34 PM, xxxxx@gmail.com wrote:

It is unfortunate then. Maybe it’s time I start researching windows 8 too.
If metro apps are the the new things that are going to be too popular then I have to rethink.
Managing the cache from mini filter seems hard enough; oplocks seem to make it a nightmare…

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer