Re[2]: minifilter Microsoft validation question

In addition to Rod’s comments …

Are you doing this for ‘everything’ on a given volume? Or for specific
processes and/or directories? If the former, then you will need to be
particularly careful about handling requests to the paging file. Don’t
confuse this with paging IO for file backed sections but IO destined for
the paging file such as paging out pageable memory, etc. There are many
additional restrictions on these pathways. If you are not processing
‘everything’;i.e. the latter, note that tests such as the IFSTest suite
create folders to perform they’re tests.

Thanks,

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

------ Original Message ------
From: “Rod Widdowson ”
To: “Windows File Systems Devs Interest List”
Sent: 9/14/2017 8:11:12 AM
Subject: Re:[ntfsd] minifilter Microsoft validation question

>>The reason I am suspicious about this not getting thought
>>certification is that such a minifilter would
>>not allow any I/O whatsoever go down below it. Is this behaviour
>>permitted by Microsoft?
>
>It actually sounds as if you are implementing a file system - just
>using minifilter to do the work. I can see the attraction.
>
>There is no reason why your filter shouldn’t pass the tests -
>eventually. As Peter says, you have to make it look like everything
>else, and given that I haven’t seen you in here before (forgive me if I
>don’t recognise your name) I’ll give the the statutory warning that
>this could take calendar years and 10s of many months.
>
>You are certainly going to have to have implemented the cache manager
>interfaces (pretty much like an Isolate filter would). Apart from the
>fact that things like Notepad won’t work there are a whole series of
>tests which exercise that very fully (causing reads and write via the
>cache, around the cache and with pagefaults).
>
>You may well need to have some careful consideration as to how you
>respond to the FSCTLs which query for the layout of the file on disk.
>I’m not aware of any test which actually goes around the file system to
>check (although there are plenty of other “tools” that do so).
>
>Other thoughts. You are going to have to make sure that you implement
>directory oplocks too,. I am not sure what the state of those tests
>are as of the most recent builds but there is a huge, very extensive
>(and thoroughly useful, if naggingly annoying) test which just tests
>that.
>
>Oh yea, and reparse points (although ISTR you can turn what off in the
>Volume Query Response and recent versions of the tests do test to
>respect that).
>
>If none of the above frightens you then you should be OK (eventually).
>If anything has you scratching your head you may need to look very
>closely (and soon) at these tests.
>
>/R
>
>—
>NTFSD is sponsored by OSR
>
>
>MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>software drivers!
>Details at http:
>
>To unsubscribe, visit the List Server section of OSR Online at
>http:</http:></http:>

Thank you, Rod. That was very full of valuable information for me.

You got that right, I haven’t been around here, this is my first question.

However, I didn’t understand whether everything that you wrote targets only minifilters that attach to NTFS-formatted volumes, or it applies to every minifilter, irrespective of the volume’s formatting. I’m asking this because I just read another question asked and answered here, on osronline.com, where the topic was whether a minifilter can attach to a FAT32-formatted volume. The link is this: https://www.osronline.com/showthread.cfm?link=285527.

It appears from that discussion that one minifilter could avoid all NTFS-formatted volumes by simply looking at the VolumeFilesystemType param for PFLT_INSTANCE_SETUP_CALLBACK routine.

My new question is: if I solely attach to FLT_FSTYPE_EXFAT volumes, would I still have to deal with all that hard work that you described? (btw it had me scratch my head…)

In short, no. The IFSTest suite requires that you format a volume for
NTFS, compressed NTFS, FAT16, FAT32, ExFat and UDF … so if you do not
attach to some of those volumes then you’ll ‘pass’ the tests on those
volumes. You’ll just need to deal with the volumes you do attach to and
correctly pass back information for that particular FS. As Rod pointed
out, there are lots of FS specific tests but also it looks for FS
specific failure codes for requests. For example, NTFS and FAT return
different failures codes for an invalid path name, IIRC, abd the IFSTest
checks the return based on the FS.

In the end, once you have your system running, it’s quite easy to run
the IFSTest suite. And running specific tests is also quite easy so you
can narrow down specific failures and address them.

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

------ Original Message ------
From: “xxxxx@simplision.com
To: “Windows File Systems Devs Interest List”
Sent: 9/14/2017 9:07:59 AM
Subject: RE:[ntfsd] minifilter Microsoft validation question

>Thank you, Rod. That was very full of valuable information for me.
>
>You got that right, I haven’t been around here, this is my first
>question.
>
>However, I didn’t understand whether everything that you wrote targets
>only minifilters that attach to NTFS-formatted volumes, or it applies
>to every minifilter, irrespective of the volume’s formatting. I’m
>asking this because I just read another question asked and answered
>here, on osronline.com, where the topic was whether a minifilter can
>attach to a FAT32-formatted volume. The link is this:
>https://www.osronline.com/showthread.cfm?link=285527.
>
>It appears from that discussion that one minifilter could avoid all
>NTFS-formatted volumes by simply looking at the VolumeFilesystemType
>param for PFLT_INSTANCE_SETUP_CALLBACK routine.
>
>My new question is: if I solely attach to FLT_FSTYPE_EXFAT volumes,
>would I still have to deal with all that hard work that you described?
>(btw it had me scratch my head…)
>
>—
>NTFSD is sponsored by OSR
>
>
>MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>software drivers!
>Details at http:
>
>To unsubscribe, visit the List Server section of OSR Online at
>http:</http:></http:>

> My new question is: if I solely attach to FLT_FSTYPE_EXFAT volumes, would

I still have to deal with all
that hard work that you described? (btw it had me scratch my head…)

Most of it. You’d need to worry about the cache (HUGE), but not reparse
points, compression (I think), or EFS - but all of those can be wished away
at the volume query level…

Let’s do a wings on pigs reset
(http://www.osronline.com/downloads/pp_asking.pdf). What are you trying to
achieve?

Thank you, Peter. Thank you, Rod.

Hope I’m not trying to glue wings on a pig here :slight_smile: Actually, what’s all this about is for a file system which will work with a DB instead of a disk. So, each relevant I/O would be mapped to a corresponding DB query. Of course that not all I/Os would have a translation. For those I would look for a trivial implementation or even an unsupported/unimplemented kind of response that would not disrupt the normal flow that the OS would except from that respective minifiltered volume.

That’s why I said about ExFat, because it lacks certain features of NTFS that would anyhow not get treated by a DB query, so instead of ending in a trivial implementation, I was thinking that is better to not have those calls at all.

Is this a fair approach?

Mr. Darau… So, you ARE writing a file system, but you just want to implement it as a MiniFilter?

You might be MUCH better off with just writing a new Installable File System. If your goal isn’t to actually FILTER a given volume, but rather to instantiate an entirely NEW volume, this would definitely be the way to go.

AND… as a bonus… you won’t have to pass the mini-filter HLK tests.

Peter
OSR
@OSRDrivers

I decided to follow your advices and use an installable file system. From what I can see in the windows-drivers-samples repo folder, there are only 2 such file systems available, namely cdfs and fastfat, the former being readonly. Is this a good point to start my work?

Thank you for assistance.

> I decided to follow your advices and use an installable file system. From

what I can
see in the windows-drivers-samples repo folder, there are only 2 such file
systems
available, namely cdfs and fastfat, the former being readonly. Is this a
good point
to start my work?

Training is the very best place to start. This is a huge area which you
don’t “just implement”.

Our lists hosts, OSR, do an excellent class. There is/was a book floating
about, but I found it inpenetrable (and other books point out that it is
“more accurate for Windows NT 3.51 than it is for later versions”, which is
not a great recommendation for its up-to-dateness)

Failing that FAT is my go to reference for the easy bits, although I find
its comments a bit opaque at times if not down right frustrating (e.g. line
4147 of [1])

Good luck

[1]
https://github.com/Microsoft/Windows-driver-samples/blob/master/filesys/fastfat/fileinfo.c

At least they are consistent. They repeat the same comment on line 4453!

Sent from Mailhttps: for Windows 10

From: Rod Widdowson mailto:xxxxx
Sent: September 16, 2017 10:12 AM
To: Windows File Systems Devs Interest Listmailto:xxxxx
Subject: Re:[ntfsd] Re[2]: minifilter Microsoft validation question

> I decided to follow your advices and use an installable file system. From
> what I can
> see in the windows-drivers-samples repo folder, there are only 2 such file
> systems
> available, namely cdfs and fastfat, the former being readonly. Is this a
> good point
>to start my work?

Training is the very best place to start. This is a huge area which you
don’t “just implement”.

Our lists hosts, OSR, do an excellent class. There is/was a book floating
about, but I found it inpenetrable (and other books point out that it is
“more accurate for Windows NT 3.51 than it is for later versions”, which is
not a great recommendation for its up-to-dateness)

Failing that FAT is my go to reference for the easy bits, although I find
its comments a bit opaque at times if not down right frustrating (e.g. line
4147 of [1])

Good luck

[1]
https://github.com/Microsoft/Windows-driver-samples/blob/master/filesys/fastfat/fileinfo.c


NTFSD is sponsored by OSR

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></mailto:xxxxx></mailto:xxxxx></https:>