Problem with Cache Manager after minifilter provides dynamic file contents

NTFSD
1

Dear members of the ntfsd mailing list,

I am using a Kernel Mode minifilter driver in one of the most unlikely scenarios -
providing global photorealistic terrain to Microsoft’s Flight Simulator X. It’s a
noncommerical project, a hobby so to say. The project works fine on Windows XP
so far and is well received by testers. I am not an experienced kernel mode driver
programmer yet - this is my first endeavour in this direction.

Let me quickly explain how the driver works before I describe my problem that I see
on Windows Vista.

I use a driver to modify the content of BMP (bitmap) files on disk as they are read by
an application (fsx.exe). If the BMPs don’t exist yet or the level of detail is below the
requirement, I pass the filename on to a userspace application which downloads and
the files from the Internet . The filter driver then makes the BMP files appear at any
required detail level (resolution, and thus file size) depending on distance to the aircraft:

My project is based on the “scanner” minifilter sample by Microsoft. The following
essential operations (among others) are performed by the driver for BMP files that
meet certain filename requirements. All other I/O is handled transparently.

* In IRP_MJ_NETWORK_QUERY_OPEN (Preop) I provide a modified file size for the file on
disk. Turns out the flight sim uses this information to determine whether a file exists
or not. I provide the intended file size here as well.

* In IRP_MJ_QUERY_INFORMATION (Preop) I intercept queries for FileStandardInformation and
return the intended file size for the BMP file.

* In IRP_MJ_READ (Preop), I satisfy read calls to the file contents with bitmap header
and data in the intended resolution.

Note that the file size of the BMP and contents as provided by the filter driver may
differ from the file size and content on disk! That seems to be one cause of my problem.
The content as seen by the application is *volatile* after going through the filter driver,
however the Cache Manager in the Windows Kernel assumes it can cache both the file size
and the file contents. If the aplication explicitly requested non-cached I/O, this
would not be a problem. But the application (Flight Simulator) uses CreateFile() with
standard parameters, and that triggers the Cache Manager.

Nevertheless the filter has worked fine on Windows XP so far, users are happy.

However on Windows Vista I am seeing some problems. That is some reads seem to
bypass the Minifilter driver and are serviced directly by the Cache Manager. Bad thing.
The contents of the BMP files need to keep changing to increase level of detail as
aircraft nears a given terrain tile. However when CacheManager satisfies the reads,
the content is static. It works most of the time, but some terrain tiles are getting
corrupted or remain blurred (low detail).

At first I suspected the SuperFetch feature of Vista to be responsible. But that appears
not to be the problem. A “net stop superfetch” command did not help out. In particular
I would like to know why Vista shows so different behavior to Windows XP although both
systems have the Cache Manager.

Before you point out the obvious, that is “Get Mark Russinovich’s Book Windows Internals
and learn the delicacies of Paged I/O, Memory mapping and the Cache Manager”, I would
like to hear an opinion whether my approach is totally insane and not workable, or whether
a slight modification could make the thing work on Vista. (By the way, I have just
ordered the above book for further studies.)

Can a minifilter driver tell the upper layers to not cache the returned data - or does
only the filesystem have this power? The intended effect is similar to an application
using CreateFile() with FILE_FLAG_NO_BUFFERING at the application level.

I cannot really change the application’s behaviour unless I hooked into the running
process and changed the respective CreateFile() calls to use nonbuffered I/O exclusively.
If I can’t solve the problem at the driver level, this is going to be my last resort -
kind of hacky though.

Christian

2

Apologies for the bad formatting, I added my own line breaks in web access to the mailing list and won’t do this the next time I post here.

3

I’d say that you will need to *disable* superfetch, not stop it (it starts
stuffing things into the cache shortly after boot time). This may well be
a bad thing to do to your users.

insane and not workable

I had “certain reservations” about this when I read your description, which
are only somewhat moderated by the fact that you are not writing a
commercial product and are targetting an specific application. As I think
some moree I am verging on the “it’s unworkable”, however to help you as you
await the text books:

Can a minifilter driver tell the upper layers to not cache the returned
data -
or does only the filesystem have this power?

The minifilter can try (set the correct bits in the FileObject->Flags during
pre-create), but the FSD has every right to ignore it and some do as a
matter of course. If the data is in the cache when the open happens
the damage may be done already.

Some applications (notepad is the classic) don’t even use a Read() to get
the data - they just map a section. It could well be that your application
is doing this in which case you are SOL (you don’t mention which flavors of
read you are filtering and scanner doesn’t have a read path so I’m guessing)

Note that the file size of the BMP and contents as provided by the filter
driver may differ from the file size and content on disk! That seems to be
one cause of my problem.

Absolutely, many have fought long and hard to change the size (in encryption
& compression filters) and what you are doing is *much* harder - if I
understand you, the same file will return different data (and hence size)
for the same read to the same file depending on where the plane is (if it
was always the same you’d just do the copy in the precreate, right?). This
sounds pretty hairy to me.

Given you are doing this as a hobby and cannot expect your manager to have
trained you, you have a long, hard learning cliff to go up.

I’d take a look at the Russinovitch book, the Nagar book and then look in
this conference for postings on size changing in filter drivers, then
consider
your architecture again.

So to go back to your question. I am actually not sure whether this scheme
*is* workable (I’d need to spend a lot of time doing a detailed analysis and
nature study of the application and what you want to achieve), if it is
workable it is probably dependant on specific version of the application you
are targetting. Even then it will be extremely difficult to do a throrough
job and less than 100% is usually not an option in kernel mode. Making
something work under a specific set of circumstances is nothing like a
guarantee that it will always work (even against just one application).
Remember that the filesystem is system wide and even though you may only be
serving one application you will have side effects on others.

Have fun !