such as retrieving the file name. I understand
FltGetFileNameInformation() might be expensive due
to the name cache path, but I’m wondering how bad this could be
comparing to the legacy IRP-based query.
Thanks
Vitaly
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Shangwu
Sent: Monday, December 17, 2007 2:29 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] What are the limitations of Minifilter
Which function causes the mini-filter’s overhead twice of the legacy
one?
Thanks,
Shangwu
“Vitaly Vatnikov” wrote in message news:xxxxx@ntfsd… Sorry for steering the discussion into a slightly different direction, but does anyone have data/opinion on the relative performance minifilter vs. legacy implementation? In some test cases I’ve seen a minifilter driver overhead being almost twice of the legacy one (for a similar functionality, of course). Just curious if anyone has some insights on the subject.
Thanks,
- Vitaly
-----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn Sent: Monday, December 17, 2007 12:14 PM To: Windows File Systems Devs Interest List Subject: Re:[ntfsd] What are the limitations of Minifilter
“Ayush Gupta” wrote in message news:xxxxx@ntfsd… > Hi Don, > > > Flt manager makes the file system stack reliability WORSE, since it > makes it look easy to do a filter driver. > > > Interesting point Don. Can you please give some insight as to why?
I make this claim, since mini-filters create the illusion that is much easier to create a file system filter than it is. What this is doing is create a situation where companies are jumping into things needing a file system filter without the people, the budget or a plan on how they are going to get there or maintain it after the fact.
Unlike WDF which definitely took a number of the nasties out of kernel driver developlment, mini-filters hide some things (none of which were that hard) and add challenges due to the limits of the current implementation. I like mini-filters but they really do not solve any of the hard problems of filter drivers.
The answer to the “is it better” question is, of course, “it depends”… For
example, the main advantage of the FDDK at this point is that it’s supported
on versions of the O/S that the Filter Manager is not. So, if your customers
are demanding support for Win2K Sp2 (trust me, it happens) then the FDDK is
really the only choice outside of starting from a WDK sample.
OTOH, Filter Manager *is* the blessed, forward moving model, so maybe it’s
better to switch to that model so that you’re on the train and can influence
the direction.
For the DMK, we’ve chosen to implement it with both models. This greatly
increases the test matrix, but allows us to give our customers down rev O/S
support while sticking to the recommended model where available.
“Ayush Gupta” wrote in message news:xxxxx@ntfsd… > Thanks Don, > > Has anyone any idea of advantages of using the FDDK? > The max that I know about it is that it makes development of legacy > filters > a bit easier… > Is it better to develop a mini-filter or use FFDK to develop a legacy > filter? > > >
I’ve worked through most of the filter manager sources doing my first
mini-filter. I think for many things the performance should be comparable.
Yes you can always optimize things more when you have total control, but you
are doing more work. My impression is they tried to keep the layer thin,
and the implementation is reasonable.
“Vitaly Vatnikov” wrote in message news:xxxxx@ntfsd… Sorry for steering the discussion into a slightly different direction, but does anyone have data/opinion on the relative performance minifilter vs. legacy implementation? In some test cases I’ve seen a minifilter driver overhead being almost twice of the legacy one (for a similar functionality, of course). Just curious if anyone has some insights on the subject.
I did a test to compare the communication port vs device IO control. The
user mode program sends and receive 1 million messages with 4K bytes data
packet. The communication port only takes 6 seconds. The device IO control
takes 12 seconds on my test machine.
What is your test result?
-Shangwu
“Iñaki Castillo” wrote in message news:xxxxx@ntfsd… In all our tests the performance of our minifilter implementation was worse than the legacy filter. Of course it may be because or legacy filter is a mature product while our minifilter is not.
I suspect in our case the penalty might be located in the kernel-user communications path (we use the api provided by MS for MF). In our application there is plenty of communication overhead. In other minifilters we have, not as heavy, we did not notice a significant decrease in performance.
I would be interested to hear how well minifilters behave in terms of performance for other applications.
Inaki.
-----Mensaje original----- De: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] En nombre de Vitaly Vatnikov Enviado el: lunes, 17 de diciembre de 2007 20:04 Para: Windows File Systems Devs Interest List Asunto: RE: [ntfsd] What are the limitations of Minifilter
Sorry for steering the discussion into a slightly different direction, but does anyone have data/opinion on the relative performance minifilter vs. legacy implementation? In some test cases I’ve seen a minifilter driver overhead being almost twice of the legacy one (for a similar functionality, of course). Just curious if anyone has some insights on the subject.
Thanks,
- Vitaly
-----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn Sent: Monday, December 17, 2007 12:14 PM To: Windows File Systems Devs Interest List Subject: Re:[ntfsd] What are the limitations of Minifilter
“Ayush Gupta” wrote in message news:xxxxx@ntfsd… > Hi Don, > > > Flt manager makes the file system stack reliability WORSE, since it > makes it look easy to do a filter driver. > > > Interesting point Don. Can you please give some insight as to why?
I make this claim, since mini-filters create the illusion that is much easier to create a file system filter than it is. What this is doing is create a situation where companies are jumping into things needing a file system filter without the people, the budget or a plan on how they are going to get there or maintain it after the fact.
Unlike WDF which definitely took a number of the nasties out of kernel driver developlment, mini-filters hide some things (none of which were that hard) and add challenges due to the limits of the current implementation. I like mini-filters but they really do not solve any of the hard problems of filter drivers.
>Not to jump into the middle of the thread, but has anybody looked to see
if any of the structures pointed to by the FLT_CALLBACK_DATA structure
actually live in the IRP? If they were, you could probably get your
paws on the IRP using CONTAINING RECORD.
No, they do not live in IRP.
They probably have some undocumented wrapper around them which references the
IRP, but they do not live in IRP for sure.
>Fix FltPerformXXXIo so that all requests can be composed and forwarded
to lower levels. The current restriction to a subset of the FltXXX handle
calls is a total joke.
I think what the documentation says is :
For FltPerformAsynchronousIo: -----------------------------
Minifilter drivers should use FltPerformAsynchronousIo only for asynchronous I/O operations for which routines such as the following cannot be used:
FltReadFile
FltWriteFile
For FltPerformSynchronousIo: -----------------------------
Minifilter drivers should use FltPerformSynchronousIo only in cases where routines such as the following cannot be used:
FltClose
FltCreateFile
FltQueryInformationFile
FltQueryVolumeInformation
FltReadFile
FltSetInformationFile
FltSetVolumeInformation
FltTagFile
FltUntagFile
FltWriteFile
And it simply means that since functions like FltReadFile, FltWriteFile, etc can be called ONLY at PASSIVE_LEVEL, FltPerformXXXIo can be used in cases where a caller needs to read/ write (or some other operation) at an IRQL = APC_LEVEL. Of course, the usual restrictions that exist for Legacy filters also apply… For example IRP_MJ_CREATE cannot be issued using FltPerformAsynchronousIo, since this IRP cannot be processed asynchronously.
Maybe you were talking about some other restrictions.
That is an interesting list, I can query security but not set it. There are
no calls to hanlde quotas, etc. In fact the documentation is misleading in
that is says to use these calls for things that can’t be done with things
like FltWriteFile, but in actuallity most of the calls you can do have
FltXXX routines.
“Ayush Gupta” wrote in message news:xxxxx@ntfsd… > >>Fix FltPerformXXXIo so that all requests can be composed and forwarded >>to lower levels. The current restriction to a subset of the FltXXX handle >>calls is a total joke. > > I think what the documentation says is : > > > > For FltPerformAsynchronousIo: > ----------------------------- > > Minifilter drivers should use FltPerformAsynchronousIo only for > asynchronous > I/O operations for which routines such as the following cannot be used: > > FltReadFile > > FltWriteFile > > For FltPerformSynchronousIo: > ----------------------------- > > Minifilter drivers should use FltPerformSynchronousIo only in cases where > routines such as the following cannot be used: > > FltClose > > FltCreateFile > > FltQueryInformationFile > > FltQueryVolumeInformation > > FltReadFile > > FltSetInformationFile > > FltSetVolumeInformation > > FltTagFile > > FltUntagFile > > FltWriteFile > > > > And it simply means that since functions like FltReadFile, FltWriteFile, > etc > can be called ONLY at PASSIVE_LEVEL, FltPerformXXXIo can be used in cases > where a caller needs to read/ write (or some other operation) at an IRQL = > APC_LEVEL. > Of course, the usual restrictions that exist for Legacy filters also > apply… For example IRP_MJ_CREATE cannot be issued using > FltPerformAsynchronousIo, since this IRP cannot be processed > asynchronously. > > Maybe you were talking about some other restrictions. > > Regards, > Ayush Gupta > >
To add to everything said, unlike legacies, Filter Manager mini-filters do not support filtering on the named pipe file system. This feature has been requested a number of times but apparently not enough for Microsoft to start working on it. Please add your voices to that extent.
That is an interesting list, I can query security but not set it. There
are
no calls to hanlde quotas, etc. In fact the documentation is misleading in
that is says to use these calls for things that can’t be done with things
like FltWriteFile, but in actuallity most of the calls you can do have
FltXXX routines.
I am not an expert in English language, but don’t you think that the
statement " Minifilter drivers should use FltPerformSynchronousIo only in
cases where routines such as the following cannot be used" means that if you
cannot use the following x no. of routines to do the task, you can make use
of FltPerformSynchronousIo (right now considering the documentation of
FltPerformSynchronousIo only).
So, if you need to do a task for which Flt is not available, you can use FltPerformSynchronousIo to do that task? For instance, if you needed to do quota manipulation and you did not find FltXXXQuotaInformationFile, you could use FltPerformSynchronousIo with Major Function = IRP_xxx_QUOTA set in the callback data.
Of course, if you meant that FltQueryQuotaInformationFile and FltSetQuotaInformationFile should be provided, it sure is needed. And just in case " I can query security but not set it" was not a typo, FltSetSecurityObject is there to set security. And again, if you meant that it is BROKEN, that’s another case,
“Ayush Gupta” wrote in message news:xxxxx@ntfsd… > Hi Don, > >>That is an interesting list, I can query security but not set it. There > are >>no calls to hanlde quotas, etc. In fact the documentation is misleading >>in > >>that is says to use these calls for things that can’t be done with things >>like FltWriteFile, but in actuallity most of the calls you can do have >>FltXXX routines. > > > I am not an expert in English language, but don’t you think that the > statement " Minifilter drivers should use FltPerformSynchronousIo only in > cases where routines such as the following cannot be used" means that if > you > cannot use the following x no. of routines to do the task, you can make > use > of FltPerformSynchronousIo (right now considering the documentation of > FltPerformSynchronousIo only). > > So, if you need to do a task for which Flt is not available, you can > use FltPerformSynchronousIo to do that task? > For instance, if you needed to do quota manipulation and you did not find > FltXXXQuotaInformationFile, you could use FltPerformSynchronousIo with > Major > Function = IRP_xxx_QUOTA set in the callback data.
My interpretation was that if there was not a FltXXX routine, then use FltPerformXXX to do it, which unfortunately is not the case. It is actually closer to if there is a FltXXX routine and it does not have all the options you want then use FltPerformXXX.
> Of course, if you meant that FltQueryQuotaInformationFile and > FltSetQuotaInformationFile should be provided, it sure is needed. > And just in case " I can query security but not set it" was not a typo, > FltSetSecurityObject is there to set security.
There is a FltSetSecurityObject FOR VISTA ONLY!!! My customers are primarily server customers and want support for Windows 2000 and Windows 2003. My one non-server customer said “I have no Vista customers or interest, but be sure to support XP!”. Given the limitations of mini-filters it is really hard to address what my customers want.
That is an interesting list, I can query security but not set it.
There are no calls to hanlde quotas, etc. In fact the documentation is
misleading in that is says to use these calls for things that can’t be
done with things like FltWriteFile, but in actuallity most of the calls
you can do have FltXXX routines.
I am not an expert in English language, but don’t you think that the
statement " Minifilter drivers should use FltPerformSynchronousIo only in
cases where routines such as the following cannot be used" means that if you
cannot use the following x no. of routines to do the task, you can make use
of FltPerformSynchronousIo (right now considering the documentation of
FltPerformSynchronousIo only).
So, if you need to do a task for which Flt is not available, you can use FltPerformSynchronousIo to do that task? For instance, if you needed to do quota manipulation and you did not find FltXXXQuotaInformationFile, you could use FltPerformSynchronousIo with Major Function = IRP_xxx_QUOTA set in the callback data.
Of course, if you meant that FltQueryQuotaInformationFile and FltSetQuotaInformationFile should be provided, it sure is needed. And just in case " I can query security but not set it" was not a typo, FltSetSecurityObject is there to set security. And again, if you meant that it is BROKEN, that’s another case,
> And again, if you meant that it is BROKEN, that’s another case,
I would argue that having broken functionality is usually *WORSE* than
not having it at all. By analogy, have you ever tried loosening a
really tight bolt with a cheap crescent wrench and rounded the head?
Sure you “can” use it on some nuts, but you’re going to get really
screwed once in a while.
I would argue that having broken functionality is usually *WORSE* than
not having it at all.
True, kinda like with *MmGetSystemRoutineAddress. *If your going to have
to check the version
of the system, you kinda already know if the function your looking for
is there or not already.
By analogy, have you ever tried loosening a
really tight bolt with a cheap crescent wrench and rounded the head?
Sure you “can” use it on some nuts, but you’re going to get really
screwed once in a while.
Hey, just use a shot gun to loosen things up like this guy did…
>There is a FltSetSecurityObject FOR VISTA ONLY!!!
Hmm… Correct. How could I forget the problem that I faced while trying to
set security and getting and error code in Windows XP…
My interpretation was that if there was not a FltXXX routine, then use
FltPerformXXX to do it, which unfortunately is not the case. It is
actually closer to if there is a FltXXX routine and it does not have all
the
options you want then use FltPerformXXX.
>> And again, if you meant that it is BROKEN, that’s another case,
I would argue that having broken functionality is usually *WORSE* than
not having it at all. By analogy, have you ever tried loosening a
really tight bolt with a cheap crescent wrench and rounded the head?
Sure you “can” use it on some nuts, but you’re going to get really
screwed once in a while.
A mini-filter can still attach and filter, just it’s context functions
will not work. To attach a stream context, or a stream handle context,
the file system must use the
The question is if stream contexts are not vital for mini-filters. You can spare it maybe in some school sample.
I’m sorry, not quite sure what your saying… I’m confused…
Even if the underlying FSD does not use FSRTL_ADVANCED_FCB_HEADER,
filter manager can and will still
attach and filter. You just lose some functionality like your context
functions for example.
>A mini-filter can still attach and filter, just it’s context functions
>will not work. To attach a stream context, or a stream handle context,
>the file system must use the
>
>
The question is if stream contexts are not vital for mini-filters. You can spare it maybe in some school sample.
-bg
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit: http://www.osr.com/seminars
