> If you want to get something stable and functional, you
should ask an engineer to analyze and give you the best solution. This is
how life work, natural and simple.
I dare to disagree. I would say it rather works like:
“Hey we don’t have budget to pay for a rookie in the field,
but I’ve heard that fltmgr is easy to use so just write
some code, kk ?”
Maybe *this* is the reason why MS does it.
L.
>Maybe *this* is the reason why MS does it.
Doubts. Experienced person with FltMgr will develop the minifilter with lesser number of bugs the he (the same person) would have with the legacy filter.
Just lesser code to type. Smaller surface for typo-style bugs and so on.
For me, raising Windows stability due to this is the 1st cause why MS is pushing minifilters.
Also note that FltMgr is a framework - like KMDF, or like MFC for UI apps. The “framework” means that you can start the project from a very simple dummy, adding the features one by one. This is not so with WDM, legacy FSF model and so on, which means - very steep learning curve of these technologies.
This is probably the 2nd cause for MS.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Here’s my contribution to the list of what’s possible with legacy file system filters but not mini-filters: running over named pipes and mail slots. We’ve brought this fact to the attention of the Microsoft file system filter group a few times over the years but to no avail. This would in fact be a simple enough addition to the Filter Manager but the volume of requests to add NPFS/MSFS has apparently not been high enough so far.
Ilya Faenson
Rockville, MD USA
Thank you for your feedback.
I hear you. This unfortunately did not make Windows 7.
NPFS and Mailslots do not register as file systems and do not support
the advanced FCB header. The changes to get fltmgr working seamlessly
over NPFS and Mailslots are complex and extensive.
This is on top of our TODO list for Windows 8.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
xxxxx@hotmail.com wrote:
Here’s my contribution to the list of what’s possible with legacy file system filters but not mini-filters: running over named pipes and mail slots. We’ve brought this fact to the attention of the Microsoft file system filter group a few times over the years but to no avail. This would in fact be a simple enough addition to the Filter Manager but the volume of requests to add NPFS/MSFS has apparently not been high enough so far.
Ilya Faenson
Rockville, MD USA
Sarosh,
The biggest thing I see, as Tony stated, is simply if you make legacy
filters uninstallable at some point in the future, developers will find
a way to get them to run. This will introduce more instabilities in the
platform, regardless of how the ‘blocking’ is implemented.
Pete
Sarosh Havewala wrote:
Thank you for your feedback.
I hear you. This unfortunately did not make Windows 7.
NPFS and Mailslots do not register as file systems and do not support
the advanced FCB header. The changes to get fltmgr working seamlessly
over NPFS and Mailslots are complex and extensive.This is on top of our TODO list for Windows 8.
Regards,
Sarosh.
File System Filter Lead
Microsoft CorpThis posting is provided “AS IS” with no warranties, and confers no Rights
xxxxx@hotmail.com wrote:
> Here’s my contribution to the list of what’s possible with legacy file
> system filters but not mini-filters: running over named pipes and mail
> slots. We’ve brought this fact to the attention of the Microsoft file
> system filter group a few times over the years but to no avail. This
> would in fact be a simple enough addition to the Filter Manager but
> the volume of requests to add NPFS/MSFS has apparently not been high
> enough so far.
>
> Ilya Faenson
> Rockville, MD USA
>
>
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsYou are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
Tony - I have been following this conversation for a bit and there are some misconceptions I’d like to clear up.
Firstly - we have never encouraged the notion that minifilters are a panacea to all filter development complexities, nor that folks can now go and write them willy nilly. We keep a watch internally as well to first dissuade anybody from attempting a filter unless there is really no way around, and if it really has commensurate business benefits.
I have personally been successful in choosing alternate paths to writing file system filters in a number of cases.
However what mini-filters do is that make some problems that make a developer skilling developing file system filters go away. They are analogous what WDF is for drivers (minifilters preceded WDF), where the objective is to not encourage folks to develop more filters, but to make filter authoring free of certain complexities which could be resolved by a framework. Specifically these are:
However, filter manager does not help with cache manager/memory manager complexities - as they are part of the FS design complexity in Windows. We do have some longer term plans to simplify, but they are not appropriate to discuss here.
While I understand the resistance from the more experienced filter devs - such as you, Maxim - who understand all the details of the OS/FS complexity and can work around them using legacy filters, the majority of the filter developers - even before filter manager appeared on the scene, didn’t like the complexity and found a lot of the interfacing as a legacy filter quite unintuitive. There is no question, that we eliminated a whole bunch of standard issues that folks ran repeatedly into with filter manager and a majority of filters fit that pattern (in terms of numbers developed)
However there are always a somewhat complex, domain specific filter or one that needs to have intricate knowledge of the internals of the OS (which may not be documented) and you are willing to put up with the cost that brings with that. There are folks who still hook NT APIs or dig into private kernel structures, but that comes with a cost, and a support overhead that is significantly higher. Our goal is certainly to constantly clean up interactions which are problematic and make them more transparent if they are relevant for filters. I think you should send the specifics of what your requirements are and why you need that to Sarosh (unless you have already done that).
Thanks!
Ravi
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Monday, January 19, 2009 9:05 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Forcing mini-filters (since this isn’t Win7 WDK anymore)
Indeed Max, my point was that it seems difficult to envision how one
would enforce the “no legacy filters except the one written by us”
model. Sarosh’s comment suggests that no mechanism has been decided
yet.
The filter we implemented recently needed to sit between MUP and RDR on
Vista. Since filter manager sits above MUP, a mini-filter cannot be
used to observe the interactions between MUP and RDR. A legacy filter
can be used to do that (ironically, it must also sit on top of MUP, a
trick we’ve used for 10+ years to discover network redirectors that
don’t register as file systems. But that’s a detail here.)
I suppose the “out” here is that one can argue about the definition of a
file system. In Vista, since RDR doesn’t register, we can argue that it
isn’t a file system, while because MUP registers it IS a file system.
If we restrict the definition in this way it could be policed in the
attachment code (“if this is a registered file system driver, only allow
the attach if it is filter manager.”) My concern is that this will
encourage people to use hacks to implement their functionality - they
certainly do that in other areas when an interface is unavailable.
Don’t get me wrong - I would suggest to ANYONE that is starting out
building a filter driver that they do so as a mini-filter unless there
is a compelling technical reason why it cannot be done within the
mini-filter framework. Compelling technical reasons include: supporting
legacy systems where filter manager does not exist and filtering
components that filter manager does not filter. The problem is that
while some of us have the technical expertise to choose the right path,
others joining the process now may not. Thus, the goal would be to
force those without that technical expertise to “choose the right path”
and to forcibly remove the option of doing things “correctly” for those
that do have the technical expertise.
For those that believe mini-filters are a panacea to filtering problems,
rest assured they are not. My observation has been that filter manager
makes building filters look deceptively easy, without actually making
the “hard part” of filtering (understanding file system behavior) any
easier. It has encouraged a greater number of people to build filter
drivers. Whether or not this equates to fewer interop issues is not
clear - perhaps fewer per filter but offset by more filters?
Tony
OSR
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
>While I understand the resistance from the more experienced filter devs - such as you, Maxim
I would be glad very much to switch to minifilter model (and to reduce the support overhead, interop issues especially) - but I must be sure that FltMgr will support the features I will need.
For now, it looks like so for me.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Ravi,
I’d like to clear up some misconceptions as well:
My concern is that, if you prohibit a specific class of drivers in a
systematic fashion you will create a de-facto hole that prohibits those
of us with sufficient experience to implement clean solutions for our
customers - I haven’t been hearing folks at Microsoft complaining about
our own legacy filter framework, so I assume we’ve not been a source of
problems such as those you mention. Even now, we’re forced to support
legacy and mini-filters for our Data Modification Kit because of the
demands of the market.
In an article I recently wrote I observed that much of the friction here
is that those of us writing drivers in general make our money by selling
product across the entire platform base. As long as there are people
willing to spend enough incremental money for Windows 2000 support, we
will be supporting Windows 2000. Microsoft, on the other hand, makes
nothing now for a copy of Windows 2000 that you sold back in 1999 (10
years ago) and instead you make money by selling new OS versions.
Hence, there is a strong incentive for you to force people to move
forward, while there is a strong incentive for those of us building
commercial products to support the broadest range of viable markets.
It’s inevitable that there will be friction here. All I’m hoping is
that we can find some way of not doubling our work load again simply to
push forward your own agenda. But, hey, ultimately it IS your OS and
we’ll have to live with your decisions.
One point you make is a very good one: the quality of the samples must
be above par. They must be BETTER than production quality code because
they will be duplicated forever. They should be self-contained - clear,
lucid comments explaining why specific things are done in a specific
way. They require more proactive testing and code review than
production code (even filter manager) since they are not “just”
examples, but rather they are the base for hundreds of versions of that
same code. ANY bug in the code, will very likely show up in each of
those products derived from the sample.
Personally, I’m very glad to see someone from Microsoft not only
realizing this but stating it in a public forum. It makes me hopeful
that the quality of the samples will be incredibly high and that you
will continue to push an agenda of ensuring that they are truly the best
in terms of quality.
Tony
OSR
Like everyone else, I can’t claim to be objective here or in any other
circumstance, but as someone who knows very little about FS/FSF’s, I can
approximate someone who has no great interest either way, and at the
moment, I have perfected being someone that is completely confused.
That is:
it sure seems like examples have been provided of when a legacy filter
might be necessary. I don’t know if they are correct or, only that they
have been mentioned a few times, by at least Tony, and unless I’m
missing something there has been no comment on them either way, really.
I saw items 1-7 about the benefits of the minifilter model; I’ve not
yet seen a list of cons, and I’m curious as to what those might be,
because everything has some.
in particular, I think it would be very helpful if we (that is, people
who know what they are talking about, which most definitely does not
include me on this topic) could come up with some sort of list of what
will be problematic or impossible without legacy filters that everyone
at least agrees to in principle, if not magnitude
also, understanding what exactly ‘prohibited’ means would be useful as
well. I know it’s already been stated that the details of
implementation would not be provided, but it would be helpful to at
least narrow it down to ‘actively blocked,’ ‘no whql (if that even makes
sense here),’ et. c., because the thing is, isn’t filter manager a
legacy fsf? So apparently that will be able to load, assuming that I
have it correct and that it doesn’t change. It would appear to be a bit
of a double standard, especially since people will have to pay to port
there stuff, the sort that is perhaps not the wisest thing to do from a
DOJ point of view, in my opinion, though I really don’t care either way
on this issue specifically, but I would like understand the situation a
little better, if that’s possible.
what are the platform requirements for the FSF model? That is, is it
supported on W2K at all level, SP4 Rollup, et. c.? I don’t know what
it’s like for the FS crowd, but for many of us in other areas, although
I certainly understand why msft wants to move forward and have no issue
with that in and of itself, the thing is that W2K is more important for
many of us than Vista.
I don’t hear anyone disagreeing that the minifilter model is easier
for beginners, but I still see two issues. The first is that it seems
to me that no matter how obliquely one approaches it, anyone who already
owns a fsf that has to be ported to the new model almost certainly will
incur new bugs in the process, so I don’t really see the how immediately
this will help, unless the point is for them not to port in some cases,
which might not be a bad thing. I mean, it’s easier, ok, but assuming
that you have something that is already working pretty well, there
really isn’t anywhere to go but down, no matter how easy the port is,
especially when this is pure cost to developer, that he or she was
presumably not planning on, and even knowing it, it’s still pure cost
which can probably be billed to a client in many cases, but that doesn’t
mean that they will like it, or that people happen to have spare fs devs
sitting around with a bunch of time on their hands.
just as general comment, haven’t we been down this general road
before? That is, ‘this will be all you need, we’ll maybe not, but now
that’s all you have, but don’t worry, because we’re developing something
new that will do everything you need, and will actually even sound the
same with similar naming conventions, so the documentation will look the
same.’ For example, I don’t claim to know anything about the various
storage driver models, but they sure look to be a result of
outmoding/replacement/redesign, yet they are all still with us, which I
think is the troubling part, because it would seem to imply that they
all are necessary, despite sounding like they do they same thing. As a
general phenomenon, it’s also what makes the documentation pretty much
incomprehensible unless you really work at it. Don’t get me wrong; this
situation is clearly different, and I don’t hear anyone saying that the
minifilter model is not preferable in most if not almost all cases, but
I guess what I’m saying is that, in my opinion, policies like this
almost never work all that well, it’s all or nothing this time, and
that I don’t hear any of the downside being detailed makes me nervous.
I mean, it is what is, but it would be nice to know it ahead of time,
even for someone like me whom it will probably not affect, but still
wants to understand the environment in which decisions like this are
being made.
I don’t know whether people are resistant to moving to the minifilter
model or not, but what I do think I know is that they are resistant to
be forced to do so, especially if they have to rewrite something that
already exists, and either way, maybe I’m missing something or just not
understanding it all, but I think that there has been no fleshing out of
the downside to this policy is what’s making everyone particularly
apprehensive.
Thanks,
mm
Tony Mason wrote:
Ravi,
I’d like to clear up some misconceptions as well:
- First, that I’m resistant to building mini-filters. In fact, I wrote
my first mini-filter six years ago and I routinely encourage people to
go that route if a filter is appropriate for their problem. I’ve
developed a class on mini-filter development and I’ve written various
articles on it. I am resistant to being told that Microsoft will
PROHIBIT a class of solutions which, even you admit, may be necessary in
specific circumstances.- I have actually provided very specific concrete examples in the past.
I’m now wondering if they have fallen on deaf ears, or if perhaps the
message has been drowned out by some internal Microsoft pressures to
force a mini-filter only model. So let me list a few: (1) a common
problem for network redirectors is that any time their network path name
is passed to CIFS, it can impose a 30 second timeout delay. In the past
we were able to deal with this via correct ordering, but in newer OS
versions this does not work because the algorithms have changed. An
unusual, but valid solution for this is to filter CIFS (SMBMRX, RDR,
whatever you’d like to call it these days) and examine the names. Since
this is a MUP-to-redirector protocol, it cannot be filtered via a
mini-filter. Truthfully, the same thing can happen any time you want to
perform redirector specific filtering (I *only* want to filter DAV. But
I can’t do that easily because to determine what redirector will handle
the request I have to open the object and ask. That’s a lot of overhead
if all I’m going to do is then reparse it to something local, for
example, or my goal is to eliminate a long timeout for an invalid name.)
(2) DFS is a constant thorn for us. For example, we have a persistent
bug that shows up - but ONLY when a mini-filter is installed - based
upon the DFS name that is passed. This issue shows up over and over
again at customer sites. Right now, we’re dealing with a case in which a
specific sequence of operations between DFS, MUP and RDR trigger
incorrect behavior. While we’re working to more succinctly categorize
this, at this point anything we find will be a Windows 7 SP1 fix (at
best) and will require that we fix it - which, amazingly enough looks
like we need to sit between DFS and RDR. Mini-filters sit on MUP now,
not on DFS. In this case we expect we’ll need to make it work using an
RDR specific filter. The NPFS/MSFS example has been raised before, and
I personally find it minimally compelling.My concern is that, if you prohibit a specific class of drivers in a
systematic fashion you will create a de-facto hole that prohibits those
of us with sufficient experience to implement clean solutions for our
customers - I haven’t been hearing folks at Microsoft complaining about
our own legacy filter framework, so I assume we’ve not been a source of
problems such as those you mention. Even now, we’re forced to support
legacy and mini-filters for our Data Modification Kit because of the
demands of the market.In an article I recently wrote I observed that much of the friction here
is that those of us writing drivers in general make our money by selling
product across the entire platform base. As long as there are people
willing to spend enough incremental money for Windows 2000 support, we
will be supporting Windows 2000. Microsoft, on the other hand, makes
nothing now for a copy of Windows 2000 that you sold back in 1999 (10
years ago) and instead you make money by selling new OS versions.
Hence, there is a strong incentive for you to force people to move
forward, while there is a strong incentive for those of us building
commercial products to support the broadest range of viable markets.
It’s inevitable that there will be friction here. All I’m hoping is
that we can find some way of not doubling our work load again simply to
push forward your own agenda. But, hey, ultimately it IS your OS and
we’ll have to live with your decisions.One point you make is a very good one: the quality of the samples must
be above par. They must be BETTER than production quality code because
they will be duplicated forever. They should be self-contained - clear,
lucid comments explaining why specific things are done in a specific
way. They require more proactive testing and code review than
production code (even filter manager) since they are not “just”
examples, but rather they are the base for hundreds of versions of that
same code. ANY bug in the code, will very likely show up in each of
those products derived from the sample.Personally, I’m very glad to see someone from Microsoft not only
realizing this but stating it in a public forum. It makes me hopeful
that the quality of the samples will be incredibly high and that you
will continue to push an agenda of ensuring that they are truly the best
in terms of quality.Tony
OSR
Thanks Tony, this cleared up things considerably for me.
I’m a strong advocate of providing filtering support for NPFS and MSFF (Sarosh knows about it), however by the time it became a very strong requirement, we were smack in the middle of win 7, and it was impossible to reshuffle resources. All of us here regret that we couldn’t add this for Win 7, and this is very high on the priority list for the next release. I have a lot of hope that it will make it.
Regarding DFS, MUP, RDR: there were things that the new design solved for the simpler filters - i.e. not having to worry about individual redirectors, knowing by name and attaching etc. - but I hear you loud and clear about certain filters wanting to specifically filter LanMan. I will discuss this with Sarosh, I believe we already have an email regarding your request. We are certainly lacking in good samples for redirector/lanman filtering - and we certainly need to fill the gap as soon as possible.
Ravi
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Wednesday, January 21, 2009 1:03 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Forcing mini-filters (since this isn’t Win7 WDK anymore)
Ravi,
I’d like to clear up some misconceptions as well:
My concern is that, if you prohibit a specific class of drivers in a
systematic fashion you will create a de-facto hole that prohibits those
of us with sufficient experience to implement clean solutions for our
customers - I haven’t been hearing folks at Microsoft complaining about
our own legacy filter framework, so I assume we’ve not been a source of
problems such as those you mention. Even now, we’re forced to support
legacy and mini-filters for our Data Modification Kit because of the
demands of the market.
In an article I recently wrote I observed that much of the friction here
is that those of us writing drivers in general make our money by selling
product across the entire platform base. As long as there are people
willing to spend enough incremental money for Windows 2000 support, we
will be supporting Windows 2000. Microsoft, on the other hand, makes
nothing now for a copy of Windows 2000 that you sold back in 1999 (10
years ago) and instead you make money by selling new OS versions.
Hence, there is a strong incentive for you to force people to move
forward, while there is a strong incentive for those of us building
commercial products to support the broadest range of viable markets.
It’s inevitable that there will be friction here. All I’m hoping is
that we can find some way of not doubling our work load again simply to
push forward your own agenda. But, hey, ultimately it IS your OS and
we’ll have to live with your decisions.
One point you make is a very good one: the quality of the samples must
be above par. They must be BETTER than production quality code because
they will be duplicated forever. They should be self-contained - clear,
lucid comments explaining why specific things are done in a specific
way. They require more proactive testing and code review than
production code (even filter manager) since they are not “just”
examples, but rather they are the base for hundreds of versions of that
same code. ANY bug in the code, will very likely show up in each of
those products derived from the sample.
Personally, I’m very glad to see someone from Microsoft not only
realizing this but stating it in a public forum. It makes me hopeful
that the quality of the samples will be incredibly high and that you
will continue to push an agenda of ensuring that they are truly the best
in terms of quality.
Tony
OSR
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Thanks Maxim. I appreciate the great service you are doing to the filter community in general on this list!
Ravi
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Wednesday, January 21, 2009 12:48 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Forcing mini-filters (since this isn’t Win7 WDK anymore)
While I understand the resistance from the more experienced filter devs - such as you, Maxim
I would be glad very much to switch to minifilter model (and to reduce the support overhead, interop issues especially) - but I must be sure that FltMgr will support the features I will need.
For now, it looks like so for me.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com