Blocking file open in filter driver

Happy new year all,

I’m new to file system drivers and I’m currently developing a W2K file
system filter driver. My previous driver writing experiences have been with
monolithic NT kernel mode PCI hardware device drivers.

The filter runs on a W2K file serving machine and gets involved when remote
clients make modifications to files on the server.

The approach that I’m taking is that when an IRP_MJ_CREATE indicates that a
file is to be opened for modification, this 'CREATE would be blocked while
the filter would communicate with a user space process that accesses the
file associated with this 'CREATE. After the user space process is done
accessing the file, the user space process notifies the filter and finally
the original 'CREATE is allowed to proceed. In other words, while the
original 'CREATE is being blocked, 'CREATEs that do not modify the file are
allowed to proceed.

Is this a reasonable approach? Will the I/O Manager or some other part of
the file system driver hierarchy disallow the 'CREATEs to be handled out of
order? Might there be reasons to push the file accesses that the user space
process will be doing down into the filter driver?

Any thoughts on this would be greatly appreciated.

Brad


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> Is this a reasonable approach? Will the I/O Manager or some other part of

the file system driver hierarchy disallow the 'CREATEs to be handled out
of
order? Might there be reasons to push the file accesses that the user
space
process will be doing down into the filter driver?

Yes, this is OK - I have the product which does similar things.

Max


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Brad,

There are cooperating Service / Driver products with the architecture you
describe, in widespread production use. Hopefully some of their engineers
will reply.

You don’t indicate whether you care about performance; nor whether your
driver will be deployed generally, on systems over whose configuration you
have no control. If either of these considerations apply, I recommend
against the architecture. Recursing through the stack of file system filter
drivers (yours won’t be the only one installed) in this way creates
performance and interoperability problems.

  1. Horrendous performance overhead. I haven’t used this architecture, but
    I have done significant kernel-mode processing while filtering Creates,
    involving nested file system calls. One implementation used Zw* calls for
    nested file operations. Performance analysis showed this to be a hot spot,
    so we re-implemented the calls by rolling our own IRPs. This eliminated
    driver stack recursion and some I/O Manager overhead, by passing the IRPs
    strictly down the stack. The performance gain was phenomenal. The
    architecture you describe imposes another quantum leap of overhead, not only
    recursing through the driver stack, but adding user-kernel-user transitions
    for every nested I/O op. Just say no.

  2. Interoperability agony. If your product will be deployed generally,
    then it must coexist with other, arbitrary file system filter drivers. This
    is a tough problem; perhaps the single greatest maintenance expense your
    product will encounter. You should select an architecture that minimizes FS
    filter interactions, where possible. The architecture you describe, on the
    contrary, maximizes those interactions.

One test for the validity of an architecture for general deployment is
whether it is self-compatible: If two products use this architecture, will
they interoperate, or will they conflict? This architecture is not
self-compatible. If you adopt it, be prepared to develop ad hoc tricks and
built-in knowledge of other third-party products. For example: Product A
blocks Creates, transitions to User Mode, where it issues a Create; A’s
driver presumably recognizes A’s own Creates, and passes them down. Layered
under A, Product B attempts to do exactly the same sequence – but A’s
driver intercepts B’s nested Create, and does not recognize it as nested.
If the script you describe is followed, then A will block B’s nested Create,
while B is already blocking A’s nested Create.

There are a number of ways to avoid the deadlock described above, and the
other deadlocks you will encounter with this architecture. Be prepared to
learn them, the hard way.

  1. False positives. You say “… when an IRP_MJ_CREATE indicates that a
    file is to be opened for modification.” Be aware, application writers are
    extremely lazy in indicating their intents. You will see an enormous number
    of Creates that specify ALL intents (e.g., read and write content and
    attributes), where the resulting file object is never used for any
    modification. You can only regard a Write intent as a tentative hint that
    writes might, maybe, perhaps occur.

  2. Blocking *is* Locking. There is a long-standing rule for file system
    filters: A driver should never hold a lock over the course of a call to the
    file system. The dirty little secret about this rule is, it is impossible
    to obey it. Generalizing a bit, a lock is a data object shared between
    threads, owned by only one at a time, that can block the continued execution
    of other threads, while one thread owns the object. IRPs are, therefore,
    locks. Like all locks, there must be an IRP (b)locking hierarchy, to
    prevent deadlock. The only hierarchy I have been able to imagine (and I am
    open to suggestions) for this case is: Pass nested IRPs strictly downward.
    I believe this protocol works to achieve driver interoperability, for
    drivers that do not filter paging I/O. This means targeting rolled-IRPs or
    FastIO calls to the driver below you, rather than using Zw* calls or
    cooperating with user mode processes.

  3. The good news is, FastFAT. The few problems listed above, and a
    mountain of others that you will encounter, have solutions that are laid out
    in the FastFAT source code you have in your IFSKit (this kit is mandatory).
    There is a long learning curve to studying this source code, but a much
    longer one without it.

Geoff

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Monday, January 01, 2001 5:37 PM
To: File Systems Developers
Subject: [ntfsd] Re: Blocking file open in filter driver

Is this a reasonable approach? Will the I/O Manager or some other part
of the file system driver hierarchy disallow the 'CREATEs to be handled
out of order? Might there be reasons to push the file accesses that the
user space process will be doing down into the filter driver?

Yes, this is OK - I have the product which does similar things.

Max

-----Original Message-----
From: Brad Sahr [mailto:xxxxx@macromedia.com]
Sent: Monday, January 01, 2001 5:51 PM
To: File Systems Developers
Subject: [ntfsd] Blocking file open in filter driver

Happy new year all,

I’m new to file system drivers and I’m currently developing a W2K file
system filter driver. My previous driver writing experiences have been
with monolithic NT kernel mode PCI hardware device drivers.

The filter runs on a W2K file serving machine and gets involved when
remote clients make modifications to files on the server.

The approach that I’m taking is that when an IRP_MJ_CREATE indicates
that a file is to be opened for modification, this 'CREATE would be blocked
while the filter would communicate with a user space process that accesses
the file associated with this 'CREATE. After the user space process is done
accessing the file, the user space process notifies the filter and finally
the original 'CREATE is allowed to proceed. In other words, while the
original 'CREATE is being blocked, 'CREATEs that do not modify the file are
allowed to proceed.

Is this a reasonable approach? Will the I/O Manager or some other part of
the file system driver hierarchy disallow the 'CREATEs to be handled out of
order? Might there be reasons to push the file accesses that the user space
process will be doing down into the filter driver?

Any thoughts on this would be greatly appreciated.

Brad


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Thank you Geoff for the well-considered reply. Here are my thoughts and
additional information about the points you brought up.

  1. performance - Our filter will be interested in a -relatively- few number
    of files within the file system, i.e. not a file backup utility. We will
    definitely keep an eye on this.

  2. interoperability - We suspect there will be issues here. The plan will be
    to minimize the filter interactions (as you suggest below) after we get the
    filter working.

  3. lazy programmers - Seen this before. There must be a few ‘bad’ apps that
    open for write unnecessarily (I know of one - RealPlayer). So the intent is
    to monitor for 'WRITE calls on file objects that have been opened for write.

As the application level open() flags for writing are mapped into 'CREATE
options, can someone tell me the correct options that indicate whether a
file can be written or not? Or must the filter inspect every 'WRITE
operation and determine if it is occurring on a file it is interested in?

  1. blocking - I understand that blocking is locking. I’m wondering how this
    can be done technically. I’m familiar with queueing 'READ and 'WRITE irps in
    monolithic kernel drivers. Can I queue a 'CREATE irp in the same way and
    return STATUS_PENDING? If I can, would I use a system thread to communicate
    with the user app and pass along the 'CREATE at the appropriate time? Could
    there be context issues associated with passing along the 'CREATE from the
    system thread?

  2. fastfat - Thanks for the tip here. I’ve been looking at filespy and
    another filter driver. Someone on list was kind enough to forward a nice
    code sample.

Thanks again for the help.

Brad

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Geoff Clow
Sent: Tuesday, January 02, 2001 10:17 AM
To: File Systems Developers
Cc: ‘xxxxx@macromedia.com’
Subject: [ntfsd] Re: Blocking file open in filter driver

Brad,

There are cooperating Service / Driver products with the architecture you
describe, in widespread production use. Hopefully some of their engineers
will reply.

You don’t indicate whether you care about performance; nor whether your
driver will be deployed generally, on systems over whose configuration you
have no control. If either of these considerations apply, I recommend
against the architecture. Recursing through the stack of file
system filter
drivers (yours won’t be the only one installed) in this way creates
performance and interoperability problems.

  1. Horrendous performance overhead. I haven’t used this
    architecture, but
    I have done significant kernel-mode processing while filtering Creates,
    involving nested file system calls. One implementation used Zw* calls for
    nested file operations. Performance analysis showed this to be a
    hot spot,
    so we re-implemented the calls by rolling our own IRPs. This eliminated
    driver stack recursion and some I/O Manager overhead, by passing the IRPs
    strictly down the stack. The performance gain was phenomenal. The
    architecture you describe imposes another quantum leap of
    overhead, not only
    recursing through the driver stack, but adding user-kernel-user
    transitions
    for every nested I/O op. Just say no.

  2. Interoperability agony. If your product will be deployed generally,
    then it must coexist with other, arbitrary file system filter
    drivers. This
    is a tough problem; perhaps the single greatest maintenance expense your
    product will encounter. You should select an architecture that
    minimizes FS
    filter interactions, where possible. The architecture you
    describe, on the
    contrary, maximizes those interactions.

One test for the validity of an architecture for general deployment is
whether it is self-compatible: If two products use this
architecture, will
they interoperate, or will they conflict? This architecture is not
self-compatible. If you adopt it, be prepared to develop ad hoc
tricks and
built-in knowledge of other third-party products. For example: Product A
blocks Creates, transitions to User Mode, where it issues a Create; A’s
driver presumably recognizes A’s own Creates, and passes them
down. Layered
under A, Product B attempts to do exactly the same sequence – but A’s
driver intercepts B’s nested Create, and does not recognize it as nested.
If the script you describe is followed, then A will block B’s
nested Create,
while B is already blocking A’s nested Create.

There are a number of ways to avoid the deadlock described above, and the
other deadlocks you will encounter with this architecture. Be prepared to
learn them, the hard way.

  1. False positives. You say “… when an IRP_MJ_CREATE indicates that a
    file is to be opened for modification.” Be aware, application writers are
    extremely lazy in indicating their intents. You will see an
    enormous number
    of Creates that specify ALL intents (e.g., read and write content and
    attributes), where the resulting file object is never used for any
    modification. You can only regard a Write intent as a tentative hint that
    writes might, maybe, perhaps occur.

  2. Blocking *is* Locking. There is a long-standing rule for file system
    filters: A driver should never hold a lock over the course of a
    call to the
    file system. The dirty little secret about this rule is, it is impossible
    to obey it. Generalizing a bit, a lock is a data object shared between
    threads, owned by only one at a time, that can block the
    continued execution
    of other threads, while one thread owns the object. IRPs are, therefore,
    locks. Like all locks, there must be an IRP (b)locking hierarchy, to
    prevent deadlock. The only hierarchy I have been able to imagine
    (and I am
    open to suggestions) for this case is: Pass nested IRPs strictly
    downward.
    I believe this protocol works to achieve driver interoperability, for
    drivers that do not filter paging I/O. This means targeting
    rolled-IRPs or
    FastIO calls to the driver below you, rather than using Zw* calls or
    cooperating with user mode processes.

  3. The good news is, FastFAT. The few problems listed above, and a
    mountain of others that you will encounter, have solutions that
    are laid out
    in the FastFAT source code you have in your IFSKit (this kit is
    mandatory).
    There is a long learning curve to studying this source code, but a much
    longer one without it.

Geoff

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Monday, January 01, 2001 5:37 PM
To: File Systems Developers
Subject: [ntfsd] Re: Blocking file open in filter driver

> Is this a reasonable approach? Will the I/O Manager or some other part
> of the file system driver hierarchy disallow the 'CREATEs to be handled
> out of order? Might there be reasons to push the file accesses that the
> user space process will be doing down into the filter driver?

Yes, this is OK - I have the product which does similar things.

Max

-----Original Message-----
From: Brad Sahr [mailto:xxxxx@macromedia.com]
Sent: Monday, January 01, 2001 5:51 PM
To: File Systems Developers
Subject: [ntfsd] Blocking file open in filter driver

Happy new year all,

I’m new to file system drivers and I’m currently developing a W2K file
system filter driver. My previous driver writing experiences have been
with monolithic NT kernel mode PCI hardware device drivers.

The filter runs on a W2K file serving machine and gets involved when
remote clients make modifications to files on the server.

The approach that I’m taking is that when an IRP_MJ_CREATE indicates
that a file is to be opened for modification, this 'CREATE would
be blocked
while the filter would communicate with a user space process that accesses
the file associated with this 'CREATE. After the user space
process is done
accessing the file, the user space process notifies the filter and finally
the original 'CREATE is allowed to proceed. In other words, while the
original 'CREATE is being blocked, 'CREATEs that do not modify
the file are
allowed to proceed.

Is this a reasonable approach? Will the I/O Manager or some other part of
the file system driver hierarchy disallow the 'CREATEs to be
handled out of
order? Might there be reasons to push the file accesses that the
user space
process will be doing down into the filter driver?

Any thoughts on this would be greatly appreciated.

Brad


You are currently subscribed to ntfsd as: xxxxx@macromedia.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Brad,

Don’t complete Creates asynchronously. It achieves nothing.
They are synchronous by definition, at the user API level. At
the kernel level, the caller is still effectively blocked, so the
synchronization and opportunity for deadlocks remain identical.

Further, it is such an unusual thing to do, some filters (and possibly
some system components) will break when they get back Pending
on a Create (they’ve never encountered it before).

-----Original Message-----
From: Brad Sahr [mailto:xxxxx@macromedia.com]
Sent: Friday, January 05, 2001 8:11 AM
To: File Systems Developers
Subject: [ntfsd] Re: Blocking file open in filter driver

  1. blocking - I understand that blocking is locking. I’m wondering how this
    can be done technically. I’m familiar with queueing 'READ and 'WRITE irps in
    monolithic kernel drivers. Can I queue a 'CREATE irp in the same way and
    return STATUS_PENDING? If I can, would I use a system thread to communicate
    with the user app and pass along the 'CREATE at the appropriate time? Could
    there be context issues associated with passing along the 'CREATE from the
    system thread?

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com