Using FileObject->FileName in filter driver

Hi everyone.

Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.

So, if someone below my filter (another filter or FSD itself) has changed
FileName during IRP_MJ_CREATE I will not touch it (1-3)
If someone has changed FileName afterwards I will detect it by checking the
signature and will use some “conventional” mechanism to get
associated data.

To me these sound reasonable. How about you? :slight_smile:

Any feedback is highly welcome!

Vladimir


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

No.

If 1-3 will be required to work, simply use your normal mechanism.
Trying to squeeze into cracks like this will generate problems not just
with drivers which may, erroneously as you observe, sniff the filename
field, but with the OS if we try to make the rule more obvious in the
future by freeing the buffer and/or using the verifier to catch folks
peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any
guarantee here as to how it would/could work in the future. As a
starting point, one could expect the IO manager to do it (if it ever
does) in IopParseDevice, after the create is complete. This would be
after your completion routine runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to
utilize it to store my filter’s private information associated with this
FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all
    FileName’s structure fields.
  2. After underlying FSD has successfully completed the IRP I compare
    current FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think
    that I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s
    private data in string format (including a signature at the beginning of
    the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature
    and if it matches what I’m expecting I will parse that string and
    extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has
    changed FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking
    the signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Using FileObject->FileName in filter driverIn General using Someone Structure is not Advisable. May be this version it will work In future version of SP may not.

Regards,
Satish K.S

Hi everyone.

Can I store my filter’s private data in FileObject->FileName field after file gets opened (i.e. Create to the underlying FSD has been completed)?

Since FileName field is no longer “valid” at this point I’d like to utilize it to store my filter’s private information associated with this FileObject.

What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s structure fields.

  2. After underlying FSD has successfully completed the IRP I compare current FileName fields with those that I have saved at step 1.

  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that I can reuse it and deallocate FileName.Buffer (if it has not been

“zeroed” yet), prepare a UNICODE string that represents my filter’s private data in string format (including a signature at the beginning of the string)

and then save this string in FileName.
4. When filtering subsequent requests I check FileName for the signature and if it matches what I’m expecting I will parse that string and extract

whatever I have saved during Create.

So, if someone below my filter (another filter or FSD itself) has changed FileName during IRP_MJ_CREATE I will not touch it (1-3)

If someone has changed FileName afterwards I will detect it by checking the signature and will use some “conventional” mechanism to get

associated data.

To me these sound reasonable. How about you? :slight_smile:

Any feedback is highly welcome!

Vladimir


You are currently subscribed to ntfsd as: xxxxx@aalayance.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Vladimir,

Since you are looking at rather ugly techniques anyway, another thing to
consider is that you can also create a second file object instead. Thus,
when the IRP_MJ_CREATE comes into your filter, you create a second file
object and pass that second file object into the underlying file system.
Then, when the IRP operation arrives into your filter YOU own the file
object. Follow it, find the second file object, and set up the subsequent
call to use the second file object.

There is a problem with this approach: there are six fast I/O calls that get
passed AROUND a filter - and thus your file object is passed to the
underlying file system (there is now a callback you can register in Windows
XP, but I’m not sure it would allow you to substitute the file object.) The
best solution of which I’m aware to this problem is to use TWO device
objects for your filter (one that is “attached” and the other that is “not
attached”.) Fix up your file objects so that they point at the “not
attached” device. Then when subsequent calls arrive, they will come in on
the “not attached” device - and you won’t get skipped for those six fast I/O
calls.

Yes, this is an ugly approach, but it is more savory than utilizing the file
name field (I’m SURE that’s going to break something else - like another
filter on top of yours!)

And while I know it doesn’t solve the problem for you now (in NT 4.0 or
Windows 2000,) in Windows XP there will be a whole context-tracking
mechanism supported by the FsRtl package specifically to allow file system
filter drivers to associate context with the file object.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:
> 10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):
1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
structure fields.
2. After underlying FSD has successfully completed the IRP I compare current
FileName fields with those that I have saved at step 1.
3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
I can reuse it and deallocate FileName.Buffer (if it has not been
“zeroed” yet), prepare a UNICODE string that represents my filter’s private
data in string format (including a signature at the beginning of the string)
and then save this string in FileName.
4. When filtering subsequent requests I check FileName for the signature and
if it matches what I’m expecting I will parse that string and extract
whatever I have saved during Create.
So, if someone below my filter (another filter or FSD itself) has changed
FileName during IRP_MJ_CREATE I will not touch it (1-3)
If someone has changed FileName afterwards I will detect it by checking the
signature and will use some “conventional” mechanism to get
associated data.
To me these sound reasonable. How about you? :slight_smile:
Any feedback is highly welcome!
Vladimir


You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com</http:>

Tony:

Thank you very much! The technique you just described is interesting to me
aside from the original question. One of the headaches that I have is that
I need to perform a lot of actions (and waste a lot of time) during
IRP_MJ_CREATE
handling to enable further read-write handlers. But most of the time (and
that’s
where I have the biggest lost in overall performance) those creates are
followed
by QueryInformation to get attributes, times and size of the file in
question
and then goes CleanUp. So, file content has not been touched and all my
preparations
were useless. So, what I thought is to “fake” a successful create in my
filter and
don’t pass the original Create down to the underlying FSD until first read,
write or
EA or whatever but query attributes are detected on that FileObject. What I
was afraid
of is that I have to fully initialize FileObject so it looks like opened.
I.e. I have
to prepare FCB and CCB, save VPB (or fake it too by zeroing VPB and setting
FileObject->DeviceObject pointing to my device), etc. Then, when lets say
Read IRP is
detected on that FileObject I need to “uninitialize” it (and I’m not sure
that this is
safe because some of the fields may be cached somevhere in the OS or file is
currently
in use in another thread or you name it). So I need to uninit that FO, roll
new Create
IRP with params saved for the original one and pass it down to underlying
FS. So, because
I was not certain that uninitializing is safe (in fact, I was sure that it
is unsafe) I
stopped thinking about that approach. With your technique that utilizes two
separate FOs
I’m starting to see some light at the end of the tunnel :slight_smile: However, I’m not
clear how can
I create my own FileObject? I didn’t see any DDK functions allowing to roll
my own FO.
Second thing is that I’m not clear how I’m going to keep consistency between
Section Object
Pointers in those two separate FOs (or should I have them in the “fake” FO
and have the
second one non-cached)? Can you give more details on the approach that you
have described
before I jump into it?

Thanks in advance,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 09, 2001 8:55 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Vladimir,

Since you are looking at rather ugly techniques anyway, another thing to
consider is that you can also create a second file object instead. Thus,
when the IRP_MJ_CREATE comes into your filter, you create a second file
object and pass that second file object into the underlying file system.
Then, when the IRP operation arrives into your filter YOU own the file
object. Follow it, find the second file object, and set up the subsequent
call to use the second file object.

There is a problem with this approach: there are six fast I/O calls that get
passed AROUND a filter - and thus your file object is passed to the
underlying file system (there is now a callback you can register in Windows
XP, but I’m not sure it would allow you to substitute the file object.) The
best solution of which I’m aware to this problem is to use TWO device
objects for your filter (one that is “attached” and the other that is “not
attached”.) Fix up your file objects so that they point at the “not
attached” device. Then when subsequent calls arrive, they will come in on
the “not attached” device - and you won’t get skipped for those six fast I/O
calls.

Yes, this is an ugly approach, but it is more savory than utilizing the file
name field (I’m SURE that’s going to break something else - like another
filter on top of yours!)

And while I know it doesn’t solve the problem for you now (in NT 4.0 or
Windows 2000,) in Windows XP there will be a whole context-tracking
mechanism supported by the FsRtl package specifically to allow file system
filter drivers to associate context with the file object.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:
> 10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):
1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
structure fields.
2. After underlying FSD has successfully completed the IRP I compare current
FileName fields with those that I have saved at step 1.
3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
I can reuse it and deallocate FileName.Buffer (if it has not been
“zeroed” yet), prepare a UNICODE string that represents my filter’s private
data in string format (including a signature at the beginning of the string)
and then save this string in FileName.
4. When filtering subsequent requests I check FileName for the signature and
if it matches what I’m expecting I will parse that string and extract
whatever I have saved during Create.
So, if someone below my filter (another filter or FSD itself) has changed
FileName during IRP_MJ_CREATE I will not touch it (1-3)
If someone has changed FileName afterwards I will detect it by checking the
signature and will use some “conventional” mechanism to get
associated data.
To me these sound reasonable. How about you? :slight_smile:
Any feedback is highly welcome!
Vladimir


You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com</http:>

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I think Joel says it better than I could. There are plenty of ways to
get better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I’m trying to get across is that this is not your data:
do not touch it. This is how we run into really nasty problems. Just
because the data should not be used after create does not mean it should
be reused by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s
wrong with the tried and true mechanisms of maintaining a volume
relative balanced tree or hash table mapping file objects to whatever
data you need to associate them? I mean, is log
(number_of_file_objects) to near constant time (avg) for lookups really
that bad? Unless you have a large and experienced testing department or
Tony Mason-ish brainpower, don’t try to be too clever (Been there done
that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no
problems with implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is
reasonable approach). And nobody use it because docs say “don’t rely on
it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%.
I’m simply trying to optimize my driver’s performance and one of the
bottlenecks
that I’ve found is searching for my FileObject’s descriptor in
each and every IRP and Fast I/O request. On heavy loads (like building
big projects: > 10K
files) my table consists of hundreds (if not thousands) of
descriptors and browsing through it can be costly. Of course, if
performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m
not even sure if I will use it in any case. I just wanted to see some
implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal
mechanism. Trying to squeeze into cracks like this will generate
problems not just with drivers which may, erroneously as you observe,
sniff the filename field, but with the OS if we try to make the rule
more obvious in the future by freeing the buffer and/or using the
verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing
when the OS could decide to free it and I definitely am not going to
give you any guarantee here as to how it would/could work in the future.
As a starting point, one could expect the IO manager to do it (if it
ever does) in IopParseDevice, after the create is complete. This would
be after your completion routine runs.

The basic rule here is that you should never reuse
fields of internal structures, ever, unless they are documented that
way.

-----Original Message-----
From: Chtchetkine, Vladimir
[mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter
driver

Hi everyone.
Can I store my filter’s private data in
FileObject->FileName field after file gets opened (i.e. Create to the
underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point
I’d like to utilize it to store my filter’s private information
associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I
    save all FileName’s structure fields.
  2. After underlying FSD has successfully completed the
    IRP I compare current FileName fields with those that I have saved at
    step 1.
  3. If they are equal or FileName has been “zeroed” (i.e.
    freed) I think that I can reuse it and deallocate FileName.Buffer (if it
    has not been
    “zeroed” yet), prepare a UNICODE string that represents
    my filter’s private data in string format (including a signature at the
    beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName
    for the signature and if it matches what I’m expecting I will parse that
    string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD
    itself) has changed FileName during IRP_MJ_CREATE I will not touch it
    (1-3)
    If someone has changed FileName afterwards I will detect
    it by checking the signature and will use some “conventional” mechanism
    to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as:
xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as:
xxxxx@Starbase.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to me)
it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the key
uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to get
better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I?m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Here’s what’s used in ‘filemon’:

//
// Number of hash buckets in the hash table
//
#define NUMHASH 0x100

//
// Hash function. Basically chops the low few bits of the file object
//
#define HASHOBJECT(_fileobject) (((ULONG)_fileobject)>>5)%NUMHASH
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 6:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to me)
it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the key
uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to get
better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I?m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

If I’m correct, hashing via the FileObject is a bad thing to do. At the
November IFS plugfest, David Goebel said in his talk that it’s better to
hash FileObject->FsContext.

Be aware that the FsContext is only valid after the lower-lever drivers
have processed IRP_MJ_CREATE. So you can only use it in the completion routine.

Just my $0.02

Bartjan.

At 08:53 AM 5/10/01 -0400, you wrote:

“urn:schemas-microsoft-com:office:office” xmlns:w =
“urn:schemas-microsoft-com:office:word” xmlns:st1 =
“urn:schemas-microsoft-com:office:smarttags”>
Here’s what’s used in ‘filemon’:

//
// Number of hash buckets in the hash table
//
#define NUMHASH 0x100

//
// Hash function. Basically chops the low few bits of the file object
//
#define HASHOBJECT(_fileobject) (((ULONG)_fileobject)>>5)%NUMHASH
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 6:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to
me) it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the
key uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to get
better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I’m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s
wrong with the tried and true mechanisms of maintaining a volume relative
balanced tree or hash table mapping file objects to whatever data you need
to associate them? I mean, is log (number_of_file_objects) to near
constant time (avg) for lookups really that bad? Unless you have a large
and experienced testing department or Tony Mason-ish brainpower, don’t try
to be too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big
projects: > 10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even
sure if I will use it in any case. I just wanted to see some implications
that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal mechanism. Trying
to squeeze into cracks like this will generate problems not just with
drivers which may, erroneously as you observe, sniff the filename field,
but with the OS if we try to make the rule more obvious in the future by
freeing the buffer and/or using the verifier to catch folks peeking at it,
etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.

Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to
utilize it to store my filter’s private information associated with this
FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare
    current FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think
    that I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s
    private data in string format (including a signature at the beginning of
    the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature
    and if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.

So, if someone below my filter (another filter or FSD itself) has changed
FileName during IRP_MJ_CREATE I will not touch it (1-3)
If someone has changed FileName afterwards I will detect it by checking
the signature and will use some “conventional” mechanism to get
associated data.

To me these sound reasonable. How about you? :slight_smile:

Any feedback is highly welcome!

Vladimir


You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@zeelandnet.nl
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Not sure if this helps, but I use a splay tree to track file objects and add
context. On dispatch entry, I look up the file object in the play tree.
There is a structure I have associated with the entry:

typedef struct _FILE_CONTEXT_RECORD
{
PFILE_OBJECT FileObject;
PVOID FileContext;
} FILE_CONTEXT_RECORD, *PFILE_CONTEXT_RECORD;

Splay tree are quite fast and reorder the list using a MRU algorithm.

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Wednesday, May 09, 2001 8:55 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Vladimir,

Since you are looking at rather ugly techniques anyway, another thing to
consider is that you can also create a second file object instead. Thus,
when the IRP_MJ_CREATE comes into your filter, you create a second file
object and pass that second file object into the underlying file system.
Then, when the IRP operation arrives into your filter YOU own the file
object. Follow it, find the second file object, and set up the subsequent
call to use the second file object.

There is a problem with this approach: there are six fast I/O
calls that get
passed AROUND a filter - and thus your file object is passed to the
underlying file system (there is now a callback you can register
in Windows
XP, but I’m not sure it would allow you to substitute the file
object.) The
best solution of which I’m aware to this problem is to use TWO device
objects for your filter (one that is “attached” and the other that is “not
attached”.) Fix up your file objects so that they point at the “not
attached” device. Then when subsequent calls arrive, they will come in on
the “not attached” device - and you won’t get skipped for those
six fast I/O
calls.

Yes, this is an ugly approach, but it is more savory than
utilizing the file
name field (I’m SURE that’s going to break something else - like another
filter on top of yours!)

And while I know it doesn’t solve the problem for you now (in NT 4.0 or
Windows 2000,) in Windows XP there will be a whole context-tracking
mechanism supported by the FsRtl package specifically to allow file system
filter drivers to associate context with the file object.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:
> -----Original Message-----
> From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
> Sent: Wednesday, May 09, 2001 10:33 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
>
>
> Daniel:
>
> Thank you for the reply. So, as I understood, there are no problems with
> implementing that mechanism. At least on W2K SP1 - NTFS
> FileName is not freed until FileObject gets closed (which is reasonable
> approach). And nobody use it because docs say “don’t rely on it”.
> What a waste of space! :slight_smile:
> I understand all that you’ve said and I’m agreed with you 100%. I’m simply
> trying to optimize my driver’s performance and one of the bottlenecks
> that I’ve found is searching for my FileObject’s descriptor in each and
> every IRP and Fast I/O request. On heavy loads (like building big
> projects:
> > 10K
> files) my table consists of hundreds (if not thousands) of descriptors and
> browsing through it can be costly. Of course, if performance benefit will
> be insignificant I’m not going to use that “dirty trick”. I’m not
> even sure
> if I will use it in any case. I just wanted to see some
> implications that I
> might oversee when I originally thought about that mechanism.
>
> Best regards,
>
> Vladimir
>
> -----Original Message-----
> From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
> Sent: Tuesday, May 08, 2001 6:47 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
>
>
> No.
>
> If 1-3 will be required to work, simply use your normal
> mechanism. Trying to
> squeeze into cracks like this will generate problems not just with drivers
> which may, erroneously as you observe, sniff the filename field, but with
> the OS if we try to make the rule more obvious in the future by
> freeing the
> buffer and/or using the verifier to catch folks peeking at it, etc.
>
> With regard to 3, you really have no way of guessing when the OS could
> decide to free it and I definitely am not going to give you any guarantee
> here as to how it would/could work in the future. As a starting point, one
> could expect the IO manager to do it (if it ever does) in IopParseDevice,
> after the create is complete. This would be after your completion routine
> runs.
>
> The basic rule here is that you should never reuse fields of internal
> structures, ever, unless they are documented that way.
>
> -----Original Message-----
> From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
> Sent: Tuesday, May 08, 2001 4:36 PM
> To: File Systems Developers
> Subject: [ntfsd] Using FileObject->FileName in filter driver
>
> Hi everyone.
> Can I store my filter’s private data in FileObject->FileName field after
> file gets opened (i.e. Create to the underlying FSD has been completed)?
> Since FileName field is no longer “valid” at this point I’d like
> to utilize
> it to store my filter’s private information associated with this
> FileObject.
> What I’m thinking is (kinda algorithm):
> 1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
> structure fields.
> 2. After underlying FSD has successfully completed the IRP I
> compare current
> FileName fields with those that I have saved at step 1.
> 3. If they are equal or FileName has been “zeroed” (i.e. freed) I
> think that
> I can reuse it and deallocate FileName.Buffer (if it has not been
> “zeroed” yet), prepare a UNICODE string that represents my
> filter’s private
> data in string format (including a signature at the beginning of
> the string)
> and then save this string in FileName.
> 4. When filtering subsequent requests I check FileName for the
> signature and
> if it matches what I’m expecting I will parse that string and extract
> whatever I have saved during Create.
> So, if someone below my filter (another filter or FSD itself) has changed
> FileName during IRP_MJ_CREATE I will not touch it (1-3)
> If someone has changed FileName afterwards I will detect it by
> checking the
> signature and will use some “conventional” mechanism to get
> associated data.
> To me these sound reasonable. How about you? :slight_smile:
> Any feedback is highly welcome!
> Vladimir
>
> —
> You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> —
> You are currently subscribed to ntfsd as: xxxxx@Starbase.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com</http:>

Using FileObject->FileName in filter driverTry the RtlGenericTable package
defined in NTIFS.H. It uses a splay tree and has performaed great in all of
my filter drivers.

jamey
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chtchetkine, Vladimir
Sent: Wednesday, May 09, 2001 3:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to me)
it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the key
uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to
get better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I’m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Daniel:

Thank you for the reply. So, as I understood, there are no problems
with implementing that mechanism. At least on W2K SP1 - NTFS

FileName is not freed until FileObject gets closed (which is
reasonable approach). And nobody use it because docs say “don’t rely on it”.

What a waste of space! :slight_smile:

I understand all that you’ve said and I’m agreed with you 100%. I’m
simply trying to optimize my driver’s performance and one of the bottlenecks

that I’ve found is searching for my FileObject’s descriptor in each
and every IRP and Fast I/O request. On heavy loads (like building big
projects: > 10K

files) my table consists of hundreds (if not thousands) of descriptors
and browsing through it can be costly. Of course, if performance benefit
will

be insignificant I’m not going to use that “dirty trick”. I’m not even
sure if I will use it in any case. I just wanted to see some implications
that I

might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal mechanism.
Trying to squeeze into cracks like this will generate problems not just with
drivers which may, erroneously as you observe, sniff the filename field, but
with the OS if we try to make the rule more obvious in the future by freeing
the buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS
could decide to free it and I definitely am not going to give you any
guarantee here as to how it would/could work in the future. As a starting
point, one could expect the IO manager to do it (if it ever does) in
IopParseDevice, after the create is complete. This would be after your
completion routine runs.

The basic rule here is that you should never reuse fields of
internal structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.

Can I store my filter’s private data in FileObject->FileName field
after file gets opened (i.e. Create to the underlying FSD has been
completed)?

Since FileName field is no longer “valid” at this point I’d like to
utilize it to store my filter’s private information associated with this
FileObject.

What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all
    FileName’s structure fields.

  2. After underlying FSD has successfully completed the IRP I compare
    current FileName fields with those that I have saved at step 1.

  3. If they are equal or FileName has been “zeroed” (i.e. freed) I
    think that I can reuse it and deallocate FileName.Buffer (if it has not been

“zeroed” yet), prepare a UNICODE string that represents my filter’s
private data in string format (including a signature at the beginning of the
string)

and then save this string in FileName.
4. When filtering subsequent requests I check FileName for the
signature and if it matches what I’m expecting I will parse that string and
extract

whatever I have saved during Create.

So, if someone below my filter (another filter or FSD itself) has
changed FileName during IRP_MJ_CREATE I will not touch it (1-3)

If someone has changed FileName afterwards I will detect it by
checking the signature and will use some “conventional” mechanism to get

associated data.

To me these sound reasonable. How about you? :slight_smile:

Any feedback is highly welcome!

Vladimir


You are currently subscribed to ntfsd as:
xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Speaking of the generic table package, someone once told me that this splay
tree implementation at times rebalances during read operations, which seemed
strange to me (but having never actually implemented a splay tree, I took
this persons word for it). Later, a different person (rather emphatically)
indicated that this notion was incorrect. Could someone clear this up for
me? If true, it would have some serious implications on how access to the
tables is synchronized and should be documented.
-----Original Message-----
From: Jamey Kirby [mailto:xxxxx@storagecraft.com]
Sent: Thursday, May 10, 2001 11:04 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Try the RtlGenericTable package defined in NTIFS.H. It uses a splay tree and
has performaed great in all of my filter drivers.

jamey
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chtchetkine, Vladimir
Sent: Wednesday, May 09, 2001 3:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to me)
it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the key
uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to get
better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I’m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Nevermind… It IS documented:
If a matching element is found, RtlLookupElementGenericTable rebalances the
generic table’s splay tree.
-----Original Message-----
From: Smith, Joel
Sent: Thursday, May 10, 2001 11:12 AM
To: ‘File Systems Developers’
Subject: RE: [ntfsd] RE: Using FileObject->FileName in filter driver

Speaking of the generic table package, someone once told me that this splay
tree implementation at times rebalances during read operations, which seemed
strange to me (but having never actually implemented a splay tree, I took
this persons word for it). Later, a different person (rather emphatically)
indicated that this notion was incorrect. Could someone clear this up for
me? If true, it would have some serious implications on how access to the
tables is synchronized and should be documented.
-----Original Message-----
From: Jamey Kirby [mailto:xxxxx@storagecraft.com]
Sent: Thursday, May 10, 2001 11:04 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Try the RtlGenericTable package defined in NTIFS.H. It uses a splay tree and
has performaed great in all of my filter drivers.

jamey
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chtchetkine, Vladimir
Sent: Wednesday, May 09, 2001 3:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this is not
something for which I would sacrifice reliability of my product. But (to me)
it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing FOs
(generating the keys)? With reasonable table size and good level of the key
uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways to get
better than linear access times.

I am also definitely not saying there are no problems with your proposed
mechanism. What I’m trying to get across is that this is not your data: do
not touch it. This is how we run into really nasty problems. Just because
the data should not be used after create does not mean it should be reused
by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing something), but
aren’t we talking about associating contexts with file objects? What’s wrong
with the tried and true mechanisms of maintaining a volume relative balanced
tree or hash table mapping file objects to whatever data you need to
associate them? I mean, is log (number_of_file_objects) to near constant
time (avg) for lookups really that bad? Unless you have a large and
experienced testing department or Tony Mason-ish brainpower, don’t try to be
too clever (Been there done that!).
-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
Daniel:

Thank you for the reply. So, as I understood, there are no problems with
implementing that mechanism. At least on W2K SP1 - NTFS
FileName is not freed until FileObject gets closed (which is reasonable
approach). And nobody use it because docs say “don’t rely on it”.
What a waste of space! :slight_smile:
I understand all that you’ve said and I’m agreed with you 100%. I’m simply
trying to optimize my driver’s performance and one of the bottlenecks
that I’ve found is searching for my FileObject’s descriptor in each and
every IRP and Fast I/O request. On heavy loads (like building big projects:

10K
files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it can be costly. Of course, if performance benefit will
be insignificant I’m not going to use that “dirty trick”. I’m not even sure
if I will use it in any case. I just wanted to see some implications that I
might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
No.

If 1-3 will be required to work, simply use your normal mechanism. Trying to
squeeze into cracks like this will generate problems not just with drivers
which may, erroneously as you observe, sniff the filename field, but with
the OS if we try to make the rule more obvious in the future by freeing the
buffer and/or using the verifier to catch folks peeking at it, etc.

With regard to 3, you really have no way of guessing when the OS could
decide to free it and I definitely am not going to give you any guarantee
here as to how it would/could work in the future. As a starting point, one
could expect the IO manager to do it (if it ever does) in IopParseDevice,
after the create is complete. This would be after your completion routine
runs.

The basic rule here is that you should never reuse fields of internal
structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.
Can I store my filter’s private data in FileObject->FileName field after
file gets opened (i.e. Create to the underlying FSD has been completed)?
Since FileName field is no longer “valid” at this point I’d like to utilize
it to store my filter’s private information associated with this FileObject.
What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
    structure fields.
  2. After underlying FSD has successfully completed the IRP I compare current
    FileName fields with those that I have saved at step 1.
  3. If they are equal or FileName has been “zeroed” (i.e. freed) I think that
    I can reuse it and deallocate FileName.Buffer (if it has not been
    “zeroed” yet), prepare a UNICODE string that represents my filter’s private
    data in string format (including a signature at the beginning of the string)
    and then save this string in FileName.
  4. When filtering subsequent requests I check FileName for the signature and
    if it matches what I’m expecting I will parse that string and extract
    whatever I have saved during Create.
    So, if someone below my filter (another filter or FSD itself) has changed
    FileName during IRP_MJ_CREATE I will not touch it (1-3)
    If someone has changed FileName afterwards I will detect it by checking the
    signature and will use some “conventional” mechanism to get
    associated data.
    To me these sound reasonable. How about you? :slight_smile:
    Any feedback is highly welcome!
    Vladimir

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@Starbase.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Jamey,

There are some limitations to splay trees; the most serious we’ve noted is
that a lookup in a splay tree requires exclusive access to the tree, since
the tree is rebalanced around the entry that you locate within the tree
(which gives it the MRU characteristic.) Thus, we’ve observed bottlenecks
around certain types of concurrent activity. There are ways to mitigate
against this (oh, like a very quick CPU-bound single entry cache, since it
is VERY common for a thread of control to keep performing operations against
the same file on the same CPU.) Hashing is another good one (and was
separately suggested.)

But Vladimir’s original comment was that he found the cost of doing a lookup
prohibitive; I’ve certainly worked with customers who had similar concerns
(but then again, I tend to work with customer’s who have fairly
bleeding-edge kinds of requirements, as well.) Thus, I suggested one of the
techniques we’ve used in the past to totally eliminate the lookup
requirement (and to control the use of file objects in the underlying file
system.)

Fair warning: this technique of using duplicate file object and device
objects isn’t simple to implement. Unless you have specific need for it, I
wouldn’t recommend it (getting it “right” can be challenging.)

But I agree totally with Daniel Lovinger - DON’T try to “overload” a field
of the file object. Down that road lies madness.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Jamey Kirby [mailto:xxxxx@storagecraft.com]
Sent: Thursday, May 10, 2001 11:04 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Not sure if this helps, but I use a splay tree to track file objects and add
context. On dispatch entry, I look up the file object in the play tree.
There is a structure I have associated with the entry:

typedef struct _FILE_CONTEXT_RECORD
{
PFILE_OBJECT FileObject;
PVOID FileContext;
} FILE_CONTEXT_RECORD, *PFILE_CONTEXT_RECORD;

Splay tree are quite fast and reorder the list using a MRU algorithm.

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Wednesday, May 09, 2001 8:55 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Vladimir,

Since you are looking at rather ugly techniques anyway, another thing to
consider is that you can also create a second file object instead. Thus,
when the IRP_MJ_CREATE comes into your filter, you create a second file
object and pass that second file object into the underlying file system.
Then, when the IRP operation arrives into your filter YOU own the file
object. Follow it, find the second file object, and set up the subsequent
call to use the second file object.

There is a problem with this approach: there are six fast I/O
calls that get
passed AROUND a filter - and thus your file object is passed to the
underlying file system (there is now a callback you can register
in Windows
XP, but I’m not sure it would allow you to substitute the file
object.) The
best solution of which I’m aware to this problem is to use TWO device
objects for your filter (one that is “attached” and the other that is “not
attached”.) Fix up your file objects so that they point at the “not
attached” device. Then when subsequent calls arrive, they will come in on
the “not attached” device - and you won’t get skipped for those
six fast I/O
calls.

Yes, this is an ugly approach, but it is more savory than
utilizing the file
name field (I’m SURE that’s going to break something else - like another
filter on top of yours!)

And while I know it doesn’t solve the problem for you now (in NT 4.0 or
Windows 2000,) in Windows XP there will be a whole context-tracking
mechanism supported by the FsRtl package specifically to allow file system
filter drivers to associate context with the file object.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:
> -----Original Message-----
> From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
> Sent: Wednesday, May 09, 2001 10:33 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
>
>
> Daniel:
>
> Thank you for the reply. So, as I understood, there are no problems with
> implementing that mechanism. At least on W2K SP1 - NTFS
> FileName is not freed until FileObject gets closed (which is reasonable
> approach). And nobody use it because docs say “don’t rely on it”.
> What a waste of space! :slight_smile:
> I understand all that you’ve said and I’m agreed with you 100%. I’m simply
> trying to optimize my driver’s performance and one of the bottlenecks
> that I’ve found is searching for my FileObject’s descriptor in each and
> every IRP and Fast I/O request. On heavy loads (like building big
> projects:
> > 10K
> files) my table consists of hundreds (if not thousands) of descriptors and
> browsing through it can be costly. Of course, if performance benefit will
> be insignificant I’m not going to use that “dirty trick”. I’m not
> even sure
> if I will use it in any case. I just wanted to see some
> implications that I
> might oversee when I originally thought about that mechanism.
>
> Best regards,
>
> Vladimir
>
> -----Original Message-----
> From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
> Sent: Tuesday, May 08, 2001 6:47 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Using FileObject->FileName in filter driver
>
>
> No.
>
> If 1-3 will be required to work, simply use your normal
> mechanism. Trying to
> squeeze into cracks like this will generate problems not just with drivers
> which may, erroneously as you observe, sniff the filename field, but with
> the OS if we try to make the rule more obvious in the future by
> freeing the
> buffer and/or using the verifier to catch folks peeking at it, etc.
>
> With regard to 3, you really have no way of guessing when the OS could
> decide to free it and I definitely am not going to give you any guarantee
> here as to how it would/could work in the future. As a starting point, one
> could expect the IO manager to do it (if it ever does) in IopParseDevice,
> after the create is complete. This would be after your completion routine
> runs.
>
> The basic rule here is that you should never reuse fields of internal
> structures, ever, unless they are documented that way.
>
> -----Original Message-----
> From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
> Sent: Tuesday, May 08, 2001 4:36 PM
> To: File Systems Developers
> Subject: [ntfsd] Using FileObject->FileName in filter driver
>
> Hi everyone.
> Can I store my filter’s private data in FileObject->FileName field after
> file gets opened (i.e. Create to the underlying FSD has been completed)?
> Since FileName field is no longer “valid” at this point I’d like
> to utilize
> it to store my filter’s private information associated with this
> FileObject.
> What I’m thinking is (kinda algorithm):
> 1. Before I pass IRP_MJ_CREATE to the underlying FSD I save all FileName’s
> structure fields.
> 2. After underlying FSD has successfully completed the IRP I
> compare current
> FileName fields with those that I have saved at step 1.
> 3. If they are equal or FileName has been “zeroed” (i.e. freed) I
> think that
> I can reuse it and deallocate FileName.Buffer (if it has not been
> “zeroed” yet), prepare a UNICODE string that represents my
> filter’s private
> data in string format (including a signature at the beginning of
> the string)
> and then save this string in FileName.
> 4. When filtering subsequent requests I check FileName for the
> signature and
> if it matches what I’m expecting I will parse that string and extract
> whatever I have saved during Create.
> So, if someone below my filter (another filter or FSD itself) has changed
> FileName during IRP_MJ_CREATE I will not touch it (1-3)
> If someone has changed FileName afterwards I will detect it by
> checking the
> signature and will use some “conventional” mechanism to get
> associated data.
> To me these sound reasonable. How about you? :slight_smile:
> Any feedback is highly welcome!
> Vladimir
>
> —
> You are currently subscribed to ntfsd as: xxxxx@exchange.microsoft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> —
> You are currently subscribed to ntfsd as: xxxxx@Starbase.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com</http:>

Using FileObject->FileName in filter driver>Can I store my filter’s private
data in FileObject->FileName field after file gets opened (i.e.

Create to the underlying FSD has been completed)?

IMHO this is dangerous. I would prefer to allocate my own structure and
associate it with a file object pointer using the Rtl generic table.

Max


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Using FileObject->FileName in filter driver>request. On heavy loads (like
building big projects: > 10K

files) my table consists of hundreds (if not thousands) of descriptors and
browsing through it
can be costly. Of course, if performance benefit will

Use an Rtl generic table for it - you will need only 10 compares to scan a
table of 1000.

Max


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Using FileObject->FileName in filter driverIf you enumerate the tree with
splay, the list will be flattened. You can enumerate the list without
playing and it will not effect the balance.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Smith, Joel
Sent: Thursday, May 10, 2001 8:26 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Nevermind… It IS documented:
If a matching element is found, RtlLookupElementGenericTable rebalances
the generic table’s splay tree.

-----Original Message-----
From: Smith, Joel
Sent: Thursday, May 10, 2001 11:12 AM
To: ‘File Systems Developers’
Subject: RE: [ntfsd] RE: Using FileObject->FileName in filter driver

Speaking of the generic table package, someone once told me that this
splay tree implementation at times rebalances during read operations, which
seemed strange to me (but having never actually implemented a splay tree, I
took this persons word for it). Later, a different person (rather
emphatically) indicated that this notion was incorrect. Could someone clear
this up for me? If true, it would have some serious implications on how
access to the tables is synchronized and should be documented.
-----Original Message-----
From: Jamey Kirby [mailto:xxxxx@storagecraft.com]
Sent: Thursday, May 10, 2001 11:04 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Try the RtlGenericTable package defined in NTIFS.H. It uses a splay
tree and has performaed great in all of my filter drivers.

jamey
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Chtchetkine, Vladimir
Sent: Wednesday, May 09, 2001 3:05 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Well guys, you have very strong arguments! :slight_smile: And definitely, this
is not something for which I would sacrifice reliability of my product. But
(to me) it worth a try.
And sorry if I took too much of your time on this topic.

Just as a follow-up. What would be the best algorithm for hashing
FOs (generating the keys)? With reasonable table size and good level of the
key uniqueness?

Best regards,

Vladimir
-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Wednesday, May 09, 2001 1:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

I think Joel says it better than I could. There are plenty of ways
to get better than linear access times.

I am also definitely not saying there are no problems with your
proposed mechanism. What I’m trying to get across is that this is not your
data: do not touch it. This is how we run into really nasty problems. Just
because the data should not be used after create does not mean it should be
reused by the first component to get to it.

-----Original Message-----
From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
Sent: Wednesday, May 09, 2001 1:00 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Forgive me for being dense (I have a feeling I’m missing
something), but aren’t we talking about associating contexts with file
objects? What’s wrong with the tried and true mechanisms of maintaining a
volume relative balanced tree or hash table mapping file objects to whatever
data you need to associate them? I mean, is log (number_of_file_objects) to
near constant time (avg) for lookups really that bad? Unless you have a
large and experienced testing department or Tony Mason-ish brainpower, don’t
try to be too clever (Been there done that!).

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 09, 2001 10:33 AM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

Daniel:

Thank you for the reply. So, as I understood, there are no
problems with implementing that mechanism. At least on W2K SP1 - NTFS

FileName is not freed until FileObject gets closed (which is
reasonable approach). And nobody use it because docs say “don’t rely on it”.

What a waste of space! :slight_smile:

I understand all that you’ve said and I’m agreed with you 100%.
I’m simply trying to optimize my driver’s performance and one of the
bottlenecks

that I’ve found is searching for my FileObject’s descriptor in
each and every IRP and Fast I/O request. On heavy loads (like building big
projects: > 10K

files) my table consists of hundreds (if not thousands) of
descriptors and browsing through it can be costly. Of course, if performance
benefit will

be insignificant I’m not going to use that “dirty trick”. I’m
not even sure if I will use it in any case. I just wanted to see some
implications that I

might oversee when I originally thought about that mechanism.

Best regards,

Vladimir

-----Original Message-----
From: Daniel Lovinger [mailto:xxxxx@exchange.microsoft.com]
Sent: Tuesday, May 08, 2001 6:47 PM
To: File Systems Developers
Subject: [ntfsd] RE: Using FileObject->FileName in filter driver

No.

If 1-3 will be required to work, simply use your normal
mechanism. Trying to squeeze into cracks like this will generate problems
not just with drivers which may, erroneously as you observe, sniff the
filename field, but with the OS if we try to make the rule more obvious in
the future by freeing the buffer and/or using the verifier to catch folks
peeking at it, etc.

With regard to 3, you really have no way of guessing when the
OS could decide to free it and I definitely am not going to give you any
guarantee here as to how it would/could work in the future. As a starting
point, one could expect the IO manager to do it (if it ever does) in
IopParseDevice, after the create is complete. This would be after your
completion routine runs.

The basic rule here is that you should never reuse fields of
internal structures, ever, unless they are documented that way.

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Tuesday, May 08, 2001 4:36 PM
To: File Systems Developers
Subject: [ntfsd] Using FileObject->FileName in filter driver

Hi everyone.

Can I store my filter’s private data in FileObject->FileName
field after file gets opened (i.e. Create to the underlying FSD has been
completed)?

Since FileName field is no longer “valid” at this point I’d
like to utilize it to store my filter’s private information associated with
this FileObject.

What I’m thinking is (kinda algorithm):

  1. Before I pass IRP_MJ_CREATE to the underlying FSD I save
    all FileName’s structure fields.

  2. After underlying FSD has successfully completed the IRP I
    compare current FileName fields with those that I have saved at step 1.

  3. If they are equal or FileName has been “zeroed” (i.e.
    freed) I think that I can reuse it and deallocate FileName.Buffer (if it has
    not been

“zeroed” yet), prepare a UNICODE string that represents my
filter’s private data in string format (including a signature at the
beginning of the string)

and then save this string in FileName.
4. When filtering subsequent requests I check FileName for the
signature and if it matches what I’m expecting I will parse that string and
extract

whatever I have saved during Create.

So, if someone below my filter (another filter or FSD itself)
has changed FileName during IRP_MJ_CREATE I will not touch it (1-3)

If someone has changed FileName afterwards I will detect it by
checking the signature and will use some “conventional” mechanism to get

associated data.

To me these sound reasonable. How about you? :slight_smile:

Any feedback is highly welcome!

Vladimir


You are currently subscribed to ntfsd as:
xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as:
xxxxx@Starbase.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as:
xxxxx@exchange.microsoft.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as:
xxxxx@Starbase.com
To unsubscribe send a blank email to
leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Where can I find some information about generic table package and related
Rtl routines / RtlGenericTable(), RtlLookupElementGenericTable() … / ??

I didn’t see anything about it in my DDK nor IFS documents

Kristian


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com