Hi all,
When does file info get added to fltmgr’s cache? Does this happen once
the FileObject is created, pre-create or post-create - where?
Trying to query the cache in Pre-create using FltGetFileNameInformation
with FLT_FILE_NAME_QUERY_CACHE_ONLY,
but keep bugchecking…
The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
m
Why would it be available in pre-create? The file-object has just been created,
and no cache is yet available - the name is only generated once you call the
FltGetFileNameInformation.
I don’t understand why you get a bugcheck, it should return an error code only -
checked build maybe?
Regards, Dejan.
MM wrote:
Hi all,
When does file info get added to fltmgr’s cache? Does this happen once
the FileObject is created, pre-create or post-create - where?Trying to query the cache in Pre-create using FltGetFileNameInformation
with FLT_FILE_NAME_QUERY_CACHE_ONLY,
but keep bugchecking…The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
m
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
–
Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.
ok, managed to get rid of the bugcheck (kinda, still haven’t got it to
release the nameinfo without bugchecking so far, thus leaking memory
right now)… Now I’m getting cache misses on everything just about,
there are a few exceptions…
What I’m trying to do Dejan is avoid a hash table… In the pre-create
callback, I want query the cache to see if the file is open (or has been
opened).
If the name is in the cache, I simply want to return
FLT_PREOP_SUCCESS_NO_CALLBACK.
If there is a cache miss (file is closed or was never opened), then I
want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
and molest the file in my Post-create callback.
m
Dejan Maksimovic wrote:
Why would it be available in pre-create? The file-object has just been created,
and no cache is yet available - the name is only generated once you call the
FltGetFileNameInformation.
I don’t understand why you get a bugcheck, it should return an error code only -
checked build maybe?Regards, Dejan.
MM wrote:
>Hi all,
>
>When does file info get added to fltmgr’s cache? Does this happen once
>the FileObject is created, pre-create or post-create - where?
>
>Trying to query the cache in Pre-create using FltGetFileNameInformation
>with FLT_FILE_NAME_QUERY_CACHE_ONLY,
>but keep bugchecking…
>
>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>
>m
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@alfasp.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>–
Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Hmm, the idea is right if the file was already opened, the FsContext should match.
Though I don’t know if/how long the FltMgr would keep such a cache after the file is
closed completely. Obviously not for long judging from your results.
I think you must look for a different solution.
MM wrote:
ok, managed to get rid of the bugcheck (kinda, still haven’t got it to release the
nameinfo without bugchecking so far, thus leaking memory right now)… Now I’m getting
cache misses on everything just about, there are a few exceptions…What I’m trying to do Dejan is avoid a hash table… In the pre-create callback, I want
query the cache to see if the file is open (or has been opened). If the name is in the
cache, I simply want to return FLT_PREOP_SUCCESS_NO_CALLBACK.If there is a cache miss (file is closed or was never opened), then I want to return
FLT_PREOP_SUCCESS_WITH_CALLBACK and molest the file in my Post-create callback.
–
Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.
In reading the patent MS filed, the cache for an item is destroyed once
the last reference to a file is closed. This is fine with me…
If the file is already opened, I don’t care about it… If the file has
been closed, I want it to pass threw the post-op molestation again anyways.
This is irritating the hell out of me, such a simple thing I can’t get
to work right, urgh!!!
m
Dejan Maksimovic wrote:
Hmm, the idea is right if the file was already opened, the FsContext should match.
Though I don’t know if/how long the FltMgr would keep such a cache after the file is
closed completely. Obviously not for long judging from your results.
I think you must look for a different solution.MM wrote:
>ok, managed to get rid of the bugcheck (kinda, still haven’t got it to release the
>nameinfo without bugchecking so far, thus leaking memory right now)… Now I’m getting
>cache misses on everything just about, there are a few exceptions…
>
>What I’m trying to do Dejan is avoid a hash table… In the pre-create callback, I want
>query the cache to see if the file is open (or has been opened). If the name is in the
>cache, I simply want to return FLT_PREOP_SUCCESS_NO_CALLBACK.
>
>If there is a cache miss (file is closed or was never opened), then I want to return
>FLT_PREOP_SUCCESS_WITH_CALLBACK and molest the file in my Post-create callback.
>
>–
Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Ok, running out of idea’s here… Here is a tid bit of code I placed in
the pre-create callback. Practically every
thing is a Cache Miss (with random exceptions).
For example, if I open multiple instances of Calc.exe. Each file it
loads along with itself shows a cache miss. Even
though there are multiple FO’s created, the fltmgr’s cache should be
universal… FltReleaseFileNameInformation
bugchecks when it’s called…
What I’m trying to do should be more than possible, I know I’m making
some armature mistake… Can someone
please point out where I’m going wrong…
status = FltGetFileNameInformation( Data,
FLT_FILE_NAME_OPENED |
FLT_FILE_NAME_QUERY_CACHE_ONLY,
&nameInfo );
if(status == STATUS_FLT_NAME_CACHE_MISS)
{
DbgPrint(“Callback NEEDED\n”);
//FltReleaseFileNameInformation( nameInfo );
return FLT_PREOP_SUCCESS_WITH_CALLBACK;
}else{
DbgPrint(“NO CALLBACK\n”);
FltReleaseFileNameInformation( nameInfo );
return FLT_PREOP_SUCCESS_NO_CALLBACK;
MM wrote:
In reading the patent MS filed, the cache for an item is destroyed
once the last reference to a file is closed. This is fine with me…If the file is already opened, I don’t care about it… If the file
has been closed, I want it to pass threw the post-op molestation again
anyways.This is irritating the hell out of me, such a simple thing I can’t get
to work right, urgh!!!m
Dejan Maksimovic wrote:
> Hmm, the idea is right if the file was already opened, the
> FsContext should match.
> Though I don’t know if/how long the FltMgr would keep such a cache
> after the file is
> closed completely. Obviously not for long judging from your results.
> I think you must look for a different solution.
>
>
> MM wrote:
>
>
>
>> ok, managed to get rid of the bugcheck (kinda, still haven’t got it
>> to release the
>> nameinfo without bugchecking so far, thus leaking memory right
>> now)… Now I’m getting
>> cache misses on everything just about, there are a few exceptions…
>>
>> What I’m trying to do Dejan is avoid a hash table… In the
>> pre-create callback, I want
>> query the cache to see if the file is open (or has been opened). If
>> the name is in the
>> cache, I simply want to return FLT_PREOP_SUCCESS_NO_CALLBACK.
>>
>> If there is a cache miss (file is closed or was never opened), then
>> I want to return
>> FLT_PREOP_SUCCESS_WITH_CALLBACK and molest the file in my
>> Post-create callback.
>>
>
>
> –
> Kind regards, Dejan M.
> http://www.alfasp.com E-mail: xxxxx@alfasp.com
> Alfa Transparent File Encryptor - Transparent file encryption services.
> Alfa File Protector - File protection and hiding library for Win32
> developers.
> Alfa File Monitor - File monitoring library for Win32 developers.
>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@comcast.net
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Using name lookup functions to determine whether a file is already open is
just plain broken. There could be a filter above you that queries the name
in its precreate, which would cause the name to be cached, even though
you’ve never seen a create for the FsContext in question. Also, it is
difficult to associate an incoming create with existing open files. Do not
expect fltmgr to make that association (and provide the cached name) until
after the create has gone to the file system.
You should make your decision in the post-create callback, and depend on
your stream context, not on whether a name is in cache, to determine what
processing you need to do.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 3:52 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Minifilter cache
ok, managed to get rid of the bugcheck (kinda, still haven’t got it to
release the nameinfo without bugchecking so far, thus leaking memory
right now)… Now I’m getting cache misses on everything just about,
there are a few exceptions…
What I’m trying to do Dejan is avoid a hash table… In the pre-create
callback, I want query the cache to see if the file is open (or has been
opened).
If the name is in the cache, I simply want to return
FLT_PREOP_SUCCESS_NO_CALLBACK.
If there is a cache miss (file is closed or was never opened), then I
want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
and molest the file in my Post-create callback.
m
Dejan Maksimovic wrote:
Why would it be available in pre-create? The file-object has just
been created, and no cache is yet available - the name is only
generated once you call the FltGetFileNameInformation.
I don’t understand why you get a bugcheck, it should return an
error code only - checked build maybe?Regards, Dejan.
MM wrote:
>Hi all,
>
>When does file info get added to fltmgr’s cache? Does this happen once
>the FileObject is created, pre-create or post-create - where?
>
>Trying to query the cache in Pre-create using
>FltGetFileNameInformation with FLT_FILE_NAME_QUERY_CACHE_ONLY, but
>keep bugchecking…
>
>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>
>m
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@alfasp.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>–
Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32
developers. Alfa File Monitor - File monitoring library for Win32
developers.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@comcast.net To
unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@privtek.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Dan Kyler wrote:
Using name lookup functions to determine whether a file is already open is
just plain broken.
And you know this to be true with minifilters?
There could be a filter above you that queries the name
in its precreate, which would cause the name to be cached,
From the patent info I read, if a filter above me/or below me queries
the name, this shouldn’t matter.
Once a file is fully opened, the fltmgr’s cache count is incremented,
and every time a file is closed, it
is decremented. The cache is shared between all mini’s. Once the ref
count hits zero, the cache is deleted.
Querying for the file shouldn’t affect this unless the filter
above/below me was broken.
At least that is my take on it…
Also, it is
difficult to associate an incoming create with existing open files. Do not
expect fltmgr to make that association (and provide the cached name) until
after the create has gone to the file system.
That is one thing I don’t understand fully, the doc’s are confusing
me… The flag FLT_FILE_NAME_OPENED
states “Returns the name that was used when the file was opened.” - and
it is stated this can be used in Pre-
create(FltGetFileNameInfo). It sounds like an open can be associated
with a previous open…
You should make your decision in the post-create callback, and depend on
your stream context, not on whether a name is in cache, to determine what
processing you need to do.
I’ll give that a try after I get some sleep.
Has anyone else here tried anything like this and had it work? Sure
someone has, seems like a basic idea…
Did you try something like this Dan?
m
- Dan.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 3:52 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Minifilter cacheok, managed to get rid of the bugcheck (kinda, still haven’t got it to
release the nameinfo without bugchecking so far, thus leaking memory
right now)… Now I’m getting cache misses on everything just about,
there are a few exceptions…What I’m trying to do Dejan is avoid a hash table… In the pre-create
callback, I want query the cache to see if the file is open (or has been
opened).
If the name is in the cache, I simply want to return
FLT_PREOP_SUCCESS_NO_CALLBACK.If there is a cache miss (file is closed or was never opened), then I
want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
and molest the file in my Post-create callback.m
Dejan Maksimovic wrote:
> Why would it be available in pre-create? The file-object has just
>been created, and no cache is yet available - the name is only
>generated once you call the FltGetFileNameInformation.
> I don’t understand why you get a bugcheck, it should return an
>error code only - checked build maybe?
>
> Regards, Dejan.
>
>MM wrote:
>
>
>
>
>
>>Hi all,
>>
>>When does file info get added to fltmgr’s cache? Does this happen once
>>the FileObject is created, pre-create or post-create - where?
>>
>>Trying to query the cache in Pre-create using
>>FltGetFileNameInformation with FLT_FILE_NAME_QUERY_CACHE_ONLY, but
>>keep bugchecking…
>>
>>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>>
>>m
>>
>>—
>>Questions? First check the IFS FAQ at
>>https://www.osronline.com/article.cfm?id=17
>>
>>You are currently subscribed to ntfsd as: xxxxx@alfasp.com
>>To unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>>
>>
>–
>Kind regards, Dejan M.
>http://www.alfasp.com E-mail: xxxxx@alfasp.com
>Alfa Transparent File Encryptor - Transparent file encryption services.
>Alfa File Protector - File protection and hiding library for Win32
>developers. Alfa File Monitor - File monitoring library for Win32
>developers.
>
>
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@comcast.net To
>unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
>
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@privtek.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
>And you know this to be true with minifilters?
The purpose of name lookup functions is to get the name. The purpose of the
name cache is to improve name lookup performance. Using existence in the
name cache as heuristic to guess about the state of the file will bite you
eventually. A patent application is not IFS kit documentation, and you
cannot depend on implementation details remaining the same.
That is one thing I don’t understand fully, the doc’s are confusing
me… The flag FLT_FILE_NAME_OPENED
states “Returns the name that was used when the file was opened.” - and
it is stated this can be used in Pre- create(FltGetFileNameInfo). It sounds
like an open can be associated
with a previous open…
No, that is not what that means. The name used to open the file is
available in pre-create only because it’s right there in the file object.
This form of name simply means that you get the particular “length” that was
used to open the file (long and short name combinations), rather than a
“normalized” name (all long names).
Has anyone else here tried anything like this and had it work? Sure
someone has, seems like a basic idea…
Did you try something like this Dan?
Certainly. If you always create/retrieve a stream context in post-create,
then if the file is already open, you’ll get back your existing stream
context. If it is not, a new stream context will be created. If your
stream context already exists, you can return at that point. If not, you
can continue with your “molestation”.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 8:14 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Minifilter cache
Dan Kyler wrote:
Using name lookup functions to determine whether a file is already open
is just plain broken.
And you know this to be true with minifilters?
There could be a filter above you that queries the name
in its precreate, which would cause the name to be cached,
From the patent info I read, if a filter above me/or below me queries
the name, this shouldn’t matter.
Once a file is fully opened, the fltmgr’s cache count is incremented,
and every time a file is closed, it
is decremented. The cache is shared between all mini’s. Once the ref
count hits zero, the cache is deleted.
Querying for the file shouldn’t affect this unless the filter
above/below me was broken.
At least that is my take on it…
Also, it is
difficult to associate an incoming create with existing open files. Do
not expect fltmgr to make that association (and provide the cached
name) until after the create has gone to the file system.
That is one thing I don’t understand fully, the doc’s are confusing
me… The flag FLT_FILE_NAME_OPENED
states “Returns the name that was used when the file was opened.” - and
it is stated this can be used in Pre- create(FltGetFileNameInfo). It sounds
like an open can be associated
with a previous open…
You should make your decision in the post-create callback, and depend
on your stream context, not on whether a name is in cache, to determine
what processing you need to do.
I’ll give that a try after I get some sleep.
Has anyone else here tried anything like this and had it work? Sure
someone has, seems like a basic idea…
Did you try something like this Dan?
m
- Dan.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 3:52 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Minifilter cacheok, managed to get rid of the bugcheck (kinda, still haven’t got it to
release the nameinfo without bugchecking so far, thus leaking memory
right now)… Now I’m getting cache misses on everything just about,
there are a few exceptions…What I’m trying to do Dejan is avoid a hash table… In the pre-create
callback, I want query the cache to see if the file is open (or has been
opened).
If the name is in the cache, I simply want to return
FLT_PREOP_SUCCESS_NO_CALLBACK.If there is a cache miss (file is closed or was never opened), then I
want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
and molest the file in my Post-create callback.m
Dejan Maksimovic wrote:
> Why would it be available in pre-create? The file-object has just
>been created, and no cache is yet available - the name is only
>generated once you call the FltGetFileNameInformation.
> I don’t understand why you get a bugcheck, it should return an
>error code only - checked build maybe?
>
> Regards, Dejan.
>
>MM wrote:
>
>
>
>
>
>>Hi all,
>>
>>When does file info get added to fltmgr’s cache? Does this happen
>>once
>>the FileObject is created, pre-create or post-create - where?
>>
>>Trying to query the cache in Pre-create using
>>FltGetFileNameInformation with FLT_FILE_NAME_QUERY_CACHE_ONLY, but
>>keep bugchecking…
>>
>>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>>
>>m
>>
>>—
>>Questions? First check the IFS FAQ at
>>https://www.osronline.com/article.cfm?id=17
>>
>>You are currently subscribed to ntfsd as: xxxxx@alfasp.com To
>>unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>>
>>
>–
>Kind regards, Dejan M.
>http://www.alfasp.com E-mail: xxxxx@alfasp.com
>Alfa Transparent File Encryptor - Transparent file encryption
>services.
>Alfa File Protector - File protection and hiding library for Win32
>developers. Alfa File Monitor - File monitoring library for Win32
>developers.
>
>
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@comcast.net To
>unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
>
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@privtek.com To
unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@comcast.net To
unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@privtek.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
This will also likely break when a file has multiple names (“hard
links”) or has been opened by ID. In these cases the name you are
checking won’t match the name in the cache, even though the file is
opened.
Hard link APIs are published in W2K3 (Win32 Platform SDK) and open by ID
is used by SFU (which ships with Vista client). Thus, you should expect
to see both of these “in the real world”.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 10:14 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cache
Dan Kyler wrote:
Using name lookup functions to determine whether a file is already open
is
just plain broken.
And you know this to be true with minifilters?
There could be a filter above you that queries the name
in its precreate, which would cause the name to be cached,
From the patent info I read, if a filter above me/or below me queries
the name, this shouldn’t matter.
Once a file is fully opened, the fltmgr’s cache count is incremented,
and every time a file is closed, it
is decremented. The cache is shared between all mini’s. Once the ref
count hits zero, the cache is deleted.
Querying for the file shouldn’t affect this unless the filter
above/below me was broken.
At least that is my take on it…
Also, it is
difficult to associate an incoming create with existing open files. Do
not
expect fltmgr to make that association (and provide the cached name)
until
after the create has gone to the file system.
That is one thing I don’t understand fully, the doc’s are confusing
me… The flag FLT_FILE_NAME_OPENED
states “Returns the name that was used when the file was opened.” - and
it is stated this can be used in Pre-
create(FltGetFileNameInfo). It sounds like an open can be associated
with a previous open…
You should make your decision in the post-create callback, and depend
on
your stream context, not on whether a name is in cache, to determine
what
processing you need to do.
I’ll give that a try after I get some sleep.
Has anyone else here tried anything like this and had it work? Sure
someone has, seems like a basic idea…
Did you try something like this Dan?
m
- Dan.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 3:52 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Minifilter cacheok, managed to get rid of the bugcheck (kinda, still haven’t got it to
release the nameinfo without bugchecking so far, thus leaking memory
right now)… Now I’m getting cache misses on everything just about,
there are a few exceptions…What I’m trying to do Dejan is avoid a hash table… In the pre-create
callback, I want query the cache to see if the file is open (or has
been
opened).
If the name is in the cache, I simply want to return
FLT_PREOP_SUCCESS_NO_CALLBACK.If there is a cache miss (file is closed or was never opened), then I
want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
and molest the file in my Post-create callback.m
Dejan Maksimovic wrote:
> Why would it be available in pre-create? The file-object has just
>been created, and no cache is yet available - the name is only
>generated once you call the FltGetFileNameInformation.
> I don’t understand why you get a bugcheck, it should return an
>error code only - checked build maybe?
>
> Regards, Dejan.
>
>MM wrote:
>
>
>
>
>
>>Hi all,
>>
>>When does file info get added to fltmgr’s cache? Does this happen
once
>>the FileObject is created, pre-create or post-create - where?
>>
>>Trying to query the cache in Pre-create using
>>FltGetFileNameInformation with FLT_FILE_NAME_QUERY_CACHE_ONLY, but
>>keep bugchecking…
>>
>>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>>
>>m
>>
>>—
>>Questions? First check the IFS FAQ at
>>https://www.osronline.com/article.cfm?id=17
>>
>>You are currently subscribed to ntfsd as: xxxxx@alfasp.com
>>To unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>>
>>
>–
>Kind regards, Dejan M.
>http://www.alfasp.com E-mail: xxxxx@alfasp.com
>Alfa Transparent File Encryptor - Transparent file encryption
services.
>Alfa File Protector - File protection and hiding library for Win32
>developers. Alfa File Monitor - File monitoring library for Win32
>developers.
>
>
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@comcast.net To
>unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
>
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@privtek.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Dan Kyler wrote:
Certainly. If you always create/retrieve a stream context in post-create,
then if the file is already open, you’ll get back your existing stream
context. If it is not, a new stream context will be created. If your
stream context already exists, you can return at that point. If not, you
can continue with your “molestation”.
I’ve never implemented stream context tracking in a minifilter. I was
under the impression the fltmgr
kept the stream context info automatically, but seems it doesn’t. I was
thinking calling FltGetStreamContext
in the post-create by itself would find the stream (returned
STATUS_NOT_FOUND every time)…
So, what I need to do in the post-create is first call
FltGetStreamContext, then if that fails, call FltAllocateContext,
then FltSetStreamContext and proceed. Is this right? If so, I’ve
increased the ref count; where in the filter
should I decrement it’s count?
Thanks for your help
m
Excellent points, thanks Tony.
You mentioned SFU, what exactly is that? That is a unix network protocol
right???
m
Tony Mason wrote:
This will also likely break when a file has multiple names (“hard
links”) or has been opened by ID. In these cases the name you are
checking won’t match the name in the cache, even though the file is
opened.Hard link APIs are published in W2K3 (Win32 Platform SDK) and open by ID
is used by SFU (which ships with Vista client). Thus, you should expect
to see both of these “in the real world”.Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Monday, May 01, 2006 10:14 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cacheDan Kyler wrote:
>Using name lookup functions to determine whether a file is already open
>
>
is>just plain broken.
>
>
>
And you know this to be true with minifilters?>There could be a filter above you that queries the name
>in its precreate, which would cause the name to be cached,
>
>
>
From the patent info I read, if a filter above me/or below me queries
the name, this shouldn’t matter.
Once a file is fully opened, the fltmgr’s cache count is incremented,
and every time a file is closed, it
is decremented. The cache is shared between all mini’s. Once the ref
count hits zero, the cache is deleted.
Querying for the file shouldn’t affect this unless the filter
above/below me was broken.
At least that is my take on it…>Also, it is
>difficult to associate an incoming create with existing open files. Do
>
>
not>expect fltmgr to make that association (and provide the cached name)
>
>
until>after the create has gone to the file system.
>
>
>
>
That is one thing I don’t understand fully, the doc’s are confusing
me… The flag FLT_FILE_NAME_OPENED
states “Returns the name that was used when the file was opened.” - and
it is stated this can be used in Pre-
create(FltGetFileNameInfo). It sounds like an open can be associated
with a previous open…>You should make your decision in the post-create callback, and depend
>
>
on>your stream context, not on whether a name is in cache, to determine
>
>
what>processing you need to do.
>
>
>
>
I’ll give that a try after I get some sleep.Has anyone else here tried anything like this and had it work? Sure
someone has, seems like a basic idea…
Did you try something like this Dan?m
>- Dan.
>
>-----Original Message-----
>From: xxxxx@lists.osr.com
>[mailto:xxxxx@lists.osr.com] On Behalf Of MM
>Sent: Monday, May 01, 2006 3:52 AM
>To: Windows File Systems Devs Interest List
>Subject: Re: [ntfsd] Minifilter cache
>
>
>ok, managed to get rid of the bugcheck (kinda, still haven’t got it to
>release the nameinfo without bugchecking so far, thus leaking memory
>right now)… Now I’m getting cache misses on everything just about,
>there are a few exceptions…
>
>What I’m trying to do Dejan is avoid a hash table… In the pre-create
>callback, I want query the cache to see if the file is open (or has
>
>
been>opened).
>If the name is in the cache, I simply want to return
>FLT_PREOP_SUCCESS_NO_CALLBACK.
>
>If there is a cache miss (file is closed or was never opened), then I
>want to return FLT_PREOP_SUCCESS_WITH_CALLBACK
>and molest the file in my Post-create callback.
>
>m
>
>Dejan Maksimovic wrote:
>
>
>
>
>
>> Why would it be available in pre-create? The file-object has just
>>been created, and no cache is yet available - the name is only
>>generated once you call the FltGetFileNameInformation.
>> I don’t understand why you get a bugcheck, it should return an
>>error code only - checked build maybe?
>>
>> Regards, Dejan.
>>
>>MM wrote:
>>
>>
>>
>>
>>
>>
>>
>>>Hi all,
>>>
>>>When does file info get added to fltmgr’s cache? Does this happen
>>>
>>>
once>>>the FileObject is created, pre-create or post-create - where?
>>>
>>>Trying to query the cache in Pre-create using
>>>FltGetFileNameInformation with FLT_FILE_NAME_QUERY_CACHE_ONLY, but
>>>keep bugchecking…
>>>
>>>The fltmgr’s cache should be avaliable in pre-create, shouldn’t it?
>>>
>>>m
>>>
>>>—
>>>Questions? First check the IFS FAQ at
>>>https://www.osronline.com/article.cfm?id=17
>>>
>>>You are currently subscribed to ntfsd as: xxxxx@alfasp.com
>>>To unsubscribe send a blank email to xxxxx@lists.osr.com
>>>
>>>
>>>
>>>
>>>
>>>
>>–
>>Kind regards, Dejan M.
>>http://www.alfasp.com E-mail: xxxxx@alfasp.com
>>Alfa Transparent File Encryptor - Transparent file encryption
>>
>>
services.>>Alfa File Protector - File protection and hiding library for Win32
>>developers. Alfa File Monitor - File monitoring library for Win32
>>developers.
>>
>>
>>
>>—
>>Questions? First check the IFS FAQ at
>>https://www.osronline.com/article.cfm?id=17
>>
>>You are currently subscribed to ntfsd as: xxxxx@comcast.net To
>>unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>>
>>
>>
>>
>>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@privtek.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>—
>Questions? First check the IFS FAQ at
>
>
https://www.osronline.com/article.cfm?id=17>You are currently subscribed to ntfsd as: xxxxx@comcast.net
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
>
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
I always allocate the stream context in pre-create (so I can fail the
create operation in case of allocation failure) and then pass it to the
completion routine (that’s the general paradigm I use - allocate in
dispatch, use/free in completion.) Otherwise, in completion you get
stuck with complex failure handling (allocation fails, so you then get
into the “let’s cancel the open” error handling, etc.)
Decrement the reference count when you’re done with the context (I
generally do it in pre-close for stream handle contexts, for example,
and post-close for stream contexts.) Since you can set up your own
function to be told when the context is being torn down as well, you can
know when the last reference to the context “goes away”. I have a more
complex context management model anyway because I keep them on a list in
the instance, so that I can unload the mini-filter at any time (I just
walk the instance lists and tear everything down…)
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Tuesday, May 02, 2006 7:50 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cache
Dan Kyler wrote:
Certainly. If you always create/retrieve a stream context in
post-create,
then if the file is already open, you’ll get back your existing stream
context. If it is not, a new stream context will be created. If your
stream context already exists, you can return at that point. If not,
you
can continue with your “molestation”.
I’ve never implemented stream context tracking in a minifilter. I was
under the impression the fltmgr
kept the stream context info automatically, but seems it doesn’t. I was
thinking calling FltGetStreamContext
in the post-create by itself would find the stream (returned
STATUS_NOT_FOUND every time)…
So, what I need to do in the post-create is first call
FltGetStreamContext, then if that fails, call FltAllocateContext,
then FltSetStreamContext and proceed. Is this right? If so, I’ve
increased the ref count; where in the filter
should I decrement it’s count?
Thanks for your help
m
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
SFU = “Services for Unix” which means “NFS” plus other interoperability
software. For us, the only thing that matters is a new file server and
a new network redirector client.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Tony and Dan, thanks for pointing me in the correct direction… I owe
you guys.
m
Tony Mason wrote:
I always allocate the stream context in pre-create (so I can fail the
create operation in case of allocation failure) and then pass it to the
completion routine (that’s the general paradigm I use - allocate in
dispatch, use/free in completion.) Otherwise, in completion you get
stuck with complex failure handling (allocation fails, so you then get
into the “let’s cancel the open” error handling, etc.)Decrement the reference count when you’re done with the context (I
generally do it in pre-close for stream handle contexts, for example,
and post-close for stream contexts.) Since you can set up your own
function to be told when the context is being torn down as well, you can
know when the last reference to the context “goes away”. I have a more
complex context management model anyway because I keep them on a list in
the instance, so that I can unload the mini-filter at any time (I just
walk the instance lists and tear everything down…)Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Tuesday, May 02, 2006 7:50 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cacheDan Kyler wrote:
>Certainly. If you always create/retrieve a stream context in
>
>
post-create,>then if the file is already open, you’ll get back your existing stream
>context. If it is not, a new stream context will be created. If your
>stream context already exists, you can return at that point. If not,
>
>
you>can continue with your “molestation”.
>
>
>
I’ve never implemented stream context tracking in a minifilter. I was
under the impression the fltmgr
kept the stream context info automatically, but seems it doesn’t. I was
thinking calling FltGetStreamContext
in the post-create by itself would find the stream (returned
STATUS_NOT_FOUND every time)…So, what I need to do in the post-create is first call
FltGetStreamContext, then if that fails, call FltAllocateContext,
then FltSetStreamContext and proceed. Is this right? If so, I’ve
increased the ref count; where in the filter
should I decrement it’s count?Thanks for your help
m
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Tony Mason wrote:
I always allocate the stream context in pre-create (so I can fail the
create operation in case of allocation failure) and then pass it to the
completion routine (that’s the general paradigm I use - allocate in
dispatch, use/free in completion.) Otherwise, in completion you get
stuck with complex failure handling (allocation fails, so you then get
into the “let’s cancel the open” error handling, etc.)
I see in the doc’s *FltAllocateContext* must be running at IRQL <=
APC_LEVEL,
where Post-Create OP’s can be called at an IRQL <= Dispatch sometimes.
Thus, it looks as if some of the samples
in the wdk could result in an IRQL_NOT_LESS_OR_EQUAL bugcheck, maybe?
Regarding IRQL’s: In the CTX mini filter example, post-create calls
CtxFindOrCreateStreamContext
which uses FltAllocateContext. When a a post-op callback is running at
Dispatch and it calls a
secondary function, does it’s IRQL remain where it’s at?
Such a basic question, but I really don’t know - hanging my head in
shame…
m
There’s a number of possibilities, the most likely of which seem to be:
This may become more of an issue in Vista, where asynchronous creates
are actually exported to applications, than in previous releases, but
verifier should catch this in IRQL checking mode (where it forces
IRQL_DISPATCH in completion.) If not, I’ve got enough random file
systems and filter drivers lying around I could probably add this
“feature”.
Perhaps it’s my own conceptual mistake, but I’ve always treated filter
manager as a library and assumed that I have to follow the standard
documented I/O Manager rules about IRQL in dispatch versus completion.
Of course, even if this isn’t an IRQL problem, I still don’t like
dealing with the failure cases when the allocation fails in completion -
which is why I do it in dispatch.
What do those examples do when the allocation fails? Do they then
cancel the open on the file? That’s the ugliness I’ve always tried to
avoid by allocating in dispatch and handling the error there.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 18-21, 2006 (note new date - MS scheduled plugfest the
same week again.)
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Thursday, May 04, 2006 5:33 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cache
Tony Mason wrote:
I always allocate the stream context in pre-create (so I can fail the
create operation in case of allocation failure) and then pass it to the
completion routine (that’s the general paradigm I use - allocate in
dispatch, use/free in completion.) Otherwise, in completion you get
stuck with complex failure handling (allocation fails, so you then get
into the “let’s cancel the open” error handling, etc.)
I see in the doc’s *FltAllocateContext* must be running at IRQL <=
APC_LEVEL,
where Post-Create OP’s can be called at an IRQL <= Dispatch sometimes.
Thus, it looks as if some of the samples
in the wdk could result in an IRQL_NOT_LESS_OR_EQUAL bugcheck, maybe?
Regarding IRQL’s: In the CTX mini filter example, post-create calls
CtxFindOrCreateStreamContext
which uses FltAllocateContext. When a a post-op callback is running at
Dispatch and it calls a
secondary function, does it’s IRQL remain where it’s at?
Such a basic question, but I really don’t know - hanging my head in
shame…
m
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Tony Mason wrote:
What do those examples do when the allocation fails? Do they then
cancel the open on the file? That’s the ugliness I’ve always tried to
avoid by allocating in dispatch and handling the error there.
Honestly, I can’t tell you… I haven’t had one of these examples
blow-up on me… I was asking
because the doc’s didn’t seem to jive with the samples (specifically
Ctx) I was reading threw.
m
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.comLooking forward to seeing you at the next OSR File Systems class in
Boston, MA April 18-21, 2006 (note new date - MS scheduled plugfest the
same week again.)-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of MM
Sent: Thursday, May 04, 2006 5:33 AM
To: ntfsd redirect
Subject: Re: [ntfsd] Minifilter cacheTony Mason wrote:
>I always allocate the stream context in pre-create (so I can fail the
>create operation in case of allocation failure) and then pass it to the
>completion routine (that’s the general paradigm I use - allocate in
>dispatch, use/free in completion.) Otherwise, in completion you get
>stuck with complex failure handling (allocation fails, so you then get
>into the “let’s cancel the open” error handling, etc.)
>
>
>
>
I see in the doc’s *FltAllocateContext* must be running at IRQL <=
APC_LEVEL,
where Post-Create OP’s can be called at an IRQL <= Dispatch sometimes.
Thus, it looks as if some of the samples
in the wdk could result in an IRQL_NOT_LESS_OR_EQUAL bugcheck, maybe?Regarding IRQL’s: In the CTX mini filter example, post-create calls
CtxFindOrCreateStreamContext
which uses FltAllocateContext. When a a post-op callback is running at
Dispatch and it calls a
secondary function, does it’s IRQL remain where it’s at?Such a basic question, but I really don’t know - hanging my head in
shame…m
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
No. The docs are clear. "Post-create callback routines are called at IRQL
PASSIVE_LEVEL, in the context of the thread that originated the
IRP_MJ_CREATE operation. " This is distinct from completion of IRP_MJ_CREATE
in traditional (legacy) filter driver.
“MM” wrote in message news:xxxxx@ntfsd…
> Tony Mason wrote:
>
>>I always allocate the stream context in pre-create (so I can fail the
>>create operation in case of allocation failure) and then pass it to the
>>completion routine (that’s the general paradigm I use - allocate in
>>dispatch, use/free in completion.) Otherwise, in completion you get
>>stuck with complex failure handling (allocation fails, so you then get
>>into the “let’s cancel the open” error handling, etc.)
>>
> I see in the doc’s FltAllocateContext must be running at IRQL <=
> APC_LEVEL,
> where Post-Create OP’s can be called at an IRQL <= Dispatch sometimes.
> Thus, it looks as if some of the samples
> in the wdk could result in an IRQL_NOT_LESS_OR_EQUAL bugcheck, maybe?
>
> Regarding IRQL’s: In the CTX mini filter example, post-create calls
> CtxFindOrCreateStreamContext
> which uses FltAllocateContext. When a a post-op callback is running at
> Dispatch and it calls a
> secondary function, does it’s IRQL remain where it’s at?
>
> Such a basic question, but I really don’t know - hanging my head in
> shame…
>
> m
>
No, “The Docs are Clear”; what is the difference is a post-create and a
post-operation?:
"A minifilter’s postoperation callback routine performs completion
processing for one or more types of I/O operations.
Postoperation callback routines are similar to the completion routines
used by legacy file system filter drivers.
Postoperation callback routines are called in an arbitrary thread
context, at IRQL <= DISPATCH_LEVEL.
Because this callback routine can be called at IRQL DISPATCH_LEVEL, it
is subject to the following constraints:
* It cannot safely call any kernel-mode routine that must run at a
lower IRQL."
Lyndon J. Clarke wrote:
No. The docs are clear. "Post-create callback routines are called at IRQL
PASSIVE_LEVEL, in the context of the thread that originated the
IRP_MJ_CREATE operation. " This is distinct from completion of IRP_MJ_CREATE
in traditional (legacy) filter driver.“MM” wrote in message news:xxxxx@ntfsd…
>
>
>>Tony Mason wrote:
>>
>>
>>
>>>I always allocate the stream context in pre-create (so I can fail the
>>>create operation in case of allocation failure) and then pass it to the
>>>completion routine (that’s the general paradigm I use - allocate in
>>>dispatch, use/free in completion.) Otherwise, in completion you get
>>>stuck with complex failure handling (allocation fails, so you then get
>>>into the “let’s cancel the open” error handling, etc.)
>>>
>>>
>>>
>>I see in the doc’s FltAllocateContext must be running at IRQL <=
>>APC_LEVEL,
>>where Post-Create OP’s can be called at an IRQL <= Dispatch sometimes.
>>Thus, it looks as if some of the samples
>>in the wdk could result in an IRQL_NOT_LESS_OR_EQUAL bugcheck, maybe?
>>
>>Regarding IRQL’s: In the CTX mini filter example, post-create calls
>>CtxFindOrCreateStreamContext
>>which uses FltAllocateContext. When a a post-op callback is running at
>>Dispatch and it calls a
>>secondary function, does it’s IRQL remain where it’s at?
>>
>>Such a basic question, but I really don’t know - hanging my head in
>>shame…
>>
>>m
>>
>>
>>
>
>
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@comcast.net
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>