You will not always see an IRP_MJ_CREATE for a new file object. NTFS often creates file objects using the function IoCreateStreamFileObject() or IoCreateStreamFileObjectLite(). These bypass IRP_MJ_CREATE, but allow the file system to create a viable file object that references the FsContext. You will see an IRP_MJ_CLOSE for these in spite of not seeing an IRP_MJ_CREATE. With your current solution, you will decrement the reference count on your context prematurely due to these special “stream” file objects. You need to track the file objects that you see on IRP_MJ_CREATE so you know whether or not to decrement the reference count on your context during IRP_MJ_CLOSE.
-----Original Message-----
From: Stefan Boboc [mailto:SBoboc@ps.ro]
Sent: Friday, April 19, 2002 10:43 AM
To: File Systems Developers
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ
or IRP_MJ_WRITE ?
I decrement the reference count for FsContext in IRP_MJ_CLOSE before I send down the IRP. If the reference count is zero then I remove the FsContext from my hash table also BEFORE I send down the IRP and in this case before FSD free the FsContext! So, when FSD will associate FsContext with another file object, it will be another IRP_MJ_CREATE, so I will track again the FsContext for the new opened file!
Is correct or not?
-----Original Message-----
From: Fuller, Rob [mailto:xxxxx@inin.com]
Sent: Friday, April 19, 2002 4:16 PM
To: File Systems Developers
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ
or IRP_MJ_WRITE ?
You need to track the file objects so you can decrement the reference count on your context in order to free it when the FsContext goes away. FsContext is likely to get used for a different file after it is freed.
-----Original Message-----
From: Stefan Boboc [mailto:SBoboc@ps.ro]
Sent: Friday, April 19, 2002 8:39 AM
To: File Systems Developers
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ
or IRP_MJ_WRITE ?
Thanks Graig,
First I want to clear my ideas about file tracking and file path name.
For getting the full path name I’m using FilemonGetFullPathName in IRP_MJ_CREATE dispatch routine (before I call the next driver).
For tracking I’m using hash table (similar like the one used in Filemon but for FsContext instead of FileObject). Why I need to keep track for FileObjects? Because all the FileObjects will reference the same FsContext, it is not enough to track FsContext?
Actually, all the operations that I made on IRP_MJ_CREATE (the flush & purge file and tracking FsContext) are done on dispatch routine not in completion routine. This is done by waiting for an event set by the completion routine. It is OK with this?
Stefan
-----Original Message-----
From: David J. Craig [mailto:xxxxx@yoshimuni.com]
Sent: Thursday, April 18, 2002 9:31 PM
To: File Systems Developers
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ
or IRP_MJ_WRITE ?
You message is missing a lot of details necessary to provide missing
information. The bits you need are in pIrp->Flags: IRP_PAGING_IO &
IRP_SYNCHRONOUS_PAGING_IO. Tracking should be done by FsContext that is
captured during IRP_MJ_CREATE’s completion routine. The file name elements
must be captured during IRP_MJ_CREATE’s dispatch routine. You must also
track every FileObject referencing that FsContext and you might want to take
note of the FileObject being used by the cache manager for paging I/O. I
can’t think of much you need to do during IRP_MJ_CLEANUP, but you want to
dereference any objects when IRP_MJ_CLOSE is processed.
You need to flush the cache correctly and also purge the cache. They are
NOT the same and I think that both should be done during an IRP_MJ_CREATE’s
completion routine. Both NTFS & FASTFAT call the completion routine at IRQL
PASSIVE_LEVEL so if you set the right bit in the FileObject, you can use
that FileObject to do any other needed requests, e.g. attributes, full path
name, etc.
There are a couple of other problems you will have later dealing with files
that aren’t cached and network paths, but get this working first. On
creates you can force any write only open to be read/write so the cache
manager will participate.
----- Original Message -----
From: “Stefan Boboc”
To: “File Systems Developers”
Sent: Thursday, April 18, 2002 3:36 AM
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ or
IRP_MJ_WRITE ?
thanks for your help,
My filter wants to be a regular file filer driver (for local calls). Like
many of you, I want encryption/decryption for my protected files (from my
protected directory on the hard disk). Of course I want to offer the
functionality that you described, but first things first:-)
this is what i’m doing:
After a successful IRP_MJ_CREATE for a file in my protected dir, I flush the
cache for that file.
On IRP_MJ_READ and WRITE I allocate my own MDL for encrypt/decrypt data only
when I receive IRP->Flags & (IRP_NOCACHE | IRP_PAGING_IO |
IRP_ASYNCRONOUS_IO)
Also, I’m flushing the cache during IRP_MJ_CLEANUP and I see the
(IRP_NOCACHE | IRP_PAGING_IO | IRP_ASYNCRONOUS_IO) set during
IRP_MJ_READ/WRITE so I can encrypt the data during file copy operation.
For keeping the track of my files I’m using FsContext and my own reference
counter (after a successful IRP_MJ_CREATE I check the FsContext and If this
is already in my tracking array I increment the counter reference, else I
add the new FsContext in my tracking array)
On the IRP_MJ_CREATE I check also the FsContext and if I found it in my
tracking array and only if my counter reference is zero, I remove the
FsContext from my tracking array, else I only decrement the counter
reference.
It this the correct approach or I have to keep in mind other stuff? Because
it seams that during open a doc file my system hangs.
Btw. I’m checking the FasIoRead and FastIoWrite and I’m returning FALSE from
these.
Regards,
Stefan
-----Original Message-----
From: Sara Abraham [mailto:xxxxx@veritas.com]
Sent: Wednesday, April 17, 2002 11:08 PM
To: File Systems Developers
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ
or IRP_MJ_WRITE ?
Hi,
Here is my understanding of the problem, let me know if this is correct:
Your filter wants to see all the write requests that the redirector is
sending to the server, so that it could encrypt the data before it goes out
on the LAN. Your filter is on top of the redirector so you see user requests
before the redirector sees those requests. If the IRP has IRP_NOCACHE, you
know that the redirector is not going to cache, and you encrypt the data
before the data is passed down to the redirector. When IRP_NOCACHE is 0, you
assume that the redirector would cache and you don’t want to encrypt the
data. You expect an IRP_PAGING_IO when the data is flushed.
The problem is that even though a user issues a buffered i/o, there is no
guarantee that the redirector would cache the data. When your filter sees
the buffered request, it cannot predict if the redirector would end up
caching the data. The redirector’s decision is based on file attributes,
open options and non-paged pool size.
So, if my understanding is correct, you have a problem because your filter
is a redirector filter, but you don’t want to encrypt data that the
redirector would end up caching, you want to encrypt the data just before
it’s going out on the LAN.
Sara
----- Original Message -----
From: “Nicholas Ryan”
To: “File Systems Developers”
Sent: Wednesday, April 17, 2002 12:51 PM
Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE during IRP_MJ_READ or
IRP_MJ_WRITE ?
> Well, there may not be another create request coming
> along for a while, so you’d still have to wait for the
> lazy writer. The cleanup request, however, comes down
> as soon as the last USER handle to the file is closed,
> so it would be sent down immediately after an
> application is done copying/writing/modifying the
> file. If you flush here, you ensure that the data the
> application changed in the cache is sent to you ASAP
> as paging I/O so you can quickly encrypt it and put it
> on disk.
>
> — Stefan Boboc wrote:
> > It is not enough to flush the cache for the every
> > new open instance in IRP_MJ_CREATE? This is what I’m
> > doing but it seams that it don’t work!
> > Stefan
> >
> > -----Original Message-----
> > From: Nicholas Ryan [mailto:xxxxx@nryan.com]
> > Sent: Wednesday, April 17, 2002 9:40 AM
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Why I don’t see any IRP_NOCACHE
> > during IRP_MJ_READ
> > or IRP_MJ_WRITE ?
> >
> >
> > The paging I/O requests will come down eventually,
> > it
> > just takes time for the lazy writer to flush them
> > from
> > the cache. (Please read Nagar’s NT filesystems book,
> > it explains all this in detail). You can always
> > manually flush the cache for a given file in your
> > IRP_MJ_CLEANUP handler if you want to see these
> > requests sooner. But you will see them eventually in
> > any case (for FAT and NTFS at least), always before
> > you see the last IRP_MJ_CLOSE for the file. If you
> > aren’t then there’s a bug in your code.
> >
> > — Stefan Boboc wrote:
> > > Thanks Sara,
> > > In this case what is the best method to encrypt
> > the
> > > files during copy operation into my protected dir?
> > > I’ve tried checking Irp->Flags & (IRP_NOCACHE |
> > > IRP_PAGING_IO | IRP_SYNCHRONOUS_PAGING_IO) and
> > this
> > > is FALSE for doc and exe files! It works for txt
> > > files but not for doc and exe… why?
> > > Stefan
> > >
> > > -----Original Message-----
> > > From: Sara Abraham [mailto:xxxxx@veritas.com]
> > > Sent: Tuesday, April 16, 2002 11:03 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Why I don’t see any
> > IRP_NOCACHE
> > > during IRP_MJ_READ
> > > or IRP_MJ_WRITE ?
> > >
> > >
> > > Hi Stefan,
> > >
> > > Most of the windows copy programs issue buffered
> > > i/o. So, if you are using
> > > one of those, then you are seeing the correct
> > > behavior. If you write a small
> > > test program and issue the CreateFile() with
> > > FILE_FLAG_NO_BUFFERING, you’ll
> > > see the IRP_NOCACHE being set.
> > >
> > > Sara
> > >
> > > ----- Original Message -----
> > > From: “Stefan Boboc”
> > > To: “File Systems Developers”
> >
> > > Sent: Tuesday, April 16, 2002 7:31 AM
> > > Subject: [ntfsd] Why I don’t see any IRP_NOCACHE
> > > during IRP_MJ_READ or
> > > IRP_MJ_WRITE ?
> > >
> > >
> > > Hi all,
> > > I’m trying to develop a filter driver for
> > > encryption/decryption. I’m using
> > > FsContext as file identifier during IRP_MJ_READ
> > and
> > > IRP_MJ_WRITE.
> > > During these IRPs, I’m checking the Irp->Flags &
> > > IRP_NOCACHE and it seams
> > > that this operation is never TRUE. What I’m doing
> > > wrong?
> > > I’m using an array of FsContext and a counter for
> > > this array. On the
> > > IRP_MJ_CREATE completion routine I increment the
> > > counter if FsContext is not
> > > already in my array and during the IRP_MJ_CLOSE
> > I’m
> > > decrementing the counter
> > > and delete the FsContext from my array.
> > > Why I don’t see any IRP_NOCACHE set during
> > > IRP_MJ_READ and IRP_MJ_WRITE?
> > > I’m using windows NT 4.0 sp 6 and FileMon as
> > > framework! I tested by copying
> > > txt, doc and exe files in my protected dir and all
> > > of this seams to have
> > > FO_CACHE_SUPPORTED flag set on file object.
> > > Also, during FastIo I return FALSE if the
> > FsContext
> > > of the File Object is in
> > > my array in order to force the I/O Manger to pass
> > > down a regular IRP!
> > > What I’m doing wrong?
> > >
> > > Any help will be appreciated,
> > > Stefan
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@veritas.com
> > > To unsubscribe send a blank email to
> > %%email.unsub%%
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > sboboc@ps.ro
> > > To unsubscribe send a blank email to
> > %%email.unsub%%
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@nryan.com
> > > To unsubscribe send a blank email to
> > %%email.unsub%%
> >
> >
> > =====
> > - Nicholas Ryan
> >
> >
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> >
> > —
> > You are currently subscribed to ntfsd as:
> > sboboc@ps.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@nryan.com
> > To unsubscribe send a blank email to
> %%email.unsub%%
>
>
> =====
> - Nicholas Ryan
>
>
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/
>
> —
> You are currently subscribed to ntfsd as: xxxxx@veritas.com
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntfsd as: sboboc@ps.ro
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: xxxxx@yoshimuni.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: sboboc@ps.ro
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: sboboc@ps.ro
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%