Practical Paging Picks A Peck of Pickled Pages

Daniel_Van_Der_Werken · November 8, 2006, 6:57pm

So, is this right?
File_OBJECT Tail.Overlay.OriginalFileObject.Filename.Buffer may or may not be correct in the read and write dispatch routines of a legacy file system filter driver for ntfs on winxp?

And, is this also right?
I want to get the file name that will be correct from the create dispatch routine and save it out?

Is a lookaside list a good idea? Yes. It needs to be serialized. I have no idea how to serialize it, but I’m guessing I can find an example somewhere.

If I use a lookaside list, should I use Paged or NPaged?
ExInitializeNPagedLookasideList is what I’m thinking I’ll start my Googling with.

Good idea or bad?

I’m still working late into the evening.

OSR_Community_User · November 8, 2006, 7:14pm

In reference to your previous email, how do you know that you are not
getting the IO buffer? You are retrieving a valid buffer, correct? Is it for
the right file?

For the below information, the only time you should ever access

pIrpStack->FileObject->FileName

(Use the above, not how you are doing it below)

is in the IRP_MJ_CREATE dispatch side. You should allocate a context, push
in the filename and send it to the completion side of create. If the create
fails, free up the context. If it is successful, add the context to your
list indexed on the pIrpStack->FileObject->FsContext value. Use this value
as an index, don’t try and read into the structure. Also, be sure that you
correctly add the context only when it does not already exist and be sure to
correctly reference count the context, there is a great article on reference
counting at osronline.com. The reason why reference counting is important is
that you will see IO requests with an FsContext that matches one in your
list but you never received an IRP_MJ_CREATE for the reference, hence you
would never have referenced the node, then you would prematurely free it in
IRP_MJ_CLOSE processing and thus miss paging IO requests against it.

Pete

Kernel Drivers
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Wednesday, November 08, 2006 3:56 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Practical Paging Picks A Peck of Pickled Pages

So, is this right?
File_OBJECT Tail.Overlay.OriginalFileObject.Filename.Buffer may or may not
be correct in the read and write dispatch routines of a legacy file system
filter driver for ntfs on winxp?

And, is this also right?
I want to get the file name that will be correct from the create dispatch
routine and save it out?

Is a lookaside list a good idea? Yes. It needs to be serialized. I have
no idea how to serialize it, but I’m guessing I can find an example
somewhere.

If I use a lookaside list, should I use Paged or NPaged?
ExInitializeNPagedLookasideList is what I’m thinking I’ll start my Googling
with.

Good idea or bad?

I’m still working late into the evening.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · November 8, 2006, 7:25pm

I shall start to side with Dan Burn soon
OT: A word of advice: while in this area, we tend to go berserk sometimes, too many funny words tend to keep your focus off the subject. Trust me, been there, done that - it will
help if you name your subjects in a practical way, and construct complete questions.

For contexts. Where do you intend to use them? Read/write. Can you access paged memory during paging I/O? No, because it may lead to a page fault, and possibly to an endless
recursion.
Serialize - means synchronize.
Can lookasides be used. Are the allocations of the same size? No, as the file names are definitely of different sizes. Certain optimizations can be made here, but you should not
consider them, as when you port to the mini-filter world, it has to be implemented a bit differently.

Dejan.

xxxxx@hotmail.com wrote:

So, is this right?
File_OBJECT Tail.Overlay.OriginalFileObject.Filename.Buffer may or may not be correct in the read and write dispatch routines of a legacy file system filter driver for ntfs on winxp?

And, is this also right?
I want to get the file name that will be correct from the create dispatch routine and save it out?

Is a lookaside list a good idea? Yes. It needs to be serialized. I have no idea how to serialize it, but I’m guessing I can find an example somewhere.

If I use a lookaside list, should I use Paged or NPaged?
ExInitializeNPagedLookasideList is what I’m thinking I’ll start my Googling with.

Good idea or bad?

I’m still working late into the evening.

Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

–
King regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

Daniel_Van_Der_Werken · November 8, 2006, 7:59pm

I really don’t know where I’ll used the contexts. I’m not even sure what I will use for contexts yet.

The filespy sample has contexts.

The sfilter sample uses a paged lookaside list for holding names.

Since both apps are interested in saving out the names and dumping them, similar to filemon, etc. and it seemed a more direct way to get at the file name is via the Irp, I didn’t spend much time looking after this.

From what I understand, you can NOT access page pool memory in the read or write dispatch routines. I don’t know anything about this in create or such. Can you access page pooled memory in a create dispatch? I don’t know. Anyone?

Seems safest to just use nonpaged pool thoughout. Well? I know it’s limited, but it seems like at least for a quick start, that would be a safe bet.

Didn’t I see that the default for nonpaged pool memory for winxp is 256MB!

Heck, that should get me through a demo if I am careful not to open any really big files! Really. I have no idea. Seems like the right way to do this is only use nonpaged pool when absolutely necessary. I just don’t know exactly when that is.

Hence, I asked this question. I’m going do do an encryption filter. i’m going to get a context of sorts and attach it to the file in the create dispatch routine. Do I used paged pooled or nonpaged pool?

In the sfilter example, the lookasides are page pooled and they are the size of a structure that contains items including a 254 byte character array which should hold most filenames but can grow bigger if need be (the array).

So, while I thought my question was simple, I still feel it’s not answered. Thanks.

OSR_Community_User · November 8, 2006, 8:35pm

Some answers:

The Create is always at PASSIVE level, so paged pool is OK. But it’s an
exception.
In general, other operations are at arbitrary levels, so paged pool is
*not* OK.
If your context is to be used at any level below APC_LEVEL, then use
non-paged pool.
A lot of operations will require you to be at APC_LEVEL or PASSIVE_LEVEL.
Learn about worker threads – you’ll need them.
Look at SwapPostReadBuffers and SwapPostReadBuffersWhenSafe in
swapBuffers.c (yes, the mini-filter sample). It shows how to swap user
buffers dynamically. It may or may not help you, but…

Ken

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Wednesday, November 08, 2006 7:59 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Practical Paging Picks A Peck of Pickled Pages

I really don’t know where I’ll used the contexts. I’m not even sure what I
will use for contexts yet.

The filespy sample has contexts.

The sfilter sample uses a paged lookaside list for holding names.

Since both apps are interested in saving out the names and dumping them,
similar to filemon, etc. and it seemed a more direct way to get at the file
name is via the Irp, I didn’t spend much time looking after this.

From what I understand, you can NOT access page pool memory in the read or
write dispatch routines. I don’t know anything about this in create or
such. Can you access page pooled memory in a create dispatch? I don’t
know. Anyone?

Seems safest to just use nonpaged pool thoughout. Well? I know it’s
limited, but it seems like at least for a quick start, that would be a safe
bet.

Didn’t I see that the default for nonpaged pool memory for winxp is 256MB!

Heck, that should get me through a demo if I am careful not to open any
really big files! Really. I have no idea. Seems like the right way to do
this is only use nonpaged pool when absolutely necessary. I just don’t know
exactly when that is.

Hence, I asked this question. I’m going do do an encryption filter. i’m
going to get a context of sorts and attach it to the file in the create
dispatch routine. Do I used paged pooled or nonpaged pool?

In the sfilter example, the lookasides are page pooled and they are the size
of a structure that contains items including a 254 byte character array
which should hold most filenames but can grow bigger if need be (the array).

So, while I thought my question was simple, I still feel it’s not answered.
Thanks.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · November 8, 2006, 9:18pm

Minor corrections:

In general, other operations are at arbitrary levels, so paged pool is
*not* OK.

> The DISPATCH side will always be called at or below APC level in the
filesystem stack. The COMPLETION side can be called at or below DISPATCH
level.

If your context is to be used at any level below APC_LEVEL, then use
non-paged pool.

> This should read ‘at or below’ APC level is OK to use paged pool,
DISPATCH requires non-paged pool.

A lot of operations will require you to be at APC_LEVEL or PASSIVE_LEVEL.
Learn about worker threads – you’ll need them.

> Most Zwxxx API’s require you to be at PASSIVE level.

Pete

Kernel Drivers
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Wednesday, November 08, 2006 5:34 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Practical Paging Picks A Peck of Pickled Pages

Some answers:

The Create is always at PASSIVE level, so paged pool is OK. But it’s an
exception.
In general, other operations are at arbitrary levels, so paged pool is
*not* OK.
If your context is to be used at any level below APC_LEVEL, then use
non-paged pool.
A lot of operations will require you to be at APC_LEVEL or PASSIVE_LEVEL.
Learn about worker threads – you’ll need them.
Look at SwapPostReadBuffers and SwapPostReadBuffersWhenSafe in
swapBuffers.c (yes, the mini-filter sample). It shows how to swap user
buffers dynamically. It may or may not help you, but…