Handles, threads, and processes

NTFSD
1

Hi all,

I’m having a little trouble with my filter driver. Open receiving a Create
request for one of my files, I use ZwCreateFile on the lower file system to
open the physical file.

All goes well until my file gets a Cleanup call. Here, I flush and
uninitialise my cache map but it refuses to go as a DataSection still
exists. Is this to be expected?

The reason why this is important is that that final job of my Cleanup
routine is to call ZwClose on the physical file. If I don’t block the
Cleanup until the cache has gone, I’ll get paging writes from my cache and
will be unable to write the data to disk. (I’m doing non-paged writes to
the lower file system on a file that will have now been cleaned’up)

If I move the ZwClose call to my Close routine my paging problem is fixed,
but often, the Close call is in the context of another process, and so the
ZwClose fails. Is it possible to schedule thread in the context of the
existing process, in order to close the file?

Alternatively, in my Create routine, it it possible to create the physical
file, reference it, and close it without it sending a Cleanup message to
the file system? This will allow me to write the paged data after my
Cleanup routine has run.

My final solution was to open and close my file in a process created by my
driver and in the context of the System user. Is it possible to do this?

Many thanks,

Andy Larter

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

2

> If I move the ZwClose call to my Close routine my paging problem is fixed,

but often, the Close call is in the context of another process, and so the
ZwClose fails. Is it possible to schedule thread in the context of the
existing process, in order to close the file?

Yes, and this is the right thing.

Cc and Mm can still work with the file after CLEANUP was called.
Imagine the app calls:

CreateFile
CreateFileMapping
MapViewOfFile
CloseHandle(hFile)
CloseHandle(hMapping)

You will have CLEANUP at CloseHandle(hFile), but the mapping is still there and the app can dirty these pages. Then MM will send
writes to your FSD writing them to disk. You will not be able to write if you have closed the underlying file in CLEANUP.

Max

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

3

> > If I move the ZwClose call to my Close routine my paging problem is fixed,

> but often, the Close call is in the context of another process, and so the
> ZwClose fails. Is it possible to schedule thread in the context of the
> existing process, in order to close the file?

Yes, and this is the right thing.

Cc and Mm can still work with the file after CLEANUP was called.
Imagine the app calls:

CreateFile
CreateFileMapping
MapViewOfFile
CloseHandle(hFile)
CloseHandle(hMapping)

You will have CLEANUP at CloseHandle(hFile), but the mapping is still there and the app can dirty these pages. Then MM will send
writes to your FSD writing them to disk. You will not be able to write if you have closed the underlying file in CLEANUP.

Max

Given my test results, this was sort-of expected. The question is now, how
do I close the file? The Cleanup was almost always run in the context of
the original process, but the Close is more often not. I’m looking at
scheduling
threads or creating/closing my file in another process. Any ideas?

Andy

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

4

And do you have considered this way ?

IRP_MJ_CREATE:
ZwCreateFile, obtaining handle
ObReferenceObjectByHandle, obtaining pointer

IRP_MJ_CLEANUP:
ZwClose, closing handle (but the file still remains)

IRP_MJ_CLOSE:
ObDereferenceObject, dereferencing the pointer

Paul

PS: But I think calling the ZwCreateFile in your create
dispatch is not the best thing you can do.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of
xxxxx@ArmourSoft.com
Sent: Sunday, September 30, 2001 12:00 AM
To: File Systems Developers
Subject: [ntfsd] Re: Handles, threads, and processes

> If I move the ZwClose call to my Close routine my paging problem is
fixed,
> but often, the Close call is in the context of another process, and so
the
> ZwClose fails. Is it possible to schedule thread in the context of the
> existing process, in order to close the file?

Yes, and this is the right thing.

Cc and Mm can still work with the file after CLEANUP was called.
Imagine the app calls:

CreateFile
CreateFileMapping
MapViewOfFile
CloseHandle(hFile)
CloseHandle(hMapping)

You will have CLEANUP at CloseHandle(hFile), but the mapping is still
there and the app can dirty these pages. Then MM will send
writes to your FSD writing them to disk. You will not be able to write if
you have closed the underlying file in CLEANUP.

Max

Given my test results, this was sort-of expected. The question is now, how
do I close the file? The Cleanup was almost always run in the context of
the original process, but the Close is more often not. I’m looking at
scheduling
threads or creating/closing my file in another process. Any ideas?

Andy

You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

5

On 09/30/01, ““Pavel Hrdina” ” wrote:
> And do you have considered this way ?
>
> IRP_MJ_CREATE:
> ZwCreateFile, obtaining handle
> ObReferenceObjectByHandle, obtaining pointer
>
> IRP_MJ_CLEANUP:
> ZwClose, closing handle (but the file still remains)
>
> IRP_MJ_CLOSE:
> ObDereferenceObject, dereferencing the pointer
>
> Paul
>
> PS: But I think calling the ZwCreateFile in your create
> dispatch is not the best thing you can do.

Calling ZwCreateFile in my Create routine is the only way to make my driver
function as required.

As for using ObReferenceObjectByHandle, this is what I already do. It fails
when I call ZwClose because after this, the lower level driver will only
accept paging writes from the cache manager. I’ll get paging writes from my
cache and will be unable to write out the data as the file is now closed.
End result, data loss, and angry customer.

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

6

Yes, I know.
But when you cleanup your file (and force cleanup of
the file below by closing its handle) the file size
remains constant and there is no reason to write or
read from it in other than paging way like with your
file does the component above you.
What do you think about this ?

Paul

PS: Please tell me your reasons to call ZwCreateFile
in your create dispatch.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of
xxxxx@ArmourSoft.com
Sent: Sunday, September 30, 2001 4:57 PM
To: File Systems Developers
Subject: [ntfsd] Re: Handles, threads, and processes

On 09/30/01, ““Pavel Hrdina” ” wrote:
> And do you have considered this way ?
>
> IRP_MJ_CREATE:
> ZwCreateFile, obtaining handle
> ObReferenceObjectByHandle, obtaining pointer
>
> IRP_MJ_CLEANUP:
> ZwClose, closing handle (but the file still remains)
>
> IRP_MJ_CLOSE:
> ObDereferenceObject, dereferencing the pointer
>
> Paul
>
> PS: But I think calling the ZwCreateFile in your create
> dispatch is not the best thing you can do.

Calling ZwCreateFile in my Create routine is the only way to make my driver
function as required.

As for using ObReferenceObjectByHandle, this is what I already do. It fails
when I call ZwClose because after this, the lower level driver will only
accept paging writes from the cache manager. I’ll get paging writes from my
cache and will be unable to write out the data as the file is now closed.
End result, data loss, and angry customer.

—
You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

7

On 10/01/01, ““Pavel Hrdina” ” wrote:
> Yes, I know.
> But when you cleanup your file (and force cleanup of
> the file below by closing its handle) the file size
> remains constant and there is no reason to write or
> read from it in other than paging way like with your
> file does the component above you.
> What do you think about this ?
>
> Paul
>
> PS: Please tell me your reasons to call ZwCreateFile
> in your create dispatch.

I was originally filtering the NTFS file but this gave me problems with the
cache. Mostly, this was because I was maintaining data structures in the
file which I did not want to expose to applications. This gave me problems
maintaining the cache size due to the shared info between NTFS and the
cache manager. My solution was to move the filter above the cache manager
and maintain my own cache (used for file mapping). Applications talk to my
driver’s file, and I do normal IO to the file system. NTFS and FAT
therefore do normal (unfiltered) IO to their cache. Whilst more complex to
implement, my current solution worked well, until now.

I thought about doing paging writes directly to the file system but the
file system may (will) have its own cache. This would almost certainly give
me cache inconsistency in the lower file system. I could get around this by
creating a mapping view to the file system cache, but I could then get
double page faults if the view is paged out. Nasty.

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

8

Hi Andy,

You can use KeAttachProcess / KeDetachProcess to open/close file in the
system process context.

Alexei

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

9

This is a horrible idea.

I recommend using a thread.

Jamey
xxxxx@storagecraft.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@mondenet.com
Sent: Monday, October 01, 2001 9:26 AM
To: File Systems Developers
Subject: [ntfsd] Re: Handles, threads, and processes

Hi Andy,

You can use KeAttachProcess / KeDetachProcess to open/close file in the
system process context.

Alexei

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com To
unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

10

On 10/01/01, ““Jamey Kirby” ” wrote:
> This is a horrible idea.
>
> I recommend using a thread.
>
> Jamey
> xxxxx@storagecraft.com

Yes horrible, but not unlike the rest of my driver.

The docs for KeAttachProcess/KeStackAttachProcess only say that
asynchronous I/O will be affected. How about a doing the ZwClose as this
should only require synchronous I/O. This may be a little optimistic, but
I’ll give it a go.

Jamey, you suggested using a thread. How can this be done in the context of
another process?

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

11

Be careful, on NT4 these calls cannot be nested, so, you can have a BSOD if the code above or below you have already attached to
other process.

Max

----- Original Message -----
From:
To: “File Systems Developers”
Sent: Monday, October 01, 2001 9:25 AM
Subject: [ntfsd] Re: Handles, threads, and processes

> Hi Andy,
>
> You can use KeAttachProcess / KeDetachProcess to open/close file in the
> system process context.
>
> Alexei
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com