Blocking paging I/O IRPs

Hi devs:

My FSF application defers file reads to a server user mode application,
which reads data, decrypts it, and passes it back to my driver which in turn
passes it to the application that issued the file read request.

The problem is that, as I already expected, this scheme doesn´t work with
memory mapped files.

When an application creates a memory mapping and reads from the mapped
memory area, a page fault occurs which is caught by the memory manager. If
the requested page is not already in memory, it issues a read IRP with
IRP_PAGING_IO flag set, that arrives to my dispatch routine.

Then, I suspend the requesting thread and queue a read request for the
server process. But when the server process gets the request and tries to
fulfill it with a simple read operation, the call never returns (note that
this read also comes to my driver but it recognizes it´s coming from the
server process and lets it pass through).

The system seems to be deadlocked at that point, but only for that file,
since I can still continue performing file operations on other files on the
same drive.

So I guess that someone (the memory manager ?, the I/O manager ?) acquires
some mutex before accessing the memory mapped file, and when my server
process tries to access the same file, it deadlocks.

I knew I was going to find this problem more or less. My questions are:

1- Can anyone develop on what´s exactly happening (what´s exactly the
deadlock) ?. (just curious)

2- Does anybody know a way around this ?.

-IGP-


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hi Ignacio:

How is your (user mode) server process asking for the read? I have run into
a similar sounding problem when my “server” process does an ordinary read; I
deduced it was a deadlock with the cache. When I changed the server process
to use non-buffered I/O, everything was fine.

It appeared to me that you can’t do a cached read to satisfy a read request
from the cache manager.

Curt

-----Original Message-----
From: Ignacio Garcia-Perez [mailto:xxxxx@trymedia.com]
Sent: Friday, February 02, 2001 11:30 AM
To: File Systems Developers
Subject: [ntfsd] Blocking paging I/O IRPs

Hi devs:

My FSF application defers file reads to a server user mode
application,
which reads data, decrypts it, and passes it back to my
driver which in turn
passes it to the application that issued the file read request.

The problem is that, as I already expected, this scheme
doesn?t work with
memory mapped files.

When an application creates a memory mapping and reads from the mapped
memory area, a page fault occurs which is caught by the
memory manager. If
the requested page is not already in memory, it issues a read IRP with
IRP_PAGING_IO flag set, that arrives to my dispatch routine.

Then, I suspend the requesting thread and queue a read request for the
server process. But when the server process gets the request
and tries to
fulfill it with a simple read operation, the call never
returns (note that
this read also comes to my driver but it recognizes it?s
coming from the
server process and lets it pass through).

The system seems to be deadlocked at that point, but only for
that file,
since I can still continue performing file operations on
other files on the
same drive.

So I guess that someone (the memory manager ?, the I/O
manager ?) acquires
some mutex before accessing the memory mapped file, and when my server
process tries to access the same file, it deadlocks.

I knew I was going to find this problem more or less. My
questions are:

1- Can anyone develop on what?s exactly happening (what?s exactly the
deadlock) ?. (just curious)

2- Does anybody know a way around this ?.

-IGP-


You are currently subscribed to ntfsd as:
xxxxx@omnishift.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> My FSF application defers file reads to a server user mode application,

which reads data, decrypts it, and passes it back to my driver which in
turn
passes it to the application that issued the file read request.

The problem is that, as I already expected, this scheme doesn´t work with
memory mapped files.

I have exactly the same design in one of my current projects (though it is
not encryption) - and no problems with MM files, I can run executables from
the filtered volume.
Looks like you have some bug.

Max


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I know now exactly what was happening. Either the I/O manager or the memory
manager acquire exclusive access to the file while the IRP_PAGING_IO IRP is
being processed. My driver blocks the request and passes it to the server
process, which in turn has to make some reads from the same file in order to
serve the original read. Since it cannot access the file, a deadlock occurs.

The solution is, since anyway the exclusive access has been acquired for my
server process, do the reads instead of through the normal Win32 API,
through the driver.

-IGP-

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Maxim S. Shatskih
Sent: domingo, 04 de febrero de 2001 2:07
To: File Systems Developers
Subject: [ntfsd] Re: Blocking paging I/O IRPs

> My FSF application defers file reads to a server user mode application,
> which reads data, decrypts it, and passes it back to my driver which in
turn
> passes it to the application that issued the file read request.
>
> The problem is that, as I already expected, this scheme doesn´t
work with
> memory mapped files.

I have exactly the same design in one of my current projects (though it is
not encryption) - and no problems with MM files, I can run
executables from
the filtered volume.
Looks like you have some bug.

Max


You are currently subscribed to ntfsd as: xxxxx@trymedia.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

RE: [ntfsd] Blocking paging I/O IRPsMy user mode process uses the WIN32 API.
Actually through the C runtime (fopen and relatives).

It turned out to be that either the I/O manager or memory manager acquire
exclusive access to the file when the second issues the read IRP to fullfill
a page fault caused by the application accessing a non-loaded page in the
memory mapped file.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of
xxxxx@omnishift.com
Sent: viernes, 02 de febrero de 2001 21:28
To: File Systems Developers
Subject: [ntfsd] RE: Blocking paging I/O IRPs

Hi Ignacio:

How is your (user mode) server process asking for the read? I have run
into a similar sounding problem when my “server” process does an ordinary
read; I deduced it was a deadlock with the cache. When I changed the server
process to use non-buffered I/O, everything was fine.

It appeared to me that you can’t do a cached read to satisfy a read
request from the cache manager.

Curt

-----Original Message-----
> From: Ignacio Garcia-Perez [mailto:xxxxx@trymedia.com]
> Sent: Friday, February 02, 2001 11:30 AM
> To: File Systems Developers
> Subject: [ntfsd] Blocking paging I/O IRPs
>
>
> Hi devs:
>
> My FSF application defers file reads to a server user mode
> application,
> which reads data, decrypts it, and passes it back to my
> driver which in turn
> passes it to the application that issued the file read request.
>
> The problem is that, as I already expected, this scheme
> doesn´t work with
> memory mapped files.
>
> When an application creates a memory mapping and reads from the mapped
> memory area, a page fault occurs which is caught by the
> memory manager. If
> the requested page is not already in memory, it issues a read IRP with
> IRP_PAGING_IO flag set, that arrives to my dispatch routine.
>
> Then, I suspend the requesting thread and queue a read request for the
> server process. But when the server process gets the request
> and tries to
> fulfill it with a simple read operation, the call never
> returns (note that
> this read also comes to my driver but it recognizes it´s
> coming from the
> server process and lets it pass through).
>
> The system seems to be deadlocked at that point, but only for
> that file,
> since I can still continue performing file operations on
> other files on the
> same drive.
>
> So I guess that someone (the memory manager ?, the I/O
> manager ?) acquires
> some mutex before accessing the memory mapped file, and when my server
> process tries to access the same file, it deadlocks.
>
>
> I knew I was going to find this problem more or less. My
> questions are:
>
> 1- Can anyone develop on what´s exactly happening (what´s exactly the
> deadlock) ?. (just curious)
>
> 2- Does anybody know a way around this ?.
>
> -IGP-
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@omnishift.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntfsd as: xxxxx@trymedia.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com