Problem with 'ZwReadFile' returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish


Do you Yahoo!?
Free online calendar with sync to Outlook™.

Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com

I am passing Event Handle to ZwReadFile but I am passing the pointer of the event object to KeWaitForSingle object and the pointer is obtained using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is not being invoked in the context of the thread which issued the ZwReadFile call. To verify this I tried to put a break point in WinDbg on ‘IopCompleteRequest’ specific for the thread and process. The break point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point specific to a process and thread. So I am not sure if I am not setting the break point correctly or if the routine is not really getting invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because, if I understand correctly, this routine is the place where the event that I passed to ZwReadFile should be set when the NTFS FSD (or the underlying HDD driver) ‘completes’ the request using ‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is always invoked in the context of the original thread that caused the IO request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there is no need to queue the kernel mode APC (this route may decide NOT to queue the kernel mode APC based on certain conditions), this APC being the one through which the ‘IopCompleteRequest’ runs in the context of the original thread.

Is there a way to set break points to see if the APC is queued or if the ‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

---------------------------------
Do you Yahoo!?
Free online calendar with sync to Outlook™.

For synchronous operations the I/O manager will optimize out the
scheduling of the kernel APC to the original thread (this is what the
IRP_DEFER_IO_COMPLETION flag is for). Since the request is synchronous,
the requesting thread is still blocked, so when it wakes it will call
IopCompleteRequest directly. At least, this is what happens unless
something goes horribly wrong (bogus parameter of some sort, bad IRQL,
lower driver hangs, etc.)

The documentation for ZwReadFile states that you shouldn’t be using the
EventHandle parameter anyways. Instead, you can wait on the file handle
itself to be signaled:

“If the caller opened the file with the DesiredAccess SYNCHRONIZE flag
set, the caller can wait for this routine to set the given FileHandle to
the signaled state.”

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 1:20 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

I am passing Event Handle to ZwReadFile but I am passing the pointer of
the event object to KeWaitForSingle object and the pointer is obtained
using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is
not being invoked in the context of the thread which issued the
ZwReadFile call. To verify this I tried to put a break point in WinDbg
on ‘IopCompleteRequest’ specific for the thread and process. The break
point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point
specific to a process and thread. So I am not sure if I am not setting
the break point correctly or if the routine is not really getting
invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because,
if I understand correctly, this routine is the place where the event
that I passed to ZwReadFile should be set when the NTFS FSD (or the
underlying HDD driver) ‘completes’ the request using
‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is
always invoked in the context of the original thread that caused the IO
request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there
is no need to queue the kernel mode APC (this route may decide NOT to
queue the kernel mode APC based on certain conditions), this APC being
the one through which the ‘IopCompleteRequest’ runs in the context of
the original thread.

Is there a way to set break points to see if the APC is queued or if the
‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com

Hi Nick,

I understand what you are saying about ‘inherently synchronous’ calls where the optimization is done by the I/O manager.

But, in my case w.r.t. ZwReadFile, I am not sure if we can say that the request was treated as ‘synchronous’.

Here is my analysis.

The fact that I am getting STATUS_PENDING tells me that none of the layers that were involved considered it to be a ‘synchronous’ request.

When I say layers, I mean the following (I am assuming that there is no filtering going on, which is true in my case).

Top layer - The code for the function ZwReadFile/NtReadFile or the System entry point. This is the layer, which I believe is the one, which created the IRP and issues the 1st ‘IoCallDriver’ to the NTFS FSD. This I guess is also the ‘I/O Manager’ layer.

Second Layer - This is the NTFS FSD IRP_MJ_READ entry point. This is the code that interacts with the cache manager and ‘Completes’ the request (after a possible recursive ‘IRP_MJ_READ’ because of the fact that data is not in the cache).

The fact that I got ‘STATUS_PENDING’, I believe, means that none of the above two layers interpreted the request as a ‘inherently synchronous’ request and hence did not block and allowed the STATUS_PENDING to be returned all the way back to the caller of ‘ZwReadFile’

When I say ‘inherently synchronous’, what I mean is that, there are some functions/API’s in the Top layer mentioned above (for example some implemented by I/O Manager), that make the API ‘inherently synchronous’, which means that the I/O manager code checks for the returned code and if it is STATUS_PENDING blocks using a similar event mechanism.

If my understanding above is correct, then none of the layers mentioned above treated it as a ‘synchronous’ request and hence I got STATUS_PENDING.

Even if the above is wrong, ‘IopCompleteRequest’ should still be invoked and my event set.

I tried using the File Handle with ‘Synchronize’ attribute set (at the time of opening the handle), but that did not work.

Regards,

Manish

Nick Ryan wrote:
For synchronous operations the I/O manager will optimize out the
scheduling of the kernel APC to the original thread (this is what the
IRP_DEFER_IO_COMPLETION flag is for). Since the request is synchronous,
the requesting thread is still blocked, so when it wakes it will call
IopCompleteRequest directly. At least, this is what happens unless
something goes horribly wrong (bogus parameter of some sort, bad IRQL,
lower driver hangs, etc.)

The documentation for ZwReadFile states that you shouldn’t be using the
EventHandle parameter anyways. Instead, you can wait on the file handle
itself to be signaled:

“If the caller opened the file with the DesiredAccess SYNCHRONIZE flag
set, the caller can wait for this routine to set the given FileHandle to
the signaled state.”

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 1:20 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

I am passing Event Handle to ZwReadFile but I am passing the pointer of
the event object to KeWaitForSingle object and the pointer is obtained
using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is
not being invoked in the context of the thread which issued the
ZwReadFile call. To verify this I tried to put a break point in WinDbg
on ‘IopCompleteRequest’ specific for the thread and process. The break
point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point
specific to a process and thread. So I am not sure if I am not setting
the break point correctly or if the routine is not really getting
invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because,
if I understand correctly, this routine is the place where the event
that I passed to ZwReadFile should be set when the NTFS FSD (or the
underlying HDD driver) ‘completes’ the request using
‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is
always invoked in the context of the original thread that caused the IO
request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there
is no need to queue the kernel mode APC (this route may decide NOT to
queue the kernel mode APC based on certain conditions), this APC being
the one through which the ‘IopCompleteRequest’ runs in the context of
the original thread.

Is there a way to set break points to see if the APC is queued or if the
‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

---------------------------------
Do you Yahoo!?
Free online calendar with sync to Outlook™.

I think truly synchronous requests are an attribute of Win32 and not NT. From my reading of the DDK, the handle given by ZwCreateFile() can be waited on for IOs to complete if the correct options are set. It does not say that the call to ZwReadFile() will not return.
----- Original Message -----
From: Manish Apte
To: File Systems Developers
Sent: Tuesday, June 10, 2003 5:12 PM
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

Hi Nick,

I understand what you are saying about ‘inherently synchronous’ calls where the optimization is done by the I/O manager.

But, in my case w.r.t. ZwReadFile, I am not sure if we can say that the request was treated as ‘synchronous’.

Here is my analysis.

The fact that I am getting STATUS_PENDING tells me that none of the layers that were involved considered it to be a ‘synchronous’ request.

When I say layers, I mean the following (I am assuming that there is no filtering going on, which is true in my case).

Top layer - The code for the function ZwReadFile/NtReadFile or the System entry point. This is the layer, which I believe is the one, which created the IRP and issues the 1st ‘IoCallDriver’ to the NTFS FSD. This I guess is also the ‘I/O Manager’ layer.

Second Layer - This is the NTFS FSD IRP_MJ_READ entry point. This is the code that interacts with the cache manager and ‘Completes’ the request (after a possible recursive ‘IRP_MJ_READ’ because of the fact that data is not in the cache).

The fact that I got ‘STATUS_PENDING’, I believe, means that none of the above two layers interpreted the request as a ‘inherently synchronous’ request and hence did not block and allowed the STATUS_PENDING to be returned all the way back to the caller of ‘ZwReadFile’

When I say ‘inherently synchronous’, what I mean is that, there are some functions/API’s in the Top layer mentioned above (for example some implemented by I/O Manager), that make the API ‘inherently synchronous’, which means that the I/O manager code checks for the returned code and if it is STATUS_PENDING blocks using a similar event mechanism.

If my understanding above is correct, then none of the layers mentioned above treated it as a ‘synchronous’ request and hence I got STATUS_PENDING.

Even if the above is wrong, ‘IopCompleteRequest’ should still be invoked and my event set.

I tried using the File Handle with ‘Synchronize’ attribute set (at the time of opening the handle), but that did not work.

Regards,

Manish

Nick Ryan wrote:

For synchronous operations the I/O manager will optimize out the
scheduling of the kernel APC to the original thread (this is what the
IRP_DEFER_IO_COMPLETION flag is for). Since the request is synchronous,
the requesting thread is still blocked, so when it wakes it will call
IopCompleteRequest directly. At least, this is what happens unless
something goes horribly wrong (bogus parameter of some sort, bad IRQL,
lower driver hangs, etc.)

The documentation for ZwReadFile states that you shouldn’t be using the
EventHandle parameter anyways. Instead, you can wait on the file handle
itself to be signaled:

“If the caller opened the file with the DesiredAccess SYNCHRONIZE flag
set, the caller can wait for this routine to set the given FileHandle to
the signaled state.”

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 1:20 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

I am passing Event Handle to ZwReadFile but I am passing the pointer of
the event object to KeWaitForSingle object and the pointer is obtained
using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is
not being invoked in the context of the thread which issued the
ZwReadFile call. To verify this I tried to put a break point in WinDbg
on ‘IopCompleteRequest’ specific for the thread and process. The break
point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point
specific to a process and thread. So I am not sure if I am not setting
the break point correctly or if the routine is not really getting
invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because,
if I understand correctly, this routine is the place where the event
that I passed to ZwReadFile should be set when the NTFS FSD (or the
underlying HDD driver) ‘completes’ the request using
‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is
always invoked in the context of the original thread that caused the IO
request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there
is no need to queue the kernel mode APC (this route may decide NOT to
queue the kernel mode APC based on certain conditions), this APC being
the one through which the ‘IopCompleteRequest’ runs in the context of
the original thread.

Is there a way to set break points to see if the APC is queued or if the
‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do you Yahoo!?
Free online calendar with sync to Outlook™. — Y ou are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

------------------------------------------------------------------------------
Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently subscribed to ntfsd as: xxxxx@yoshimuni.com To unsubscribe send a blank email to xxxxx@lists.osr.com

You’re right, the request is not inherently synchronous if you aren’t
forcing it to be so by specifying one of the FILE_SYNCHRONOUS_IO* flags
in CreateOptions. If you specify one of FILE_SYNCHRONOUS_IO* (you must
still also ask for SYNCHRONIZE access), the system will wait on the file
handle on your behalf before returning from any of the Zw* file I/O
APIs, so you will never see STATUS_PENDING. If you don’t specify one of
those flags, you may still ask for SYNCHRONIZE access and wait on the
file handle yourself.

You’re saying that you are asking for SYNCHRONIZE, are NOT asking for
one of FILE_SYNCHRONOUS_IO*, and waiting on the file handle, and it’s
not working? How is it not working?

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 2:12 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

Hi Nick,

I understand what you are saying about ‘inherently synchronous’ calls
where the optimization is done by the I/O manager.

But, in my case w.r.t. ZwReadFile, I am not sure if we can say that the
request was treated as ‘synchronous’.

Here is my analysis.

The fact that I am getting STATUS_PENDING tells me that none of the
layers that were involved considered it to be a ‘synchronous’ request.
When I say layers, I mean the following (I am assuming that there is no
filtering going on, which is true in my case).

Top layer - The code for the function ZwReadFile/NtReadFile or the
System entry point. This is the layer, which I believe is the one, which
created the IRP and issues the 1st ‘IoCallDriver’ to the NTFS FSD. This
I guess is also the ‘I/O Manager’ layer.

Second Layer - This is the NTFS FSD IRP_MJ_READ entry point. This is the
code that interacts with the cache manager and ‘Completes’ the request
(after a possible recursive ‘IRP_MJ_READ’ because of the fact that data
is not in the cache).

The fact that I got ‘STATUS_PENDING’, I believe, means that none of the
above two layers interpreted the request as a ‘inherently synchronous’
request and hence did not block and allowed the STATUS_PENDING to be
returned all the way back to the caller of ‘ZwReadFile’

When I say ‘inherently synchronous’, what I mean is that, there are some
functions/API’s in the Top layer mentioned above (for example some
implemented by I/O Manager), that make the API ‘inherently synchronous’,
which means that the I/O manager code checks for the returned code and
if it is STATUS_PENDING blocks using a similar event mechanism.

If my understanding above is correct, then none of the layers mentioned
above treated it as a ‘synchronous’ request and hence I got
STATUS_PENDING.

Even if the above is wrong, ‘IopCompleteRequest’ should still be invoked
and my event set.

I tried using the File Handle with ‘Synchronize’ attribute set (at the
time of opening the handle), but that did not work.
Regards,
Manish
Nick Ryan wrote:
For synchronous operations the I/O manager will optimize out the
scheduling of the kernel APC to the original thread (this is what the
IRP_DEFER_IO_COMPLETION flag is for). Since the request is synchronous,
the requesting thread is still blocked, so when it wakes it will call
IopCompleteRequest directly. At least, this is what happens unless
something goes horribly wrong (bogus parameter of some sort, bad IRQL,
lower driver hangs, etc.)

The documentation for ZwReadFile states that you shouldn’t be using the
EventHandle parameter anyways. Instead, you can wait on the file handle
itself to be signaled:

“If the caller opened the file with the DesiredAccess SYNCHRONIZE flag
set, the caller can wait for this routine to set the given FileHandle to
the signaled state.”

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 1:20 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

I am passing Event Handle to ZwReadFile but I am passing the pointer of
the event object to KeWaitForSingle object and the pointer is obtained
using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is
not being invoked in the context of the thread which issued the
ZwReadFile call. To verify this I tried to put a break point in WinDbg
on ‘IopCompleteRequest’ specific for the thread and process. The break
point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point
specific to a process and thread. So I am not sure if I am not setting
the break point correctly or if the routine is not really getting
invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because,
if I understand correctly, this routine is the place where the event
that I passed to ZwReadFile should be set when the NTFS FSD (or the
underlying HDD driver) ‘completes’ the request using
‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is
always invoked in the context of the original thread that caused the IO
request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there
is no need to queue the kernel mode APC (this route may decide NOT to
queue the kernel mode APC based on certain conditions), this APC being
the one through which the ‘IopCompleteRequest’ runs in the context of
the original thread.

Is there a way to set break points to see if the APC is queued or if the
‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com

>ZwReadFile returns me ‘STATUS_PENDING’. so I issue

‘KeWaitForSingleObject’ using the event pointer.

If you have opened file with SYNCHRONIZE DesiredAccess, you can wait on the handle:

ZwReadFile (h, …);
ZwWaitForSingleObject(h, FALSE, NULL);

Alexei.

Are you calling ZwReadFile on APC_LEVEL?

Max

----- Original Message -----
From: Manish Apte
To: File Systems Developers
Sent: Wednesday, June 11, 2003 12:19 AM
Subject: [ntfsd] RE: Problem with ‘ZwReadFile’ returning STATUS_PENDING

I am passing Event Handle to ZwReadFile but I am passing the pointer of the event object to KeWaitForSingle object and the pointer is obtained using ‘ObReferenceObjectByHandle’.

To add some more input, my suspicision is that ‘IopCompleteRequest’ is not being invoked in the context of the thread which issued the ZwReadFile call. To verify this I tried to put a break point in WinDbg on ‘IopCompleteRequest’ specific for the thread and process. The break point did not hit.

Now, I could be wrong in my WinDbg command syntax to set the break point specific to a process and thread. So I am not sure if I am not setting the break point correctly or if the routine is not really getting invoked…

The reason why I am trying to dig into ‘IopCompleteRequest’, is because, if I understand correctly, this routine is the place where the event that I passed to ZwReadFile should be set when the NTFS FSD (or the underlying HDD driver) ‘completes’ the request using ‘IoCompleteRequest’. Based on my understanding, ‘IopCompleteRequest’ is always invoked in the context of the original thread that caused the IO request to originate.

So the problem could be that the ‘IoCompleteRequest’ thinks that there is no need to queue the kernel mode APC (this route may decide NOT to queue the kernel mode APC based on certain conditions), this APC being the one through which the ‘IopCompleteRequest’ runs in the context of the original thread.

Is there a way to set break points to see if the APC is queued or if the ‘IopCompleteRequest’ is getting invoked?

Any idea of using the Debug version of the kernel will help?

BTW, I am running on Win2K Professional SP3.

Regards,

Manish

Nick Ryan wrote:
Are you passing the event HANDLE to ZwReadFile, or the event POINTER?
It’s not clear from your description. You must pass the HANDLE to
ZwReadFile.

- Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Manish Apte
Sent: Tuesday, June 10, 2003 6:33 AM
To: File Systems Developers
Subject: [ntfsd] Problem with ‘ZwReadFile’ returning STATUS_PENDING

Folks,

I am issuing a ‘ZwReadFile’ in my driver to a file on a NTFS partition.

I create an Event object using ‘ZwCreateEvent’ and get a pointer to it
using ‘ObReferenceObjectByHandle’.

I pass a FileHandle and the Event pointer to ‘ZwReadFile’. Both these
handles are of type ‘OBJ_KERNEL_HANDLE’, so I guess there should be no
processs context issues.

ZwReadFile returns me ‘STATUS_PENDING’. ! so I issue
‘KeWaitForSingleObject’ using the event pointer.

My problem is ‘KeWait…’ call never returns (the system does not
crash).

I use the same technique (which is standard thing to do to handle
‘STATUS_PENDING’, I believe) for ‘ZwWriteFile’ and
‘ZwQueryDirectoryFile’.But I DO NOT see this problem here, even though I
DO get ‘STATUS_PENDING’
from these two calls.

Any clues on what could be wrong here?

Regards,

Manish

Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently
subscribed to ntfsd as: xxxxx@nryan.com To unsubscribe send a blank
email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@yahoo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

------------------------------------------------------------------------------
Do you Yahoo!?
Free online calendar with sync to Outlook™. — You are currently subscribed to ntfsd as: xxxxx@storagecraft.com To unsubscribe send a blank email to xxxxx@lists.osr.com