Cancelable Queue ( Csq ) Problem

OSR_Community_User · August 17, 2005, 10:17am

Hi,

Introduction:

*******************

I am implementing a function driver that communicate with user-mode, the driver consist of the following things:

A user-mode thread pool implemented by an IO Completion port.
A symbolic link used to expose the driver to the user
the symbolic link is associated with the thread pool IO Completion port.

An asynchronous IRP reporting communication mechanism.
A cancelable queue ( Csq ) used to store pending IRPs posted by a user-mode call to ReadFile.

Flow of actions:

**************************

The user mode posts an asynchronous ReadFile ( with an appropriate OVERLAPPED structure ), the response is received through a thread waiting on an IO Completion port that was associated with the driver symbolic link.

Testing environment:
*******************

Win XP SP1a

Three different applications that access the driver through a common management layer:

[*] A .NET Application ( works fine )

[*] A simple C++ tester ( works fine )

[*] A large-scale app ( doesn’t work )

The problem:

*******************

Most applications that use the driver work fine, certain application result a strange phenomenon:
Several seconds after the Cancelable IRP Queue was populated with pending IRPs, all of the IRPs are being cancelled one after the other, the IRPs cancellation is done by means other then my driver/application, my assumption is that of some reason the IO Manager cancel the IRPs ( see the call stack added at the bottom of this post ), BUT what may cause the IO Manager to cancel the IRPs [???] it must be something related to a the specific user mode application I use, what user-mode properties may cause such a behavior [???] or, should it be something else ?

Following is the call stack as reported when the first Irp was canceled:

*******************************************************

nt!Kei386EoiHelper+0x2823

SETSDriver!Kernel::Collections::IrpQueue::CsqRemoveIrp+0x4 [d:\r&d_drv_intgrtion\sw\general\technology\driver\function\irpqueue.cpp @ 113]

SETSDriver!IopCsqCancelRoutine+0x42 [d:\dnsrv\base\ntos\io\iomgr\cancelapi.c @ 106]

t!IoCancelIrp+0x5a

t!MmPrefetchPages+0x1003

t!ExWaitForRundownProtectionRelease+0x1d2

t!ExWaitForRundownProtectionRelease+0x399

t!LpcRequestPort+0x49e

t!ZwYieldExecution+0xb78

x7c90eb94

Any help, pointer or comment would be appreciated…

Thanks,

Naddav

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

OSR_Community_User · August 17, 2005, 10:58am

Make sure that the thread that issues a particular I/O request stays alive
while the request is active. All IRPs initiated by user-mode apps
(ReadFile, WriteFile, etc.) are associated with the thread that issued the
IRP. When a thread terminates, all IRPs associated with that thread are
canceled.

Some thread pool implementations will dynamically trim the set of threads,
in response to I/O activity. If such a thread terminates while there are
still I/O requests active that were started on that thread, then they will
be canceled.

To get better information, though, you should use breakpoints in your driver
code to see exactly which call path is canceling the request. You should
also inspect the user-mode portion of the stack, not just the kernel stack.
In *some* cases the user-mode stack is the relevant stack, but not always.

– arlie

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nadav
Sent: Wednesday, August 17, 2005 10:17 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Cancelable Queue ( Csq ) Problem

Hi,

Introduction:

*******************

I am implementing a function driver that communicate with user-mode, the
driver consist of the following things:

A user-mode thread pool implemented by an IO Completion port.
A symbolic link used to expose the driver to the user

a. the symbolic link is associated with the thread pool IO Completion
port.

An asynchronous IRP reporting communication mechanism.

a. A cancelable queue ( Csq ) used to store pending IRPs posted by a
user-mode call to ReadFile.

Flow of actions:

**************************

The user mode posts an asynchronous ReadFile ( with an appropriate
OVERLAPPED structure ), the response is received through a thread waiting on
an IO Completion port that was associated with the driver symbolic link.

Testing environment:
*******************

Win XP SP1a

Three different applications that access the driver through a common
management layer:

[*] A .NET Application ( works fine )

[*] A simple C++ tester ( works fine )

[*] A large-scale app ( doesn’t work )

The problem:

*******************

Most applications that use the driver work fine, certain application result
a strange phenomenon:
Several seconds after the Cancelable IRP Queue was populated with pending
IRPs, all of the IRPs are being cancelled one after the other, the IRPs
cancellation is done by means other then my driver/application, my
assumption is that of some reason the IO Manager cancel the IRPs ( see the
call stack added at the bottom of this post ), BUT what may cause the IO
Manager to cancel the IRPs [???] it must be something related to a the
specific user mode application I use, what user-mode properties may cause
such a behavior [???] or, should it be something else ?

Following is the call stack as reported when the first Irp was canceled:

*******************************************************

nt!Kei386EoiHelper+0x2823

SETSDriver!Kernel::Collections::IrpQueue::CsqRemoveIrp+0x4
[d:\r&d_drv_intgrtion\sw\general\technology\driver\function\irpqueue.cpp @
113]

SETSDriver!IopCsqCancelRoutine+0x42 [d:\dnsrv\base\ntos\io\iomgr\cancelapi.c
@ 106]

t!IoCancelIrp+0x5a

t!MmPrefetchPages+0x1003

t!ExWaitForRundownProtectionRelease+0x1d2

t!ExWaitForRundownProtectionRelease+0x399

t!LpcRequestPort+0x49e

t!ZwYieldExecution+0xb78

x7c90eb94

Any help, pointer or comment would be appreciated.

Thanks,

Naddav

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com — Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17 You are currently subscribed to
ntfsd as: xxxxx@stonestreetone.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

OSR_Community_User · August 18, 2005, 3:21am

Indeed the thread posting the IRPs was exiting just after the posting completed, this caused the IRPs to be canceled by the IO Manager, I have fixed this problem and every thing works fine now.

Thanks for you help.

Arlie Davis wrote:
Make sure that the thread that issues a particular I/O request stays alive while the request is active. All IRPs initiated by user-mode apps (ReadFile, WriteFile, etc.) are associated with the thread that issued the IRP. When a thread terminates, all IRPs associated with that thread are canceled.

Some thread pool implementations will dynamically trim the set of threads, in response to I/O activity. If such a thread terminates while there are still I/O requests active that were started on that thread, then they will be canceled.

To get better information, though, you should use breakpoints in your driver code to see exactly which call path is canceling the request. You should also inspect the user-mode portion of the stack, not just the kernel stack. In some cases the user-mode stack is the relevant stack, but not always.

– arlie

---------------------------------
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Nadav
Sent: Wednesday, August 17, 2005 10:17 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Cancelable Queue ( Csq ) Problem

Hi,

Introduction:

I am implementing a function driver that communicate with user-mode, the driver consist of the following things:

A user-mode thread pool implemented by an IO Completion port.
A symbolic link used to expose the driver to the user
the symbolic link is associated with the thread pool IO Completion port.

An asynchronous IRP reporting communication mechanism.
A cancelable queue ( Csq ) used to store pending IRPs posted by a user-mode call to ReadFile.

Flow of actions:

**************************

The user mode posts an asynchronous ReadFile ( with an appropriate OVERLAPPED structure ), the response is received through a thread waiting on an IO Completion port that was associated with the driver symbolic link.

Testing environment:

Win XP SP1a

Three different applications that access the driver through a common management layer:

[] A .NET Application ( works fine )

[] A simple C++ tester ( works fine )

[] A large-scale app ( doesn’t work )

The problem:

******************

Most applications that use the driver work fine, certain application result a strange phenomenon:
Several seconds after the Cancelable IRP Queue was populated with pending IRPs, all of the IRPs are being cancelled one after the other, the IRPs cancellation is done by means other then my driver/application, my assumption is that of some reason the IO Manager cancel the IRPs ( see the call stack added at the bottom of this post ), BUT what may cause the IO Manager to cancel the IRPs [???] it must be something related to a the specific user mode application I use, what user-mode properties may cause such a behavior [???] or, should it be something else ?

Following is the call stack as reported when the first Irp was canceled:

*******************************************************

nt!Kei386EoiHelper+0x2823

SETSDriver!Kernel::Collections::IrpQueue::CsqRemoveIrp+0x4 [d:\r&d_drv_intgrtion\sw\general\technology\driver\function\irpqueue.cpp @ 113]

SETSDriver!IopCsqCancelRoutine+0x42 [d:\dnsrv\base\ntos\io\iomgr\cancelapi.c @ 106]

t!IoCancelIrp+0x5a

t!MmPrefetchPages+0x1003

t!ExWaitForRundownProtectionRelease+0x1d2

t!ExWaitForRundownProtectionRelease+0x399

t!LpcRequestPort+0x49e

t!ZwYieldExecution+0xb78

x7c90eb94

Any help, pointer or comment would be appreciated…

Thanks,

Naddav

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com — Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17 You are currently subscribed to ntfsd as: xxxxx@stonestreetone.com To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com