System hangs during Shutdown

I want to know why my system hangs when I try to shut it down.

I have developed a wdf driver that is PCI based. I have a legacy application that interacts with the PCI hardware using my driver. The application just refuses to close even through the task manager. The only way I can close is through system shutdown. But while shutting down the system hangs.

The application has been working since a long time and is production level applicatino. My driver uses multiple IO queues under the device object. Also uses timer and interrupt. Is there a way I can know using windbg what are the resources that are being held at a given point which the driver has to release or the event the application is waiting for? Could there be any other reason? The driver works perfectly fine with the application.

I am not well versed with the verifier. I tried to use the verifier and below is the output. I know the information is too less but I am open to any driver related information sought.

0: kd> !verifier 0xf MyDrv.sys

Verify Flags Level 0x00000000

STANDARD FLAGS:
(0x00000000) Automatic Checks
(0x00000001) Special pool
(0x00000002) Force IRQL checking
(0x00000008) Pool tracking
(0x00000010) I/O verification
(0x00000020) Deadlock detection
(0x00000080) DMA checking
(0x00000100) Security checks
(0x00000800) Miscellaneous checks

ADDITIONAL FLAGS:
(0x00000004) Randomized low resources simulation
(0x00000200) Force pending I/O requests
(0x00000400) IRP logging

Indicates flag is enabled

Summary of All Verifier Statistics

RaiseIrqls 0x0
AcquireSpinLocks 0x0
Synch Executions 0x0
Trims 0x0

Pool Allocations Attempted 0x0
Pool Allocations Succeeded 0x0
Pool Allocations Succeeded SpecialPool 0x0
Pool Allocations With NO TAG 0x0
Pool Allocations Failed 0x0

Current paged pool allocations 0x0 for 00000000 bytes
Peak paged pool allocations 0x0 for 00000000 bytes
Current nonpaged pool allocations 0x0 for 00000000 bytes
Peak nonpaged pool allocations 0x0 for 00000000 bytes

wdfdrv

its ok with the verifier log. you should attach a live debug to check the
running threads with your driver included.

~-~-~-~-~-~-~-~-~-~
best regards!
–Zhang Pei
在 2014-6-25 下午6:27, 写道:

> I want to know why my system hangs when I try to shut it down.
>
> I have developed a wdf driver that is PCI based. I have a legacy
> application that interacts with the PCI hardware using my driver. The
> application just refuses to close even through the task manager. The only
> way I can close is through system shutdown. But while shutting down the
> system hangs.
>
> The application has been working since a long time and is production level
> applicatino. My driver uses multiple IO queues under the device object.
> Also uses timer and interrupt. Is there a way I can know using windbg what
> are the resources that are being held at a given point which the driver has
> to release or the event the application is waiting for? Could there be any
> other reason? The driver works perfectly fine with the application.
>
> I am not well versed with the verifier. I tried to use the verifier and
> below is the output. I know the information is too less but I am open to
> any driver related information sought.
>
>
> 0: kd> !verifier 0xf MyDrv.sys
>
> Verify Flags Level 0x00000000
>
> STANDARD FLAGS:
> (0x00000000) Automatic Checks
> (0x00000001) Special pool
> (0x00000002) Force IRQL checking
> (0x00000008) Pool tracking
> (0x00000010) I/O verification
> (0x00000020) Deadlock detection
> (0x00000080) DMA checking
> (0x00000100) Security checks
> (0x00000800) Miscellaneous checks
>
> ADDITIONAL FLAGS:
> (0x00000004) Randomized low resources simulation
> (0x00000200) Force pending I/O requests
> (0x00000400) IRP logging
>
> Indicates flag is enabled
>
>
> Summary of All Verifier Statistics
>
> RaiseIrqls 0x0
> AcquireSpinLocks 0x0
> Synch Executions 0x0
> Trims 0x0
>
> Pool Allocations Attempted 0x0
> Pool Allocations Succeeded 0x0
> Pool Allocations Succeeded SpecialPool 0x0
> Pool Allocations With NO TAG 0x0
> Pool Allocations Failed 0x0
>
> Current paged pool allocations 0x0 for 00000000 bytes
> Peak paged pool allocations 0x0 for 00000000 bytes
> Current nonpaged pool allocations 0x0 for 00000000 bytes
> Peak nonpaged pool allocations 0x0 for 00000000 bytes
>
> wdfdrv
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

xxxxx@gmail.com wrote:

I want to know why my system hangs when I try to shut it down.

I have developed a wdf driver that is PCI based. I have a legacy application that interacts with the PCI hardware using my driver. The application just refuses to close even through the task manager. The only way I can close is through system shutdown. But while shutting down the system hangs.

That often means your application has an I/O request outstanding that
your driver is refusing to complete. Does your driver put pending
requests into a queue? Remember that the system cannot force you to
release I/O requests. When you kill the application, you get a
notification, but it’s up to your driver to complete or cancel any
requests that it might be holding. While those requests are
outstanding, the file handle cannot be closed, and the application
cannot be killed.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Most likely your driver is not completing an irp. Run !wdfkd.wdfdevicequeues to see if there are any pending requests. Also !wdfkd.wdflogdump to see if wdf is waiting for your driver to ack a request. If you have power managed queues, did you register an EvtIoStop on each?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, June 25, 2014 3:27 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] System hangs during Shutdown

I want to know why my system hangs when I try to shut it down.

I have developed a wdf driver that is PCI based. I have a legacy application that interacts with the PCI hardware using my driver. The application just refuses to close even through the task manager. The only way I can close is through system shutdown. But while shutting down the system hangs.

The application has been working since a long time and is production level applicatino. My driver uses multiple IO queues under the device object. Also uses timer and interrupt. Is there a way I can know using windbg what are the resources that are being held at a given point which the driver has to release or the event the application is waiting for? Could there be any other reason? The driver works perfectly fine with the application.

I am not well versed with the verifier. I tried to use the verifier and below is the output. I know the information is too less but I am open to any driver related information sought.

0: kd> !verifier 0xf MyDrv.sys

Verify Flags Level 0x00000000

STANDARD FLAGS:
(0x00000000) Automatic Checks
(0x00000001) Special pool
(0x00000002) Force IRQL checking
(0x00000008) Pool tracking
(0x00000010) I/O verification
(0x00000020) Deadlock detection
(0x00000080) DMA checking
(0x00000100) Security checks
(0x00000800) Miscellaneous checks

ADDITIONAL FLAGS:
(0x00000004) Randomized low resources simulation
(0x00000200) Force pending I/O requests
(0x00000400) IRP logging

Indicates flag is enabled

Summary of All Verifier Statistics

RaiseIrqls 0x0
AcquireSpinLocks 0x0
Synch Executions 0x0
Trims 0x0

Pool Allocations Attempted 0x0
Pool Allocations Succeeded 0x0
Pool Allocations Succeeded SpecialPool 0x0
Pool Allocations With NO TAG 0x0
Pool Allocations Failed 0x0

Current paged pool allocations 0x0 for 00000000 bytes
Peak paged pool allocations 0x0 for 00000000 bytes
Current nonpaged pool allocations 0x0 for 00000000 bytes
Peak nonpaged pool allocations 0x0 for 00000000 bytes

wdfdrv


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Note that process termination will attempt to cancel all outstanding IRPs.
Make sure that you correctly handle cancel requests. Particularly if you
are holding an IRP pending somewhere.
joe

xxxxx@gmail.com wrote:
> I want to know why my system hangs when I try to shut it down.
>
> I have developed a wdf driver that is PCI based. I have a legacy
> application that interacts with the PCI hardware using my driver. The
> application just refuses to close even through the task manager. The
> only way I can close is through system shutdown. But while shutting down
> the system hangs.

That often means your application has an I/O request outstanding that
your driver is refusing to complete. Does your driver put pending
requests into a queue? Remember that the system cannot force you to
release I/O requests. When you kill the application, you get a
notification, but it’s up to your driver to complete or cancel any
requests that it might be holding. While those requests are
outstanding, the file handle cannot be closed, and the application
cannot be killed.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

I have implemented FileCleanUp callback. But it does not get invoked when the application ‘X’ button is clicked. I do check for pending requests in the FileCleanUp callback. Is there any other notification I should look for in the driver?

@Joseph perhaps you are referring to implementation of cancel routine for the request.

  1. I always have one request pending in a second IO queue. Driver trace shows that the second IO queue callback is invoked which tell me that the request is de-queued. In such a case the framework will not attempt to cancel the request. Is this understanding correct?

  2. I still tried to implement cancel routine for the IO requests. The WdfRequestMarkCancelableEx call gives a BSOD with the below details. Can anyone help me to resolve this issue? Thanks in advance.

*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {3, 2, 0, fffff88000f464a1}

Probably caused by : Mydrv.sys ( Mydrv!WdfRequestMarkCancelable+40 )

Followup: MachineOwner

1: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000003, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff88000f464a1, address which referenced memory

Debugging Details:

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff80002cc4100
GetUlongFromAddress: unable to read from fffff80002cc41c0
0000000000000003 Nonpaged pool

CURRENT_IRQL: 2

FAULTING_IP:
Wdf01000!FxIrpQueue::InsertIrpInQueue+61
fffff880`00f464a1 4c3910 cmp qword ptr [rax],r10

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT

BUGCHECK_STR: 0xD1

PROCESS_NAME: TRKCTRL.exe

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) x86fre

TRAP_FRAME: fffff88005d4a230 – (.trap 0xfffff88005d4a230)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000003 rbx=0000000000000000 rcx=fffffa80059f70b8
rdx=fffffa80059f7010 rsi=0000000000000000 rdi=0000000000000000
rip=fffff88000f464a1 rsp=fffff88005d4a3c8 rbp=fffff88005d4a450
r8=fffffa8003f10d78 r9=0000000000000000 r10=fffffa80051fe710
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl zr na po nc
Wdf01000!FxIrpQueue::InsertIrpInQueue+0x61:
fffff88000f464a1 4c3910 cmp qword ptr [rax],r10 ds:0000000000000003=???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80002a8c169 to fffff80002a8cbc0

STACK_TEXT:
fffff88005d4a0e8 fffff80002a8c169 : 000000000000000a 0000000000000003 0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff88005d4a0f0 fffff80002a8ade0 : ffff000000000000 fffff80002a73b10 0000000000000004 fffffa8003f10d10 : nt!KiBugCheckDispatch+0x69
fffff88005d4a230 fffff88000f464a1 : fffff88000f3f539 fffffa8005e00844 0000000000000000 fffff88005d4a508 : nt!KiPageFault+0x260
fffff88005d4a3c8 fffff88000f3f539 : fffffa8005e00844 0000000000000000 fffff88005d4a508 0000000000000000 : Wdf01000!FxIrpQueue::InsertIrpInQueue+0x61
fffff88005d4a3d0 fffff88000f4fcd5 : fffffa8003f10d10 0000000000000002 fffffa80051fe640 fffffa8003f10d10 : Wdf01000!FxRequest::InsertTailIrpQueue+0x85
fffff88005d4a410 fffff88000f55efc : fffffa80051fe600 fffffa8003f10d10 0000057ffc0ef2e8 fffff80002a3aa7e : Wdf01000!FxIoQueue::RequestCancelable+0x145
fffff88005d4a480 fffff88004d96530 : fffffa8003f10d10 fffffa8003f10d10 fffff88005d4a6b0 fffff88004d90017 : Wdf01000!imp_WdfRequestMarkCancelable+0x12c
fffff88005d4a4e0 fffff88004d978ac : 0000057ffc0ef2e8 fffff88004d95120 fffff88004d9a160 0000000000000000 : Mydrv!WdfRequestMarkCancelable+0x40 [c:\program files\windows kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h @ 722]
fffff88005d4a520 fffff88000f3fb7c : 0000057ffae019b8 0000057ffc0ef2e8 0000000000000000 0000000000000007 : Mydrv!tcpEvtDeviceIoControl+0x132c [d:\burroughs\work\dev\c++\sys\transfer.c @ 315]
fffff88005d4a670 fffff88000f3f1ff : fffffa80051fe600 fffff88000000007 fffffa80051fe640 0000000000000001 : Wdf01000!FxIoQueue::DispatchRequestToDriver+0x488
fffff88005d4a6f0 fffff88000f4a2fb : fffffa800473c660 fffffa8003f10d00 0000000000000000 fffffa8003f10d10 : Wdf01000!FxIoQueue::DispatchEvents+0x66f
fffff88005d4a770 fffff88000f4051a : fffffa800473c600 fffffa8003f10d10 fffffa80059f7010 fffff88005d4a850 : Wdf01000!FxIoQueue::QueueRequest+0x2ab
fffff88005d4a7e0 fffff88000f3c79a : fffffa8003f10d10 fffffa80059f7010 fffff88005d4ab60 fffffa80059f7010 : Wdf01000!FxPkgIo::Dispatch+0x4da
fffff88005d4a850 fffff88000f3c866 : fffffa80059f7010 fffff88005d4ab60 fffffa8004f2a8b0 0000000000000001 : Wdf01000!FxDevice::Dispatch+0x19a
fffff88005d4a890 fffff80002da9e67 : fffffa8004cac930 fffff88005d4ab60 fffffa8004cac930 fffffa80059f7010 : Wdf01000!FxDevice::DispatchWithLock+0xa6
fffff88005d4a8d0 fffff80002daa6c6 : fffff880009c0180 0000000000000000 0000000000000000 0000000000000000 : nt!IopXxxControlFile+0x607
fffff88005d4aa00 fffff80002a8be53 : 0000000000000000 fffff80002d75d50 fffff88005d4ab60 0000000000000000 : nt!NtDeviceIoControlFile+0x56
fffff88005d4aa70 00000000740c2e09 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiSystemServiceCopyEnd+0x13
0000000002cdf0f8 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x740c2e09

STACK_COMMAND: kb

FOLLOWUP_IP:
Mydrv!WdfRequestMarkCancelable+40 [c:\program files\windows kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h @ 722]
fffff880`04d96530 4883c438 add rsp,38h

FAULTING_SOURCE_LINE: c:\program files\windows kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h

FAULTING_SOURCE_FILE: c:\program files\windows kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h

FAULTING_SOURCE_LINE_NUMBER: 722

FAULTING_SOURCE_CODE:
718: PFN_WDF_REQUEST_CANCEL EvtRequestCancel
719: )
720: {
721: ((PFN_WDFREQUESTMARKCANCELABLE) WdfFunctions[WdfRequestMarkCancelableTableIndex])(WdfDriverGlobals, Request, EvtRequestCancel);

722: }
723:
724: //
725: // WDF Function: WdfRequestMarkCancelableEx
726: //
727: typedef

SYMBOL_STACK_INDEX: 7

SYMBOL_NAME: Mydrv!WdfRequestMarkCancelable+40

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Mydrv

IMAGE_NAME: Mydrv.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 53b3fbc3

FAILURE_BUCKET_ID: X64_0xD1_Mydrv!WdfRequestMarkCancelable+40

BUCKET_ID: X64_0xD1_Mydrv!WdfRequestMarkCancelable+40

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:x64_0xd1_mydrv!wdfrequestmarkcancelable+40

FAILURE_ID_HASH: {fa6fca56-6ab3-772a-dcc3-07b62ffcc881}

Followup: MachineOwner

> @Joseph perhaps you are referring to implementation of cancel routine for

the request.

  1. I always have one request pending in a second IO queue. Driver trace
    shows that the second IO queue callback is invoked which tell me that the
    request is de-queued. In such a case the framework will not attempt to
    cancel the request. Is this understanding correct?

  2. I still tried to implement cancel routine for the IO requests. The
    WdfRequestMarkCancelableEx call gives a BSOD with the below details. Can
    anyone help me to resolve this issue? Thanks in advance.

*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {3, 2, 0, fffff88000f464a1}

Probably caused by : Mydrv.sys ( Mydrv!WdfRequestMarkCancelable+40 )

Followup: MachineOwner

1: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address
at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000003, memory referenced

***What this tells you is that there was an attempt to dereference
***a NULL pointer

Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff88000f464a1, address which referenced memory

Debugging Details:

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff80002cc4100
GetUlongFromAddress: unable to read from fffff80002cc41c0
0000000000000003 Nonpaged pool

CURRENT_IRQL: 2

FAULTING_IP:
Wdf01000!FxIrpQueue::InsertIrpInQueue+61
fffff880`00f464a1 4c3910 cmp qword ptr [rax],r10

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT

BUGCHECK_STR: 0xD1

PROCESS_NAME: TRKCTRL.exe

ANALYSIS_VERSION: 6.3.9600.16384 (debuggers(dbg).130821-1623) x86fre

TRAP_FRAME: fffff88005d4a230 – (.trap 0xfffff88005d4a230)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000003 rbx=0000000000000000 rcx=fffffa80059f70b8
rdx=fffffa80059f7010 rsi=0000000000000000 rdi=0000000000000000
rip=fffff88000f464a1 rsp=fffff88005d4a3c8 rbp=fffff88005d4a450
r8=fffffa8003f10d78 r9=0000000000000000 r10=fffffa80051fe710
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl zr na po nc
Wdf01000!FxIrpQueue::InsertIrpInQueue+0x61:
fffff88000f464a1 4c3910 cmp qword ptr [rax],r10 ds:0000000000000003=???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80002a8c169 to fffff80002a8cbc0

STACK_TEXT:
fffff88005d4a0e8 fffff80002a8c169 : 000000000000000a 0000000000000003
0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff88005d4a0f0 fffff80002a8ade0 : ffff000000000000 fffff80002a73b10
0000000000000004 fffffa8003f10d10 : nt!KiBugCheckDispatch+0x69
fffff88005d4a230 fffff88000f464a1 : fffff88000f3f539 fffffa8005e00844
0000000000000000 fffff88005d4a508 : nt!KiPageFault+0x260
fffff88005d4a3c8 fffff88000f3f539 : fffffa8005e00844 0000000000000000
fffff88005d4a508 0000000000000000 :
Wdf01000!FxIrpQueue::InsertIrpInQueue+0x61
fffff88005d4a3d0 fffff88000f4fcd5 : fffffa8003f10d10 0000000000000002
fffffa80051fe640 fffffa8003f10d10 :
Wdf01000!FxRequest::InsertTailIrpQueue+0x85
fffff88005d4a410 fffff88000f55efc : fffffa80051fe600 fffffa8003f10d10
0000057ffc0ef2e8 fffff80002a3aa7e :
Wdf01000!FxIoQueue::RequestCancelable+0x145
fffff88005d4a480 fffff88004d96530 : fffffa8003f10d10 fffffa8003f10d10
fffff88005d4a6b0 fffff88004d90017 :
Wdf01000!imp_WdfRequestMarkCancelable+0x12c
fffff88005d4a4e0 fffff88004d978ac : 0000057ffc0ef2e8 fffff88004d95120
fffff88004d9a160 0000000000000000 : Mydrv!WdfRequestMarkCancelable+0x40
[c:\program files\windows kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h @
722]
fffff88005d4a520 fffff88000f3fb7c : 0000057ffae019b8 0000057ffc0ef2e8
0000000000000000 0000000000000007 : Mydrv!tcpEvtDeviceIoControl+0x132c
[d:\burroughs\work\dev\c++\sys\transfer.c @ 315]
fffff88005d4a670 fffff88000f3f1ff : fffffa80051fe600 fffff88000000007
fffffa80051fe640 0000000000000001 :
Wdf01000!FxIoQueue::DispatchRequestToDriver+0x488
fffff88005d4a6f0 fffff88000f4a2fb : fffffa800473c660 fffffa8003f10d00
0000000000000000 fffffa8003f10d10 :
Wdf01000!FxIoQueue::DispatchEvents+0x66f
fffff88005d4a770 fffff88000f4051a : fffffa800473c600 fffffa8003f10d10
fffffa80059f7010 fffff88005d4a850 :
Wdf01000!FxIoQueue::QueueRequest+0x2ab
fffff88005d4a7e0 fffff88000f3c79a : fffffa8003f10d10 fffffa80059f7010
fffff88005d4ab60 fffffa80059f7010 : Wdf01000!FxPkgIo::Dispatch+0x4da
fffff88005d4a850 fffff88000f3c866 : fffffa80059f7010 fffff88005d4ab60
fffffa8004f2a8b0 0000000000000001 : Wdf01000!FxDevice::Dispatch+0x19a
fffff88005d4a890 fffff80002da9e67 : fffffa8004cac930 fffff88005d4ab60
fffffa8004cac930 fffffa80059f7010 :
Wdf01000!FxDevice::DispatchWithLock+0xa6
fffff88005d4a8d0 fffff80002daa6c6 : fffff880009c0180 0000000000000000
0000000000000000 0000000000000000 : nt!IopXxxControlFile+0x607
fffff88005d4aa00 fffff80002a8be53 : 0000000000000000 fffff80002d75d50
fffff88005d4ab60 0000000000000000 : nt!NtDeviceIoControlFile+0x56
fffff88005d4aa70 00000000740c2e09 : 0000000000000000 0000000000000000
0000000000000000 0000000000000000 : nt!KiSystemServiceCopyEnd+0x13
0000000002cdf0f8 0000000000000000 : 0000000000000000 0000000000000000
0000000000000000 0000000000000000 : 0x740c2e09

STACK_COMMAND: kb

FOLLOWUP_IP:
Mydrv!WdfRequestMarkCancelable+40 [c:\program files\windows
kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h @ 722]
fffff880`04d96530 4883c438 add rsp,38h

FAULTING_SOURCE_LINE: c:\program files\windows
kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h

FAULTING_SOURCE_FILE: c:\program files\windows
kits\8.1\include\wdf\kmdf\1.11\wdfrequest.h

FAULTING_SOURCE_LINE_NUMBER: 722

FAULTING_SOURCE_CODE:
718: PFN_WDF_REQUEST_CANCEL EvtRequestCancel
719: )
720: {
721: ((PFN_WDFREQUESTMARKCANCELABLE)
WdfFunctions[WdfRequestMarkCancelableTableIndex])(WdfDriverGlobals,
Request, EvtRequestCancel);
> 722: }
723:
724: //
725: // WDF Function: WdfRequestMarkCancelableEx
726: //
727: typedef

SYMBOL_STACK_INDEX: 7

SYMBOL_NAME: Mydrv!WdfRequestMarkCancelable+40

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Mydrv

IMAGE_NAME: Mydrv.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 53b3fbc3

FAILURE_BUCKET_ID: X64_0xD1_Mydrv!WdfRequestMarkCancelable+40

BUCKET_ID: X64_0xD1_Mydrv!WdfRequestMarkCancelable+40

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:x64_0xd1_mydrv!wdfrequestmarkcancelable+40

FAILURE_ID_HASH: {fa6fca56-6ab3-772a-dcc3-07b62ffcc881}

Followup: MachineOwner


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

> But that would not cause the error trying to dereference NULL (which produces the same error

message as accessing paged-out memory at elevated IRQL)

…apart from the fact that it does not - you are just being mislead by the name PAGE_FAULT_IN_NONPAGED_AREA. In actuality, you get this error code only when you access invalid memory address, regardless of IRQL. When it comes to causing page faults at elevated IRQL you are going to get
IRQL_NOT_LESS_OR_EQUAL, and this is going to happen when/if the failing thread makes a blocking call…

Anton Bassov

Additional input is that the same read request (code given below) is successfully executed multiple times prior to this BSOD.

Looking at the stack trace InsertIrpInQueue seems to be trying to access location 0x3. The only two pointers going in there from the code are the Request itself and the EvtRequestCancel. Both the pointers are verified and seem to be correct.

May I know why InsertIrpInQueue needs to be called as shown in the stack trace where it tries to access the 0x3 location and page faults? Which queue is the WdfRequestMarkCancelable internally refers to when I have already de-queued the request?

Doesn’t calling WdfRequestMarkCancelable from the EvtIODeviceControl mean after de-queueing the request?

The below read IOCTL is from the EvtIODeviceControl of the second IO queue (Q2) where the BSOD seems to occurs.

I pickup the read IOCTL from the default IO queue (Q1) and place it in Q2 when the below code of EvtIODeviceControl of Q2 gets executed.

Note: I have now changed WdfRequestMarkCancelable to WdfRequestMarkCancelableEx.

case MYDRV_READ:

//IoMarkIrpPending(irp);

TraceEvents(TRACE_LEVEL_WARNING, MYDRV_READ_TRACE_IO,
“myEvtDeviceReadIoControl - MYDRV_READ: Request %p myEvtRequestCancel %p”, (PVOID)Request, myEvtRequestCancel);

ASSERT(Request != NULL);

//Mark for cancellation
WdfSpinLockAcquire(pContext->ioReadSpinLock);

status = WdfRequestMarkCancelableEx(Request, myEvtRequestCancel);

if (!NT_SUCCESS(status)) {
pContext->CurrentRequest = NULL;
}

WdfSpinLockRelease(pContext->ioReadSpinLock);

//
// Complete the request with an error when unable to mark it cancelable.
//
if (!NT_SUCCESS(status)) {
TraceEvents(TRACE_LEVEL_WARNING, DP500_TRACE_IO,
“MYDRV_INITCOMMS: WdfRequestMarkCancelableEx failed with status %d”, status);

WdfRequestCompleteWithInformation(Request, status, 0L);
break;
}

> Additional input is that the same read request (code given below) is

successfully executed multiple times prior to this BSOD.

Looking at the stack trace InsertIrpInQueue seems to be trying to access
location 0x3. The only two pointers going in there from the code are the
Request itself and the EvtRequestCancel. Both the pointers are verified
and seem to be correct.

May I know why InsertIrpInQueue needs to be called as shown in the stack
trace where it tries to access the 0x3 location and page faults? Which
queue is the WdfRequestMarkCancelable internally refers to when I have
already de-queued the request?

Doesn’t calling WdfRequestMarkCancelable from the EvtIODeviceControl mean
after de-queueing the request?

The below read IOCTL is from the EvtIODeviceControl of the second IO queue
(Q2) where the BSOD seems to occurs.

I pickup the read IOCTL from the default IO queue (Q1) and place it in Q2
when the below code of EvtIODeviceControl of Q2 gets executed.

Note: I have now changed WdfRequestMarkCancelable to
WdfRequestMarkCancelableEx.

case MYDRV_READ:

//IoMarkIrpPending(irp);

TraceEvents(TRACE_LEVEL_WARNING, MYDRV_READ_TRACE_IO,
“myEvtDeviceReadIoControl - MYDRV_READ: Request %p myEvtRequestCancel
%p”, (PVOID)Request, myEvtRequestCancel);

ASSERT(Request != NULL);

Note that if the request is == NULL, you will fall through and try to use
it. ASSERT does not change the execution flow at all. So if you want a
“clean abort” here you will have to test for NULL and take an appropriate
action (which is not a BugCheck).

//Mark for cancellation
WdfSpinLockAcquire(pContext->ioReadSpinLock);

Under WDM, before you could change the cancel routine, you had to
KeAcquireCancelSpinLock. I do not know the KMDF protocol. But if it
acquires the cancel spin lock inside WdfRequestMarkCancelableEx, this
spinlock may be redundant. Just a thought; a KMDF expert should jump in
here. Also, what other actions use this spin lock, and could a
cancelation create a deadlock opportunity?

status = WdfRequestMarkCancelableEx(Request, myEvtRequestCancel);

I would suggest moving the debug printout below to this point. Consider
the case where the Context field is causing the BSOD. So just after you
set this to NULL, the cancel action is invoked, and seeing the
CurrentRequest, tries to do something with it, and crashes. Since it
crashes, this thread never gets to reach the print statement, so you miss
seeing this case.

Just some random thoughts based on this code and the description.

Since the stack dump shows the backtrace of the crashing thread, it can’t
tell you if this thread has had a failure, and is between the setting of
the field to NULL and the printout.

if (!NT_SUCCESS(status)) {
pContext->CurrentRequest = NULL;
}

WdfSpinLockRelease(pContext->ioReadSpinLock);

//
// Complete the request with an error when unable to mark it cancelable.
//
if (!NT_SUCCESS(status)) {
TraceEvents(TRACE_LEVEL_WARNING, DP500_TRACE_IO,
“MYDRV_INITCOMMS: WdfRequestMarkCancelableEx failed with status %d”,
status);

WdfRequestCompleteWithInformation(Request, status, 0L);
break;
}


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Turn on wdf verifier (esp Io verification) and try to repro. I am guessing you are operating on an already completed request

d

Bent from my phone


From: xxxxx@gmail.commailto:xxxxx
Sent: ?7/?4/?2014 12:13 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] System hangs during Shutdown

Additional input is that the same read request (code given below) is successfully executed multiple times prior to this BSOD.

Looking at the stack trace InsertIrpInQueue seems to be trying to access location 0x3. The only two pointers going in there from the code are the Request itself and the EvtRequestCancel. Both the pointers are verified and seem to be correct.

May I know why InsertIrpInQueue needs to be called as shown in the stack trace where it tries to access the 0x3 location and page faults? Which queue is the WdfRequestMarkCancelable internally refers to when I have already de-queued the request?

Doesn’t calling WdfRequestMarkCancelable from the EvtIODeviceControl mean after de-queueing the request?

The below read IOCTL is from the EvtIODeviceControl of the second IO queue (Q2) where the BSOD seems to occurs.

I pickup the read IOCTL from the default IO queue (Q1) and place it in Q2 when the below code of EvtIODeviceControl of Q2 gets executed.

Note: I have now changed WdfRequestMarkCancelable to WdfRequestMarkCancelableEx.

case MYDRV_READ:

//IoMarkIrpPending(irp);

TraceEvents(TRACE_LEVEL_WARNING, MYDRV_READ_TRACE_IO,
“myEvtDeviceReadIoControl - MYDRV_READ: Request %p myEvtRequestCancel %p”, (PVOID)Request, myEvtRequestCancel);

ASSERT(Request != NULL);

//Mark for cancellation
WdfSpinLockAcquire(pContext->ioReadSpinLock);

status = WdfRequestMarkCancelableEx(Request, myEvtRequestCancel);

if (!NT_SUCCESS(status)) {
pContext->CurrentRequest = NULL;
}

WdfSpinLockRelease(pContext->ioReadSpinLock);

//
// Complete the request with an error when unable to mark it cancelable.
//
if (!NT_SUCCESS(status)) {
TraceEvents(TRACE_LEVEL_WARNING, DP500_TRACE_IO,
“MYDRV_INITCOMMS: WdfRequestMarkCancelableEx failed with status %d”, status);

WdfRequestCompleteWithInformation(Request, status, 0L);
break;
}


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

did you also use spinlock when you complete the read request ?

xxxxx@gmail.com编写:

Additional input is that the same read request (code given below) is successfully executed multiple times prior to this BSOD.

Looking at the stack trace InsertIrpInQueue seems to be trying to access location 0x3. The only two pointers going in there from the code are the Request itself and the EvtRequestCancel. Both the pointers are verified and seem to be correct.

May I know why InsertIrpInQueue needs to be called as shown in the stack trace where it tries to access the 0x3 location and page faults? Which queue is the WdfRequestMarkCancelable internally refers to when I have already de-queued the request?

Doesn’t calling WdfRequestMarkCancelable from the EvtIODeviceControl mean after de-queueing the request?

The below read IOCTL is from the EvtIODeviceControl of the second IO queue (Q2) where the BSOD seems to occurs.

I pickup the read IOCTL from the default IO queue (Q1) and place it in Q2 when the below code of EvtIODeviceControl of Q2 gets executed.

Note: I have now changed WdfRequestMarkCancelable to WdfRequestMarkCancelableEx.

case MYDRV_READ:

//IoMarkIrpPending(irp);

TraceEvents(TRACE_LEVEL_WARNING, MYDRV_READ_TRACE_IO,
“myEvtDeviceReadIoControl - MYDRV_READ: Request %p myEvtRequestCancel %p”, (PVOID)Request, myEvtRequestCancel);

ASSERT(Request != NULL);

//Mark for cancellation
WdfSpinLockAcquire(pContext->ioReadSpinLock);

status = WdfRequestMarkCancelableEx(Request, myEvtRequestCancel);

if (!NT_SUCCESS(status)) {
pContext->CurrentRequest = NULL;
}

WdfSpinLockRelease(pContext->ioReadSpinLock);

//
// Complete the request with an error when unable to mark it cancelable.
//
if (!NT_SUCCESS(status)) {
TraceEvents(TRACE_LEVEL_WARNING, DP500_TRACE_IO,
“MYDRV_INITCOMMS: WdfRequestMarkCancelableEx failed with status %d”, status);

WdfRequestCompleteWithInformation(Request, status, 0L);
break;
}


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

@ Doron I will try.

pointers during successful read IOCTL

Request 0000057FFC8ADB88 tcpEvtRequestCancel FFFFF88004B3B530

pointers just before the BSOD in the same read IOCTL

Request 0000057FFBF66FD8 tcpEvtRequestCancel FFFFF88004B3B530

111111111111

I initially faced an issue(1) in shutting down the Target machine. The probable cause was a pending request. I thought of implementing the cancel routine for the request which ran into an issue(2), a BSOD (details shown in this thread above) while registering the cancel routine for the request. Neither of them are resolved.

Below is some debug data using windbg which proves that a request lies pending with the driver and not in any of the queues.

  1. Is this request responsible for the shutdown lock-up?

  2. Will cancel routine be useful in this case as the request is with the driver waiting for some data from the hardware and not actually in the queue? (I hope my assumption is correct).

  3. On what event I can close this request? I have a legacy application which does not send a closehandle(), instead expects the system to cleanup instead. Hence although I have implemented FileCleanup callback it does not get invoked upon clicking the “X” button on the application (I hope my assumption is correct).

Only a hard shutdown using the power button of the system is THE option left for shutting down the target system.

PLEASE HELP.

0: kd> !wdfkd.wdfdevicequeues 0x0000057ffb1a0fd8
Treating handle as a KMDF handle!

Dumping queues of WDFDEVICE 0x0000057ffb1a0fd8

Number of queues: 2

Queue: 1 !wdfqueue 0x0000057ffb1bc978
Sequential, Auto, Power-managed, PowerOn, Passive Only, Can accept, Can dispatch, ExecutionLevelPassive, SynchronizationScopeNone
Number of driver owned requests: 0
Number of waiting requests: 0

EvtIoDefault: (0xfffff88002cc5d70)
EvtIoDeviceControl: (0xfffff88002cc62f0)

Queue: 2 !wdfqueue 0x0000057ffb1bce28
Sequential, Power-managed, PowerOn, Passive Only, Can accept, Can dispatch, ExecutionLevelPassive, SynchronizationScopeNone
Number of driver owned requests: 1
!wdfrequest 0x0000057ffa6f7b58 !irp 0xfffffa80038975d0
Number of waiting requests: 0

EvtIoDeviceControl: (0xfffff88002cc8cc0)
0: kd> !wdfkd.wdfrequest 0x0000057ffb1a0fd8
Treating handle as a KMDF handle!
Not a valid WDFREQUEST handle
0: kd> !wdfkd.wdfrequest 0x0000057ffa6f7b58
Treating handle as a KMDF handle!
!irp 0xfffffa80038975d0

!wdfqueue 0x0000057ffb1bce28
State: Pending, Allocated by WDF for incoming IRP
System Buffer 0xfffffa80038a5380, Length 0x0, !wdfmemory 0x0000057ffa6f7a91
IOCTL Output Buffer 0xfffffa80038a5380, Length 0x105, !wdfmemory 0x0000057ffa6f7a81

If the request is “on the hardware” and is not being completed on shutdown,
that would account for the hang. Yes you can cancel “on hardware” requests,
but you have to correctly handle the race condition between the hardware
completion and the cancel completion, and that is not made any easier, in
my opinion, by the layering of the WDF framework over the legacy WDM
mechanisms. See the comments here
http://msdn.microsoft.com/en-us/library/windows/hardware/ff550035(v=vs.85).aspx
regarding *WdfRequestUnmarkCancelable.*

You have to track the request state yourself in a request context or you
will BSOD.

Mark Roddy

On Mon, Jul 7, 2014 at 6:05 AM, wrote:

> I initially faced an issue(1) in shutting down the Target machine. The
> probable cause was a pending request. I thought of implementing the cancel
> routine for the request which ran into an issue(2), a BSOD (details shown
> in this thread above) while registering the cancel routine for the request.
> Neither of them are resolved.
>
> Below is some debug data using windbg which proves that a request lies
> pending with the driver and not in any of the queues.
>
> 1) Is this request responsible for the shutdown lock-up?
>
> 2) Will cancel routine be useful in this case as the request is with the
> driver waiting for some data from the hardware and not actually in the
> queue? (I hope my assumption is correct).
>
> 3) On what event I can close this request? I have a legacy application
> which does not send a closehandle(), instead expects the system to cleanup
> instead. Hence although I have implemented FileCleanup callback it does not
> get invoked upon clicking the “X” button on the application (I hope my
> assumption is correct).
>
> Only a hard shutdown using the power button of the system is THE option
> left for shutting down the target system.
>
> PLEASE HELP.
>
> 0: kd> !wdfkd.wdfdevicequeues 0x0000057ffb1a0fd8
> Treating handle as a KMDF handle!
>
> Dumping queues of WDFDEVICE 0x0000057ffb1a0fd8
> =====================================
> Number of queues: 2
> ----------------------------------
> Queue: 1 !wdfqueue 0x0000057ffb1bc978
> Sequential, Auto, Power-managed, PowerOn, Passive Only, Can accept,
> Can dispatch, ExecutionLevelPassive, SynchronizationScopeNone
> Number of driver owned requests: 0
> Number of waiting requests: 0
>
>
> EvtIoDefault: (0xfffff88002cc5d70)
> EvtIoDeviceControl: (0xfffff88002cc62f0)
> ----------------------------------
> Queue: 2 !wdfqueue 0x0000057ffb1bce28
> Sequential, Power-managed, PowerOn, Passive Only, Can accept, Can
> dispatch, ExecutionLevelPassive, SynchronizationScopeNone
> Number of driver owned requests: 1
> !wdfrequest 0x0000057ffa6f7b58 !irp 0xfffffa80038975d0
> Number of waiting requests: 0
>
>
> EvtIoDeviceControl: (0xfffff88002cc8cc0)
> 0: kd> !wdfkd.wdfrequest 0x0000057ffb1a0fd8
> Treating handle as a KMDF handle!
> Not a valid WDFREQUEST handle
> 0: kd> !wdfkd.wdfrequest 0x0000057ffa6f7b58
> Treating handle as a KMDF handle!
> !irp 0xfffffa80038975d0
>
> !wdfqueue 0x0000057ffb1bce28
> State: Pending, Allocated by WDF for incoming IRP
> System Buffer 0xfffffa80038a5380, Length 0x0, !wdfmemory
> 0x0000057ffa6f7a91
> IOCTL Output Buffer 0xfffffa80038a5380, Length 0x105, !wdfmemory
> 0x0000057ffa6f7a81
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>