BSOD analyzing with IRQL_NOT_LESS_OR_EQUAL (a)

NTDEV
1

Hi, ALL

I have spend some time on this BSOD analyzing:

!analyze -v

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 871dc000, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, bitfield :
bit 0 : value 0 = read operation, 1 = write operation
bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
Arg4: 814d2613, address which referenced memory

Debugging Details:

OVERLAPPED_MODULE: Address regions for ‘lvuvc’ and ‘lvuvc.sys’ overlap

READ_ADDRESS: 871dc000 Nonpaged pool

CURRENT_IRQL: 2

FAULTING_IP:
nt!memcpy+33
814d2613 f3a5 rep movs dword ptr es:[edi],dword ptr [esi]

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

BUGCHECK_STR: AV

PROCESS_NAME: System

TRAP_FRAME: 86ddfb68 – (.trap 0xffffffff86ddfb68)
ErrCode = 00000000
eax=871e123d ebx=00007201 ecx=0000148f edx=00000001 esi=871dc000 edi=9216b000
eip=814d2613 esp=86ddfbdc ebp=86ddfbe4 iopl=0 nv up ei pl nz ac pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010216
nt!memcpy+0x33:
814d2613 f3a5 rep movs dword ptr es:[edi],dword ptr [esi]
Resetting default scope

LOCK_ADDRESS: 815f9ee0 – (!locks 815f9ee0)

Resource @ nt!PiEngineLock (0x815f9ee0) Exclusively owned
Contention Count = 9
NumberOfExclusiveWaiters = 1
Threads: 83f5c3c0-01<*>
Threads Waiting On Exclusive Access:
83e32040

1 total locks, 1 locks currently held

PNP_TRIAGE:
Lock address : 0x815f9ee0
Thread Count : 1
Thread address: 0x83f5c3c0
Thread wait : 0x35f6

LAST_CONTROL_TRANSFER: from 81547d9b to 814ce5e0

STACK_TEXT:
86ddfb48 81547d9b 0000000a 871dc000 00000002 nt!KiBugCheck2
86ddfb48 814d2613 0000000a 871dc000 00000002 nt!KiTrap0E+0x1b3
86ddfbe4 8189fb34 9216903c 871da03c 00007201 nt!memcpy+0x33
86ddfc1c 818a11d4 00007201 aa192a64 871da03c nt!ViMapDoubleBuffer+0x227
86ddfc60 8ba26d0d 895ed650 89556030 a31dafc0 nt!VfGetScatterGatherList+0x1bc
86ddfcac 8ba26f4e b123cf18 b136cf68 00000101 USBXHCI!TransferRing_StageProcess+0x251
86ddfcd8 8ba27488 b123cf18 86ddfd0c 89712ee8 USBXHCI!TransferRing_TransferProcess+0xa0
86ddfd00 8ba214ed b123cf18 b123cf00 00000001 USBXHCI!TransferRing_DispatchEventsAndReleaseLock+0x340
86ddfd1c 84e784fd 4edc32e0 b123cf00 00000000 USBXHCI!TransferRing_WdfEvtIoQueueReadyNotification+0x57
86ddfd30 84e7a246 4edc32e0 b123cf18 00000001 Wdf01000!FxIoQueueIoState::Invoke+0x28
86ddfd60 84e2511f b123cd18 86ddfd88 86ddfda8 Wdf01000!FxIoQueue::ProcessReadyNotify+0x86
86ddfd80 84e289da b123cd00 00000000 b123cd18 Wdf01000!FxIoQueue::DispatchEvents+0x38d
86ddfda0 84e6298b b123cd00 b136ce18 73e21188 Wdf01000!FxIoQueue::QueueRequest+0x204
86ddfddc 8ba5e538 01000000 b136ce18 b12fcc98 Wdf01000!imp_WdfDeviceWdmDispatchIrpToIoQueue+0x38b
86ddfe28 8ba5cee7 73e21188 b12fcc98 871d692c ucx01000!UrbHandler_USBPORTStyle_Legacy_SCT_VendorClassCommand+0x2aa
86ddfe6c 8ba54f89 73e21188 b12fcc98 4edc32e0 ucx01000!Urb_USBPORTStyle_ProcessURB+0x391
86ddfec0 84e20b4e 73e21188 b12fcc98 8955d4f8 ucx01000!RootHub_Pdo_EvtInternalDeviceControlIrpPreprocessCallback+0x44f
86ddfeec 84e20a33 0055d4f8 8955d4f8 8955d4f8 Wdf01000!FxDevice::Dispatch+0xe1
86ddff08 8189af9b 8955d4f8 b12fcc98 b12fcc98 Wdf01000!FxDevice::DispatchWithLock+0x77
86ddff28 81429066 818b27fd b12fcf88 b12fcfac nt!IovCallDriver+0x2e3
86ddff3c 818b27fd 86ddff64 818b28f4 8955d4f8 nt!IofCallDriver+0x73
86ddff44 818b28f4 8955d4f8 b12fcc98 8f649680 nt!ViFilterIoCallDriver+0x10
86ddff64 8189af9b 8f649738 b12fcc98 00000000 nt!ViFilterDispatchGeneric+0x5e
86ddff84 81429066 8dc8533e b12fcc98 a37feee8 nt!IovCallDriver+0x2e3
86ddff98 8dc8533e a37fee2c b12fcc98 a37f000f nt!IofCallDriver+0x73
86de000c 84e20b4e 5c801290 b12fcc98 b0c31db8 UsbHub3!HUBPDO_EvtDeviceWdmIrpPreprocess+0xb65
86de0038 84e20a33 00c31db8 b0c31db8 b0c31db8 Wdf01000!FxDevice::Dispatch+0xe1
86de0054 8189af9b b0c31db8 b12fcc98 b12fcc98 Wdf01000!FxDevice::DispatchWithLock+0x77
86de0074 81429066 818b27fd b12fcfd0 b12fcff4 nt!IovCallDriver+0x2e3
86de0088 818b27fd 86de00b0 818b28f4 b0c31db8 nt!IofCallDriver+0x73
86de0090 818b28f4 b0c31db8 b12fcc98 b0d0c188 nt!ViFilterIoCallDriver+0x10
86de00b0 8189af9b b0d0c240 b12fcc98 b12fcc98 nt!ViFilterDispatchGeneric+0x5e
86de00d0 81429066 8ddac320 9229ace8 aa0ebc48 nt!IovCallDriver+0x2e3
86de00e4 8ddac320 aa0ebc48 b12fcc98 8ddab0e8 nt!IofCallDriver+0x73
86de00f0 8ddab0e8 b134cfd0 00000000 b12fcfd8 usbccgp!UsbcForwardIrp+0x15
86de0114 8ddab2fc b12fcc98 aa0ebc48 b12fcc98 usbccgp!DispatchPdoUrb+0xe3
86de0138 8ddac0f6 aa0ebc48 b12fcc98 aa0ebb88 usbccgp!DispatchPdoInternalDeviceControl+0x15b
86de0170 8189af9b aa0ebb88 b12fcc98 b12fcc98 usbccgp!USBC_Dispatch+0x1fd
86de0190 81429066 a38403b7 00000000 87133014 nt!IovCallDriver+0x2e3
86de01a4 a38403b7 871da02c 871d692c 00000000 nt!IofCallDriver+0x73
WARNING: Stack unwind information not available. Following frames may be wrong.
86de0254 a38412e3 b13f6de4 871d692c 871da02c lvuvc!calloc+0x3fc5f
86de02d4 a3841674 b13f6de4 00000084 00000600 lvuvc!calloc+0x40b8b
86de0378 a3833e7b b13f6de4 00000084 00000600 lvuvc!calloc+0x40f1c
86de03a0 a3833ee0 00000084 00007201 871da03c lvuvc!calloc+0x33723
86de03b4 a3849ad3 871da03c a3b40d1e 871d6d6c lvuvc!calloc+0x33788
86de0420 a384a3d1 871da03d 871da03e 0000000a lvuvc!calloc+0x4937b
86de0578 a381a20b 87127b05 0000000a 87127b20 lvuvc!calloc+0x49c79
86de0768 a382867a 87133014 a382beca 87133014 lvuvc!calloc+0x19ab3
86de087c a380060f b13f6d78 b13f6de4 00000000 lvuvc!calloc+0x27f22
86de0890 8bbcaa9d 00000000 a31a4c78 00000000 lvuvc+0x60f
86de08c4 8bbbf975 a31a4c78 00000001 a31a4fdc ks!CKsDevice::PnpStart+0x2b46
86de08e8 a384cb66 ae3f6378 a31a4c78 ae3f6378 ks!CKsDevice::DispatchPnp+0x120
86de0904 8189af9b ae3f6378 a31a4c78 a31a4c78 lvuvc!calloc+0x4c40e
86de0924 81429066 816ece1b a31a5000 ae3f6378 nt!IovCallDriver+0x2e3
86de0938 816ece1b 00000000 aa0ebb88 83fd7e00 nt!IofCallDriver+0x73
86de0954 8148471c 86de0974 814bd953 84c01300 nt!PnpAsynchronousCall+0x94
86de09b8 816ede56 814bd953 84c01300 894dae30 nt!PnpStartDevice+0x9a
86de0a2c 816ee091 83fd7e00 00000000 00000000 nt!PnpStartDeviceNode+0x137
86de0a50 816e93f0 00000000 00000000 b0cf7a70 nt!PipProcessStartPhase1+0x4d
86de0c4c 816f260b 894dae30 b0cf7a70 86de0c7c nt!PipProcessDevNodeTree+0x373
86de0c88 81485a9e 815de578 83f5c3c0 815de480 nt!PiProcessReenumeration+0x75
86de0cdc 814301c9 00000000 83f5c3c0 00000000 nt!PnpDeviceActionWorker+0x116
86de0d34 8145fb1b 00010000 25de3ec4 00000000 nt!ExpWorkerThread+0x111
86de0d70 81549579 814300bc 00010000 00000000 nt!PspSystemThreadStartup+0x4a
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x19

STACK_COMMAND: kb

FOLLOWUP_IP:
USBXHCI!TransferRing_StageProcess+251
8ba26d0d 8bf8 mov edi,eax

SYMBOL_STACK_INDEX: 5

SYMBOL_NAME: USBXHCI!TransferRing_StageProcess+251

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: USBXHCI

IMAGE_NAME: USBXHCI.SYS

DEBUG_FLR_IMAGE_TIMESTAMP: 51a95c17

BUCKET_ID_FUNC_OFFSET: 251

FAILURE_BUCKET_ID: AV_VRF_USBXHCI!TransferRing_StageProcess

BUCKET_ID: AV_VRF_USBXHCI!TransferRing_StageProcess

Followup: MachineOwner

+++++++++++++++++++++++++++++++++++++++++++++++++
871dc000 is a invalid pool address

1: kd> !pool 871dc000
Pool page 871dc000 region is Nonpaged pool
871dc000 is not a valid large pool allocation, checking large session pool…
871dc000 is not a valid small pool allocation, checking large pool…
unable to get pool big page table - either wrong symbols or pool tagging is disabled
871dc000 is freed (or corrupt) pool
Bad allocation size @871dc000, too large

***
*** An error (or corruption) in the pool was detected;
*** Attempting to diagnose the problem.
***
*** Use !poolval 871dc000 for more details.

Pool page [871dc000] is __inVALID.

Analyzing linked list…

Scanning for single bit errors…

None found

++++++++++++++++++++++++++++++++++++++++++++++++++
Find the buffer address and their length:

kd> dc aa192a64 L50 (from nt!ViMapDoubleBuffer+0x227)

871da03c 00000fc4 00000002 8938a000 871da03c
871db000 00001000 00000002 8938d000 871db000
871dc000 00001000 00000002 89390000 871dc000
871dd000 00001000 00000002 89393000 871dd000
871de000 00001000 00000002 89396000 871de000
871df000 00001000 00000002 89399000 871df000
871e0000 00001000 00000002 8939c000 871e0000
871e1000 0000023d 00000002 8939f000 871e1000
Totally 0x7201 bytes
+++++++++++++++++++++++++++++++++++++++++++++++++++
It is a video class control transfer
with length 0x7201

USB CONTROL IN OF CLASS REQUEST

**********************************************************************
86ddfbe4 8189fb34 9216903c 871da03c 00007201 nt!memcpy+0x33

Memcpy (des, src, len)
Des: 9216903c
Src: 871da03c
Len: 00007201 = 29185(d)
9216b000 - 9216903c = 1FC4 = 8132(d)
871dc000 - 871da03c = 1FC4 = 8132(d)
*********************************************************************

How to dig into/root cause this BSOD?

2

Hello wesley,

Seems like a classical pool corruption. I would hook up a Driver Verifier with “Special Pool” option (or beter to enable all options) and see where it fails.

With best regards, Volodymyr.

3

You’re passing wrong buffer size and/or virtual address to USB stack.

4

workingmailing@163.com wrote:

I have spend some time on this BSOD analyzing:

Have you done any Googling about this? lvuvc.sys is the Logitech UVC
driver for their webcams. Many people have reported bugchecks with this
driver.

+++++++++++++++++++++++++++++++++++++++++++++++++++
It is a video class control transfer
with length 0x7201

USB CONTROL IN OF CLASS REQUEST

That’s HUGE for a control request. Did you trace back in the code to
see whether that’s really the size of the request?

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

I have no code.

btw, I check the stack:

86ddfc60
8ba26d0d
895ed650(DmaAdapter ?)
89556030(DevObj)
a31dafc0(Mdl) nt!VfGetScatterGatherList+0x1bc

NTSTATUS
GetScatterGatherList (
IN PDMA_ADAPTER DmaAdapter,
IN PDEVICE_OBJECT DeviceObject,
IN PMDL Mdl,
IN PVOID CurrentVa,
IN ULONG Length,
IN PDRIVER_LIST_CONTROL ExecutionRoutine,
IN PVOID Context,
IN BOOLEAN WriteToDevice
);
kd> dt nt!_MDL a31dafc0
+0x000 Next : (null)
+0x004 Size : 0n60
+0x006 MdlFlags : 0n2052
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x871da03c Void
+0x010 StartVa : 0x871da000 Void
+0x014 ByteCount : 0x7201
+0x018 ByteOffset : 0x3c

It seems that the MDL really describe the memory start from 0x871da03c with length 0x7201.

6

Hi, Volodymyr

It happend with DV special pool enabled.

I want to root cause the pool corruption.

How to dig out the corruption?

Hi, Alex
fROM THE mdl PARAM for VfGetScatterGatherList, the address and length seems without issue.
What kind of issue with buffer size and/or virtual address to USB stack do you think?

Hi, Tim

From the MDL, URB, and param to ViMapDoubleBuffer function,
and dt _TRANSFER_DATA 0xb136cef8 print out
and !rcdrlogdump USBXHCI -a 0x83fd0008

they all with the same length 0x7201.
I think it is control huge in data transfer, but some times with pool corruption.

Hi, ALL,

From the following print out of command dc aa192a64

add all their length, it is 0x7201 bytes.

My question,
this bsod happen wiTH dv SPECIAL POOL enabled.

You can find, the address jump from 8938a000 to 8938 d 000 (but not 8938 B 000), RIGHT?

But it seems that nt!ViMapDoubleBuffer+0x227 got the unknown structure with ptr aa192a64

It describe the scatter gather lits, right?

From the param, 871dc000 is a valid memory VA, right?

Why when calling memcpy function, it failed?

Still how to dig out the root cause?

Any expert on BSOD analyzing?

Recently, I found that BSOD analyzing is more challenging and interest than windows driver coding.

++++++++++++++++++++++++++++++++++++++++++++++++++
Find the buffer address and their length:

kd> dc aa192a64 L50 (from nt!ViMapDoubleBuffer+0x227)

871da03c 00000fc4 00000002 8938a000 871da03c
871db000 00001000 00000002 8938d000 871db000
871dc000 00001000 00000002 89390000 871dc000
871dd000 00001000 00000002 89393000 871dd000
871de000 00001000 00000002 89396000 871de000
871df000 00001000 00000002 89399000 871df000
871e0000 00001000 00000002 8939c000 871e0000
871e1000 0000023d 00000002 8939f000 871e1000
Totally 0x7201 bytes
+++++++++++++++++++++++++++++++++++++++++++++++++++

7

By the way,

I think this is a control in transfer.

the src buffer is allocated by XHCIUSB.SYS used for scatter gather DMA.

then memcpy function call copy the src buffer to the des buffer.

therefore, the des buffer is allocate by lvuvc.sys.

But now the pool corruption happened with the src buffer.

8

You allocate insufficient buffer size. Your MDL describes bigger buffer than you actually allocated.

9

Something I have to emphasize to my students: the DETECTION of pool
corruption can happen billios or even trillions of instructions from the
ACTUAL corruption. Therefore, the module lvucv.sys is te victim of a
drive-by memory corruption. Just because the memcpy failed there is no
indication that the source of te error is that module. Or, the dtata
strucures it uses are collateral damage by another module. Or, that
module was walking down the street, and fell into an open manhole; you
don’t arrest the fallee in such a case, you have to find who removed the
manhole cover.

This is generally not easy. That’s why you use DV with special pool
enabled. It helps. It may not have acrually found it, and if it can’t,
you may need to enable it for all drivers. If this fails, you have
arrived in a very special kind of Hell from which there may be no escape.
joe

By the way,

I think this is a control in transfer.

the src buffer is allocated by XHCIUSB.SYS used for scatter gather DMA.

then memcpy function call copy the src buffer to the des buffer.

therefore, the des buffer is allocate by lvuvc.sys.

But now the pool corruption happened with the src buffer.

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

10

> IRQL_NOT_LESS_OR_EQUAL (a)

Arg1: 871dc000, memory referenced

kd> dt nt!_MDL a31dafc0
+0x000 Next : (null)
+0x004 Size : 0n60
+0x006 MdlFlags : 0n2052
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x871da03c Void
+0x010 StartVa : 0x871da000 Void
+0x014 ByteCount : 0x7201
+0x018 ByteOffset : 0x3c

What’s the output of these commands?

!pool 871da03c
!pte 871da03c
!pte 871dc000

11

Hi, Pavel

1: kd> !pool 871da03c
Pool page 871da03c region is Nonpaged pool
*871da000 size: 88 previous size: 0 (Allocated) *LCam
Pooltag LCam : WDM mini video capture driver for Logitech camera
871da088 size: f78 previous size: 88 (Free) Free

1: kd> !pte 871da03c
VA 871da03c
PDE at C06021C0 PTE at C0438ED0
contains 00000000007C0863 contains 000000001F8AEB63
pfn 7c0 —DA–KWEV pfn 1f8ae CG-DA–KWEV

1: kd> !pte 871dc000
VA 871dc000
PDE at C06021C0 PTE at C0438EE0
contains 00000000007C0863 contains 0000000000000220
pfn 7c0 —DA–KWEV not valid
DemandZero
Protect: 11 - Readonly G

12

>You allocate insufficient buffer size. Your MDL describes bigger buffer

than you actually allocated.

That’s the first thing that comes to mind but do you see any evidence for
that ? I think he is just testing on a machine with too many buggy drivers
loaded (lvuvc.sys and rwdrv.sys to start with).

//Daniel

13

>That’s the first thing that comes to mind but do you see any evidence for
that ?

The page fault occurs within the range of addresses described by the MDL.

14

Alex Grig,

You are right, it is in the middle of the range described by the MDL.

See my print out of the parameter for ViMapDoubleBuffer+0x227.

My consider is that

the src buffer is allocated by xhciusb.sys MS windows 8 native driver.
the finished the DMA transffer.

Then memcpy the data from src to application’s des buffer.

I suspect that xhciusb.sys have driver fault?

++++++++++++++++++++++++++++++++++++++++++++++++++
Find the buffer address and their length:

kd> dc aa192a64 L50 (from nt!ViMapDoubleBuffer+0x227)

871da03c 00000fc4 00000002 8938a000 871da03c
871db000 00001000 00000002 8938d000 871db000
871dc000 00001000 00000002 89390000 871dc000
871dd000 00001000 00000002 89393000 871dd000
871de000 00001000 00000002 89396000 871de000
871df000 00001000 00000002 89399000 871df000
871e0000 00001000 00000002 8939c000 871e0000
871e1000 0000023d 00000002 8939f000 871e1000
Totally 0x7201 bytes
+++++++++++++++++++++++++++++++++++++++++++++++++++