UsbHub crash during restart

NTDEV
1

I have two crashdumps which occured in the same scenario: OS (XP SP1)
crashed during restart when an USB device was in use. In one case it was
handled by my driver but in the second by the usbscan OS driver. Both
crashes occured accesing the same data so I wonder if there isn’t an OS bug.
It of course occurs on one computer only which is located on the opposite
side of the Earth.

My question is if anybody is able to identify used data structures. The
pointer to unknown data structure is stored in the PDO device extension.
Device in the moment of crash looks like this:

kd> !devobj 812cb270
Device object (812cb270) is for:
USBPDO-8 \Driver\usbhub DriverObject 819cf138
Current Irp 00000000 RefCount 0 Type 00000022 Flags 00003040
Dacl e14d934c DevExt 812cb328 DevObjExt 812cb438 DevNode 816c0448
ExtensionFlags (0x00000004) DOE_REMOVE_PENDING
AttachedDevice (Upper) 813fa030 \Driver\usbscan
Device queue is not busy.

Device extension start is following:
kd> dd 812cb328
812cb328 54524f50 812cb270 815f3aa8 00000003

The first DWORD is ‘PORT’, the second one is pointer to the device and the
third is the unknown structure referenced during crash. Crashing code is
following call in both cases:

f9b7704b ff9318020000 call dword ptr [ebx+0x218]

and ebx is 815f3aa8 i.e. pointer to the mentioned structure. There is zero
at offset 0x218 in one case and bogus pointer in the second one. In the
second case memory seems as already freed and problem occurs during device
removal. In above case memory is probably still allocated as data contained
seem valid but called address is already zeroed. The stack look like this:

STACK_TEXT:
f91c2ba4 f9b77051 80541c80 68627375 64657621 0x0
f91c2bcc f9b74a4a 816fbce8 813830a8 f91c2c30 usbhub!USBH_PdoDispatch+0x1e7
f91c2bdc 804eca36 812cb270 813830a8 813830a8 usbhub!USBH_HubDispatch+0x46
f91c2bec f85c719e 813fa030 813830a8 00000000 nt!IopfCallDriver+0x31
f91c2c30 f85c72f4 813fa030 813830a8 00000001 usbscan!USTransfer+0x40e
f91c2c60 804eca36 813fa030 00000078 806c7fe0 usbscan!USWrite+0x7c
f91c2c70 8058b076 81383184 00000000 813830a8 nt!IopfCallDriver+0x31
f91c2c84 8058dfc2 813fa030 813830a8 813331b0
nt!IopSynchronousServiceTail+0x5e
f91c2d38 804da140 00000818 00000798 00000000 nt!NtWriteFile+0x5de
f91c2d38 7ffe0304 00000818 00000798 00000000 nt!KiSystemService+0xc4
0141f3e0 00000000 00000000 00000000 00000000
SharedUserData!SystemCallStub+0x4

Has anybody seen something similar or has any info which could help?
Currently I’m a bit busy and don’t have time to disassemble usbhub and
analyse problem more closely just now…

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

2

>

kd> !devobj 812cb270
Device object (812cb270) is for:
USBPDO-8 \Driver\usbhub DriverObject 819cf138 Current Irp
00000000 RefCount 0 Type 00000022 Flags 00003040 Dacl
e14d934c DevExt 812cb328 DevObjExt 812cb438 DevNode 816c0448
ExtensionFlags (0x00000004) DOE_REMOVE_PENDING
AttachedDevice (Upper) 813fa030 \Driver\usbscan Device queue
is not busy.

Device extension start is following:
kd> dd 812cb328
812cb328 54524f50 812cb270 815f3aa8 00000003

The first DWORD is ‘PORT’, the second one is pointer to the
device and the
third is the unknown structure referenced during crash.

The third ULONG ought to be a pointer to the hub device extension. Null or
garbage would be very bad. The first ULONG where the hub device extension
ought to be should have the value 'HUB '.

=====================
Mark Roddy

3

> ----------

From: xxxxx@stratus.com[SMTP:xxxxx@stratus.com]
Reply To: xxxxx@lists.osr.com
Sent: Wednesday, October 29, 2003 10:57 PM
To: xxxxx@lists.osr.com
Subject: [ntdev] RE: UsbHub crash during restart

> kd> !devobj 812cb270
> Device object (812cb270) is for:
> USBPDO-8 \Driver\usbhub DriverObject 819cf138 Current Irp
> 00000000 RefCount 0 Type 00000022 Flags 00003040 Dacl
> e14d934c DevExt 812cb328 DevObjExt 812cb438 DevNode 816c0448
> ExtensionFlags (0x00000004) DOE_REMOVE_PENDING
> AttachedDevice (Upper) 813fa030 \Driver\usbscan Device queue
> is not busy.
>
> Device extension start is following:
> kd> dd 812cb328
> 812cb328 54524f50 812cb270 815f3aa8 00000003
>
> The first DWORD is ‘PORT’, the second one is pointer to the
> device and the
> third is the unknown structure referenced during crash.

The third ULONG ought to be a pointer to the hub device extension. Null or
garbage would be very bad. The first ULONG where the hub device extension
ought to be should have the value 'HUB '.

Thanks, you’re right. On live system is seems like this and the first DWORD
is 'HUB '. In crashdump two first DWORDs are zeroed so I guess it was
already freed and wasn’t rewritten, yet. Something went wrong.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

4

Michal Vodicka wrote:

I have two crashdumps which occured in the same scenario: OS (XP SP1)
crashed during restart when an USB device was in use. In one case it was
handled by my driver but in the second by the usbscan OS driver. Both
crashes occured accesing the same data so I wonder if there isn’t an OS bug.
It of course occurs on one computer only which is located on the opposite
side of the Earth.

I recently encountered a similar, but very repeatable, bug. It occurred
when my power handler left the device in D3 after restart and when the
device had been unplugged during hibernate/suspend. Thereafter, before
the surprise removal processing finished, a substantive code path tried
to repower the device. USBHUB crashed because its internal PDO pointer
got nulled in between a test and a subsequent use of the pointer.

–
Walter Oney, Consulting and Training
Basic and Advanced Driver Programming Seminars
Check out our schedule at http://www.oneysoft.com

5

> ----------

From: xxxxx@oneysoft.com[SMTP:xxxxx@oneysoft.com]
Reply To: xxxxx@lists.osr.com
Sent: Thursday, October 30, 2003 2:11 PM
To: xxxxx@lists.osr.com
Subject: [ntdev] Re: UsbHub crash during restart

Michal Vodicka wrote:
> I have two crashdumps which occured in the same scenario: OS (XP SP1)
> crashed during restart when an USB device was in use. In one case it was
> handled by my driver but in the second by the usbscan OS driver. Both
> crashes occured accesing the same data so I wonder if there isn’t an OS
bug.
> It of course occurs on one computer only which is located on the
opposite
> side of the Earth.

I recently encountered a similar, but very repeatable, bug. It occurred
when my power handler left the device in D3 after restart and when the
device had been unplugged during hibernate/suspend. Thereafter, before
the surprise removal processing finished, a substantive code path tried
to repower the device. USBHUB crashed because its internal PDO pointer
got nulled in between a test and a subsequent use of the pointer.

And the final resolution? Is it fixed or planned to fix by MS?

I have only vague description from the guy who encountered the problem. In
both cases two our devices were attached to the same USB hub and for one the
driver was updated via device manager. Device was opened in this moment and
OS asked for restart. In one case both devices were unplugged before restart
and BSOD occured when restart was confirmed. It seems as PDO had a pointer
to HUB device extension which was freed in other code path. There is really
something wrong if I look more closely on the device:

kd> !devobj 812cb270
Device object (812cb270) is for:
USBPDO-8 \Driver\usbhub DriverObject 819cf138 Current Irp
00000000 RefCount 0 Type 00000022 Flags 00003040 Dacl
e14d934c DevExt 812cb328 DevObjExt 812cb438 DevNode 816c0448
ExtensionFlags (0x00000004) DOE_REMOVE_PENDING
AttachedDevice (Upper) 813fa030 \Driver\usbscan Device queue
is not busy.

RefCount is 0, remove is pending and upper device is still attached. No
wonder it crashed when IRP was sent from upper device. I’ll try to simulate
this scenario. For now it seems as usbhub driver bug; I don’t see how
function driver could cause it.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

6

There is KB article http://support.microsoft.com/?kbid=822603 which contains
a statement which may apply for this problem and contains updated usbhub
driver:

Plug and Play

* Your computer may automatically restart, or you may receive a serious
error (STOP 0x0000007E in Usbhub.sys) when you use the Safely Remove
Hardware utility, and then immediately remove a USB storage device that is
connected directly to your computer.

Note This issue only occurs occasionally.

Vague as usual (is it so hard to describe problem exactly?) but both error
and buggy driver correrspond and scenario is similar. Does anybody have an
experience with this patch or more information about mentioned error? It
would also help to know how exactly the Safely Remove Hardware utility works
i.e. conditions necessary for crash.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

From:
xxxxx@veridicom.cz.nospam[SMTP:xxxxx@veridicom.cz.nospam]
Reply To: xxxxx@lists.osr.com
Sent: Thursday, October 30, 2003 8:32 PM
To: xxxxx@lists.osr.com
Subject: [ntdev] Re: UsbHub crash during restart

> ----------
> From: xxxxx@oneysoft.com[SMTP:xxxxx@oneysoft.com]
> Reply To: xxxxx@lists.osr.com
> Sent: Thursday, October 30, 2003 2:11 PM
> To: xxxxx@lists.osr.com
> Subject: [ntdev] Re: UsbHub crash during restart
>
> Michal Vodicka wrote:
> > I have two crashdumps which occured in the same scenario: OS (XP SP1)
> > crashed during restart when an USB device was in use. In one case it
was
> > handled by my driver but in the second by the usbscan OS driver. Both
> > crashes occured accesing the same data so I wonder if there isn’t an
OS
> bug.
> > It of course occurs on one computer only which is located on the
> opposite
> > side of the Earth.
>
> I recently encountered a similar, but very repeatable, bug. It occurred
> when my power handler left the device in D3 after restart and when the
> device had been unplugged during hibernate/suspend. Thereafter, before
> the surprise removal processing finished, a substantive code path tried
> to repower the device. USBHUB crashed because its internal PDO pointer
> got nulled in between a test and a subsequent use of the pointer.
>
And the final resolution? Is it fixed or planned to fix by MS?

I have only vague description from the guy who encountered the problem. In
both cases two our devices were attached to the same USB hub and for one
the
driver was updated via device manager. Device was opened in this moment
and
OS asked for restart. In one case both devices were unplugged before
restart
and BSOD occured when restart was confirmed. It seems as PDO had a pointer
to HUB device extension which was freed in other code path. There is
really
something wrong if I look more closely on the device:

kd> !devobj 812cb270
Device object (812cb270) is for:
USBPDO-8 \Driver\usbhub DriverObject 819cf138 Current Irp
00000000 RefCount 0 Type 00000022 Flags 00003040 Dacl
e14d934c DevExt 812cb328 DevObjExt 812cb438 DevNode 816c0448
ExtensionFlags (0x00000004) DOE_REMOVE_PENDING
AttachedDevice (Upper) 813fa030 \Driver\usbscan Device queue
is not busy.

RefCount is 0, remove is pending and upper device is still attached. No
wonder it crashed when IRP was sent from upper device. I’ll try to
simulate
this scenario. For now it seems as usbhub driver bug; I don’t see how
function driver could cause it.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: michal.vodicka@st.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

7

> I recently encountered a similar, but very repeatable, bug. It occurred

when my power handler left the device in D3 after restart and when the
device had been unplugged during hibernate/suspend. Thereafter, before
the surprise removal processing finished, a substantive code path tried
to repower the device. USBHUB crashed because its internal PDO pointer
got nulled in between a test and a subsequent use of the pointer.

This is a very dangerous thing with XP’s USB stack.
For instance:

  • attach a scanner to a notebook
  • hibernate
  • detach the scanner
  • resume the machine
    And you have 100% CPU load in some stray system thread.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com