Citrix, why?

NTDEV
1

Hi, All!

*** Fatal System Error: 0x000000ca
(0x0000000000000002,0xFFFFFA800E62F060,0x0000000000000000,0x0000000000000000)

Break instruction exception - code 80000003 (first chance)

A fatal system error has occurred.
Debugger entered on first try; Bugcheck callbacks have not been invoked.

A fatal system error has occurred.

Connected to Windows 7 7601 x64 target at (Wed Dec 16 16:54:57.377 2015 (UTC + 3:00)), ptr64 TRUE
Loading Kernel Symbols
…

Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.

…
…
…
Loading User Symbols

Loading unloaded module list
.
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck CA, {2, fffffa800e62f060, 0, 0}

*** ERROR: Module load completed but symbols could not be loaded for ctxusbm.sys
Probably caused by : ctxusbm.sys ( ctxusbm+1a418 )

Followup: MachineOwner

nt!RtlpBreakWithStatusInstruction:
fffff800`01a82490 cc int 3

First of all, I look into current IRP and print device stack to find filters.

1: kd> !devstack 0xfffffa80`0df9b050
!DevObj !DrvObj !DevExt ObjectName
fffffa800dfa4990 \Driver\MyDeviceFilterfffffa800dfa4ae0
fffffa800dfa5050 \Driver\usbhub fffffa800dfa51a0 000000c1
fffffa800df9e040 \Driver\MyDeviceFilterfffffa800df9e190

fffffa800df9b050 \Driver\usbuhci fffffa800df9b1a0 USBPDO-0
!DevNode fffffa800df9c840 :
DeviceInst is “USB\ROOT_HUB\5&3bb57b&0”
ServiceName is “usbhub”

Because I didn’t find any filters except mine I suspect something wrong. How other driver can intercept a PnP device request without filter? Yes, only by patch dispatch functions.

1: kd> !devobj 0xfffffa80`0dfa5050
Device object (fffffa800dfa5050) is for:
000000c1 \Driver\usbhub DriverObject fffffa800dfa2c60
Current Irp 00000000 RefCount 0 Type 00008600 Flags 00002840
Dacl fffff9a10008df51 DevExt fffffa800dfa51a0 DevObjExt fffffa800dfa8570
ExtensionFlags (0x00000800) DOE_DEFAULT_SD_PRESENT
Characteristics (0x00000180) FILE_AUTOGENERATED_DEVICE_NAME, FILE_DEVICE_SECURE_OPEN
AttachedDevice (Upper) fffffa800dfa4990 \Driver\MyDeviceFilter
AttachedTo (Lower) fffffa800df9e040 \Driver\MyDeviceFilter
Device queue is not busy.

1: kd> dt -b nt!_DRIVER_OBJECT MajorFunction fffffa800dfa2c60
+0x070 MajorFunction :
[00] 0xfffff88003801f60 [01] 0xfffff80001a701d4
[02] 0xfffff88003801f60 [03] 0xfffff80001a701d4
[04] 0xfffff80001a701d4 [05] 0xfffff80001a701d4
[06] 0xfffff80001a701d4 [07] 0xfffff80001a701d4
[08] 0xfffff80001a701d4 [09] 0xfffff80001a701d4
[10] 0xfffff80001a701d4 [11] 0xfffff80001a701d4
[12] 0xfffff80001a701d4 [13] 0xfffff80001a701d4
[14] 0xfffff88003801f60 [15] 0xfffff88001602910
[16] 0xfffff88003827540 [17] 0xfffff80001a701d4
[18] 0xfffff80001a701d4 [19] 0xfffff80001a701d4
[20] 0xfffff80001a701d4 [21] 0xfffff80001a701d4
[22] 0xfffff88003801f60 [23] 0xfffff88003801f60
[24] 0xfffff80001a701d4 [25] 0xfffff80001a701d4
[26] 0xfffff80001a701d4 [27] 0xfffff88001617750
1: kd> ln 0xfffff880`03801f60
Browse module
Set bu breakpoint

(fffff88003801f60) usbhub!UsbhGenDispatch | (fffff88003801ff8) usbhub!UsbhFdoSuccess
Exact matches:
usbhub!UsbhGenDispatch ()
1: kd> ln 0xfffff80001a701d4<br>Browse module<br>Set bu breakpoint<br><br>(fffff80001a701d4) nt!IopInvalidDeviceRequest | (fffff80001a701fc) nt!CcReleaseByteRangeFromWrite<br>Exact matches:<br> nt!IopInvalidDeviceRequest (<no parameter info>)<br>1: kd&gt; ln 0xfffff88001602910
Browse module
Set bu breakpoint

1: kd> lm a 0xfffff88001602910<br>Browse full module list<br>start end module name<br>fffff88001600000 fffff8800161f380 ctxusbm (no symbols) <br>1: kd&gt; lmDvmctxusbm<br>Browse full module list<br>start end module name<br>fffff88001600000 fffff880`0161f380 ctxusbm (no symbols)
Loaded symbol image file: ctxusbm.sys
Image path: \SystemRoot\system32\DRIVERS\ctxusbm.sys
Image name: ctxusbm.sys
Browse all global symbols functions data
Timestamp: Fri Jun 12 09:28:15 2015 (557A7BFF)
CheckSum: 00030F7D
ImageSize: 0001F380
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4

This is a new Citrix Receiver software… Will investigate further.

2

xxxxx@mail.ru wrote:

Hi, All!

*** Fatal System Error: 0x000000ca
(0x0000000000000002,0xFFFFFA800E62F060,0x0000000000000000,0x0000000000000000)

Break instruction exception - code 80000003 (first chance)
…
Use !analyze -v to get detailed debugging information.

Did you follow that advice from windbg? Bug check CA subcode 2 means
PnP was called to handle an invalid PDO, and the PDO is the second
parameter. I don’t see you dumping that !devobj to see what that memory
looks like.

One possibility here is that you had a memory overwrite that trashed the
contents of your PDEVICE_OBJECT, perhaps by passing it to a function
that expected your device context.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

That does seem patched and wrong. I actually work at citrix these days,
I’ll see if I can find out why.

Mark Roddy

On Wed, Dec 16, 2015 at 12:19 PM, wrote:

> Hi, All!
>
> Fatal System Error: 0x000000ca
>
> (0x0000000000000002,0xFFFFFA800E62F060,0x0000000000000000,0x0000000000000000)
>
> Break instruction exception - code 80000003 (first chance)
>
> A fatal system error has occurred.
> Debugger entered on first try; Bugcheck callbacks have not been invoked.
>
> A fatal system error has occurred.
>
> Connected to Windows 7 7601 x64 target at (Wed Dec 16 16:54:57.377 2015
> (UTC + 3:00)), ptr64 TRUE
> Loading Kernel Symbols
> …
>
> Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads
> that take too long.
> Run !sym noisy before .reload to track down problems loading symbols.
>
> …
> …
> …
> Loading User Symbols
>
> Loading unloaded module list
> .
>
>****************************************************************************
> *
> *
> * Bugcheck Analysis
> *
> *
> *
>
> ****************************************************************************
>
> Use !analyze -v to get detailed debugging information.
>
> BugCheck CA, {2, fffffa800e62f060, 0, 0}
>
> ERROR: Module load completed but symbols could not be loaded for
> ctxusbm.sys
> Probably caused by : ctxusbm.sys ( ctxusbm+1a418 )
>
> Followup: MachineOwner
> ---------
>
> nt!RtlpBreakWithStatusInstruction:
> fffff80001a82490 cc int 3<br>&gt;<br>&gt; First of all, I look into current IRP and print device stack to find<br>&gt; filters.<br>&gt;<br>&gt; 1: kd&gt; !devstack 0xfffffa800df9b050
> !DevObj !DrvObj !DevExt ObjectName
> fffffa800dfa4990 \Driver\MyDeviceFilterfffffa800dfa4ae0
> fffffa800dfa5050 \Driver\usbhub fffffa800dfa51a0 000000c1
> fffffa800df9e040 \Driver\MyDeviceFilterfffffa800df9e190
> > fffffa800df9b050 \Driver\usbuhci fffffa800df9b1a0 USBPDO-0
> !DevNode fffffa800df9c840 :
> DeviceInst is “USB\ROOT_HUB\5&3bb57b&0”
> ServiceName is “usbhub”
>
> Because I didn’t find any filters except mine I suspect something wrong.
> How other driver can intercept a PnP device request without filter? Yes,
> only by patch dispatch functions.
>
> 1: kd> !devobj 0xfffffa800dfa5050<br>&gt; Device object (fffffa800dfa5050) is for:<br>&gt; 000000c1 \Driver\usbhub DriverObject fffffa800dfa2c60<br>&gt; Current Irp 00000000 RefCount 0 Type 00008600 Flags 00002840<br>&gt; Dacl fffff9a10008df51 DevExt fffffa800dfa51a0 DevObjExt fffffa800dfa8570<br>&gt; ExtensionFlags (0x00000800) DOE_DEFAULT_SD_PRESENT<br>&gt; Characteristics (0x00000180) FILE_AUTOGENERATED_DEVICE_NAME,<br>&gt; FILE_DEVICE_SECURE_OPEN<br>&gt; AttachedDevice (Upper) fffffa800dfa4990 \Driver\MyDeviceFilter<br>&gt; AttachedTo (Lower) fffffa800df9e040 \Driver\MyDeviceFilter<br>&gt; Device queue is not busy.<br>&gt;<br>&gt; 1: kd&gt; dt -b nt!_DRIVER_OBJECT MajorFunction fffffa800dfa2c60<br>&gt; +0x070 MajorFunction :<br>&gt; [00] 0xfffff88003801f60
> [01] 0xfffff80001a701d4<br>&gt; [02] 0xfffff88003801f60
> [03] 0xfffff80001a701d4<br>&gt; [04] 0xfffff80001a701d4
> [05] 0xfffff80001a701d4<br>&gt; [06] 0xfffff80001a701d4
> [07] 0xfffff80001a701d4<br>&gt; [08] 0xfffff80001a701d4
> [09] 0xfffff80001a701d4<br>&gt; [10] 0xfffff80001a701d4
> [11] 0xfffff80001a701d4<br>&gt; [12] 0xfffff80001a701d4
> [13] 0xfffff80001a701d4<br>&gt; [14] 0xfffff88003801f60
> [15] 0xfffff88001602910<br>&gt; [16] 0xfffff88003827540
> [17] 0xfffff80001a701d4<br>&gt; [18] 0xfffff80001a701d4
> [19] 0xfffff80001a701d4<br>&gt; [20] 0xfffff80001a701d4
> [21] 0xfffff80001a701d4<br>&gt; [22] 0xfffff88003801f60
> [23] 0xfffff88003801f60<br>&gt; [24] 0xfffff80001a701d4
> [25] 0xfffff80001a701d4<br>&gt; [26] 0xfffff80001a701d4
> [27] 0xfffff88001617750<br>&gt; 1: kd&gt; ln 0xfffff88003801f60
> Browse module
> Set bu breakpoint
>
> (fffff88003801f60) usbhub!UsbhGenDispatch | (fffff88003801ff8)
> usbhub!UsbhFdoSuccess
> Exact matches:
> usbhub!UsbhGenDispatch ()
> 1: kd> ln 0xfffff80001a701d4<br>&gt; Browse module<br>&gt; Set bu breakpoint<br>&gt;<br>&gt; (fffff80001a701d4) nt!IopInvalidDeviceRequest | (fffff80001a701fc)<br>&gt; nt!CcReleaseByteRangeFromWrite<br>&gt; Exact matches:<br>&gt; nt!IopInvalidDeviceRequest (<no parameter info>)<br>&gt; 1: kd&gt; ln 0xfffff88001602910
> Browse module
> Set bu breakpoint
>
> 1: kd> lm a 0xfffff88001602910<br>&gt; Browse full module list<br>&gt; start end module name<br>&gt; fffff88001600000 fffff8800161f380 ctxusbm (no symbols)<br>&gt; 1: kd&gt; lmDvmctxusbm<br>&gt; Browse full module list<br>&gt; start end module name<br>&gt; fffff88001600000 fffff880`0161f380 ctxusbm (no symbols)
> Loaded symbol image file: ctxusbm.sys
> Image path: \SystemRoot\system32\DRIVERS\ctxusbm.sys
> Image name: ctxusbm.sys
> Browse all global symbols functions data
> Timestamp: Fri Jun 12 09:28:15 2015 (557A7BFF)
> CheckSum: 00030F7D
> ImageSize: 0001F380
> Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
>
> This is a new Citrix Receiver software… Will investigate further.
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>