Hello all,
I’m looking for some pointers on where to look next (not for somebody to debug the problem for me) in troubleshooting this BSOD I’m having with my TDI filter driver.
It happens *only* on Windows Server 2003 x86, while running under a specific version VMWare that I do not have, so I can’t perform any live debugging.
I am using a checked build and enabled all Driver Verifier options except Low Resources Simulation, but this doesn’t change any of the output crash dump output. For some periods of time, this crash will occur reliably and then the next day it will completely disappear. Another interesting thing is that none of the bugcheck parameters ever change. The backtrace (with correct symbols for everything, except vmxnet which I can’t get) doesn’t help very much other than letting me know it’s related to something happening on TCP Receive.
Here is the analyze -v output:
kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 1200001c, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: bac169c9, address which referenced memory
Debugging Details:
DBGHELP: C:\WinDDK\7600.16385.1\Debuggers\sym\ntkrnlpa.exe\45EBDEFD247000\ntkrnlpa.exe - OK
DBGHELP: C:\WinDDK\7600.16385.1\Debuggers\sym\tcpip.sys\47270DA661000\tcpip.sys - OK
DBGHELP: C:\WinDDK\7600.16385.1\Debuggers\sym\NDIS.sys\42435DFA36000\NDIS.sys - OK
SYMSRV: C:\WinDDK\7600.16385.1\Debuggers\sym\vmxnet.sys\4BA19E645b00\vmxnet.sys not found
SYMSRV: http://msdl.microsoft.com/download/symbols/vmxnet.sys/4BA19E645b00/vmxnet.sys not found
READ_ADDRESS: 1200001c
CURRENT_IRQL: 2
FAULTING_IP:
tcpip!IndicateData+301
bac169c9 ff700c push dword ptr [eax+0Ch]
DEFAULT_BUCKET_ID: DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: Idle
TRAP_FRAME: 80894160 – (.trap 0xffffffff80894160)
ErrCode = 00000000
eax=12000010 ebx=85d8ad6c ecx=00000000 edx=86bbaf48 esi=85eae8f8 edi=86085d80
eip=bac169c9 esp=808941d4 ebp=80894218 iopl=0 nv up ei pl zr na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010246
tcpip!IndicateData+0x301:
bac169c9 ff700c push dword ptr [eax+0Ch] ds:0023:1200001c=???
Resetting default scope
LAST_CONTROL_TRANSFER: from bac169c9 to 808860a9
STACK_TEXT:
80894160 bac169c9 badb0d00 86bbaf48 000005b4 nt!KiTrap0E+0x2a1
80894218 bac197ba 85c09738 00001050 860d2008 tcpip!IndicateData+0x301
808942d8 bac17f8f 85deedb0 0900000a fe00000a tcpip!TCPRcv+0x93f
80894338 bac2d903 00000020 85deedb0 bac1953d tcpip!DeliverToUser+0x189
808943b4 bac2d6d4 860d2008 85deedb0 85cec80e tcpip!DeliverToUserEx+0x93f
8089446c bac17c56 85deedb0 85cec822 000005c8 tcpip!IPRcvPacket+0x6c7
808944ac bac17d58 00000000 860d7828 85cec800 tcpip!ARPRcvIndicationNew+0x149
808944e8 f715f1d9 85deee70 00000000 00000002 tcpip!ARPRcvPacket+0x68
8089453c f77a2864 8615ba60 8605c42c 00000002 NDIS!ethFilterDprIndicateReceivePacket+0x318
WARNING: Stack unwind information not available. Following frames may be wrong.
8089459c f715412f 00000072 ffdffa40 8605c2d0 vmxnet+0x3864
808945b0 8082f576 8605c2d0 8605c2bc 00000000 NDIS!ndisMDpcX+0x1f
80894600 808873d7 00000000 0000000e 00000000 nt!KiRetireDpcList+0xca
80894604 00000000 0000000e 00000000 00000000 nt!KiIdleLoop+0x2f
STACK_COMMAND: kb
FOLLOWUP_IP:
vmxnet+3864
f77a2864 8b4dd8 mov ecx,dword ptr [ebp-28h]
SYMBOL_STACK_INDEX: 9
SYMBOL_NAME: vmxnet+3864
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: vmxnet
IMAGE_NAME: vmxnet.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 4ba19e64
FAILURE_BUCKET_ID: 0xD1_VRF_vmxnet+3864
BUCKET_ID: 0xD1_VRF_vmxnet+3864
Followup: MachineOwner
=====================================================
And I realize that isn’t incredibly enlightening, so here’s kd output:
kd> kd
80894160 80894218 nt!KiDoubleFaultStack+0x2968
80894164 bac169c9 tcpip!IndicateData+0x301
80894168 badb0d00 rdpdr!RxLockOperationCompletion+0x1c2
8089416c 86bbaf48
80894170 000005b4
80894174 808941b4 nt!KiDoubleFaultStack+0x2904
80894178 b9eb0cd9 myfilter32!tcp_TdiReceiveEventHandler+0x309 [c:\dev\driver_tdi\tcprecv.c @ 317]
8089417c 87086eb8
80894180 80894210 nt!KiDoubleFaultStack+0x2960
80894184 000005b4
80894188 c0000016
8089418c 00000000
80894190 85fe0000
80894194 00000023
80894198 c0000023
8089419c 86bbaf48
808941a0 00000000
808941a4 12000010
808941a8 87430e48
808941ac ffffffff
808941b0 babd0030
808941b4 86085d80
808941b8 85eae8f8
808941bc 85d8ad6c
808941c0 80894218 nt!KiDoubleFaultStack+0x2968
808941c4 00000000
808941c8 bac169c9 tcpip!IndicateData+0x301
808941cc 00000008
808941d0 00010246
808941d4 86bbaf48
808941d8 bac1c336 tcpip!TCPCancelRequest
808941dc 00000000
808941e0 85eae8f8
808941e4 000005b4
808941e8 00000000
808941ec 85eae8f8
808941f0 0000005a
808941f4 00000013
808941f8 00000000
808941fc bac192ba tcpip!FindTCB+0xe7
80894200 00000014
80894204 87430e48
80894208 00000000
8089420c b9eb09d0 myfilter32!tcp_TdiReceiveEventHandler [c:\dev\driver_tdi\tcprecv.c @ 202]
80894210 86bbaf48
80894214 000005b4
80894218 808942d8 nt!KiDoubleFaultStack+0x2a28
8089421c bac197ba tcpip!TCPRcv+0x93f
80894220 85c09738
80894224 00001050
80894228 860d2008
8089422c 000005b4
80894230 860d2008
80894234 85cec80e
80894238 85deedb0
8089423c 00008900
80894240 00000011
80894244 00000000
80894248 00000000
8089424c 85d03822
80894250 8089428c nt!KiDoubleFaultStack+0x29dc
80894254 bac14613 tcpip!UDPRcv+0x26b
80894258 bac14625 tcpip!UDPRcv+0x27d
8089425c 860d2008
80894260 85deedb0
80894264 85d03822
80894268 ff00000a
8089426c 0900000a
80894270 00008900
80894274 00001f83
80894278 85e9dab8
8089427c 0900000a
80894280 00118900
80894284 0000004c
80894288 00ffffff
8089428c 808942e0 nt!KiDoubleFaultStack+0x2a30
80894290 bac14366 tcpip!BCastRcv+0x9d
80894294 00000000
80894298 ff00000a
8089429c 1100000a
808942a0 0900000a
808942a4 00000000
808942a8 00000000
808942ac 00000000
808942b0 00000000
808942b4 00000000
808942b8 85dda670
808942bc 85dda670
808942c0 01894414
808942c4 1383f7ce
808942c8 7ea1af9c
808942cc 00006ef8
808942d0 00000000
808942d4 00001050
808942d8 80894338 nt!KiDoubleFaultStack+0x2a88
808942dc bac17f8f tcpip!DeliverToUser+0x189
808942e0 85deedb0
808942e4 0900000a
808942e8 fe00000a
808942ec 00000000
808942f0 0900000a
808942f4 85cec80e
808942f8 00000014
808942fc 860d2008
80894300 000005b4
80894304 80894202 nt!KiDoubleFaultStack+0x2952
80894308 00000006
8089430c 7ea1af42
80894310 85deedb0
80894314 00000014
80894318 00000000
8089431c 00000000
80894320 00000000
80894324 85cec822
80894328 00000000
8089432c 80894202 nt!KiDoubleFaultStack+0x2952
80894330 85cec822
80894334 00000000
80894338 808943b4 nt!KiDoubleFaultStack+0x2b04
8089433c bac2d903 tcpip!DeliverToUserEx+0x93f
80894340 00000020
80894344 85deedb0
80894348 bac1953d tcpip!TCPRcv
8089434c 00000014
80894350 000005c8
80894354 000005c8
80894358 80894414 nt!KiDoubleFaultStack+0x2b64
8089435c 000005c8
80894360 00000500
80894364 85deedb0
80894368 000005dc
8089436c 860d2008
80894370 00000001
80894374 85deedb0
80894378 00000000
8089437c 00000000
80894380 00000000
80894384 00000000
80894388 00000000
8089438c 00000000
80894390 00000000
80894394 00000000
80894398 bac51914 tcpip!FQBlock+0x14
8089439c 00000000
808943a0 00000000
808943a4 00000001
808943a8 bac51914 tcpip!FQBlock+0x14
808943ac 00000000
808943b0 00000060
808943b4 8089446c nt!KiDoubleFaultStack+0x2bbc
808943b8 bac2d6d4 tcpip!IPRcvPacket+0x6c7
808943bc 860d2008
808943c0 85deedb0
808943c4 85cec80e
808943c8 85cec80e
808943cc 000d2008
808943d0 000005c8
808943d4 80894414 nt!KiDoubleFaultStack+0x2b64
808943d8 860d7828
808943dc 00000500
808943e0 00000000
808943e4 85cec80e
808943e8 85deee70
808943ec 000005dc
808943f0 00000000
808943f4 00000000
808943f8 00000000
808943fc 00000000
80894400 00000000
80894404 8081ba07 nt!IoGetStackLimits+0x1b
80894408 80894430 nt!KiDoubleFaultStack+0x2b80
8089440c 8089442c nt!KiDoubleFaultStack+0x2b7c
80894410 00000000
80894414 00000000
80894418 80894418 nt!KiDoubleFaultStack+0x2b68
8089441c 02003e00
80894420 809b2d39 nt!ViDeadlockCheckStackLimits+0x1b
80894424 80894430 nt!KiDoubleFaultStack+0x2b80
80894428 8089442c nt!KiDoubleFaultStack+0x2b7c
8089442c 808946a0 nt!KiDoubleFaultStack+0x2df0
80894430 808918b0 nt!KiDoubleFaultStack
80894434 85cec80e
80894438 00000000
8089443c 00000000
80894440 00000000
80894444 00000000
80894448 00000000
8089444c 00000000
80894450 860d7828
80894454 00000001
80894458 85deedb0
8089445c 000005c8
80894460 00000014
80894464 85cec80e
80894468 00c18be8
8089446c 808944ac nt!KiDoubleFaultStack+0x2bfc
80894470 bac17c56 tcpip!ARPRcvIndicationNew+0x149
80894474 85deedb0
80894478 85cec822
8089447c 000005c8
80894480 00000500
80894484 860d7828
80894488 00000000
8089448c 00000000
80894490 0000000e
80894494 8619df08
80894498 808944f4 nt!KiDoubleFaultStack+0x2c44
8089449c 00000000
808944a0 860d7828
808944a4 808944f4 nt!KiDoubleFaultStack+0x2c44
808944a8 0000000e
808944ac 808944e8 nt!KiDoubleFaultStack+0x2c38
808944b0 bac17d58 tcpip!ARPRcvPacket+0x68
808944b4 00000000
808944b8 860d7828
808944bc 85cec800
808944c0 0000000e
808944c4 85cec80e
808944c8 000005dc
808944cc 000005dc
808944d0 8619df08
808944d4 808944f4 nt!KiDoubleFaultStack+0x2c44
808944d8 85feb928
808944dc 860d7828
808944e0 8615ba60
808944e4 8619df08
808944e8 8089453c nt!KiDoubleFaultStack+0x2c8c
808944ec f715f1d9 NDIS!ethFilterDprIndicateReceivePacket+0x318
808944f0 85deee70
808944f4 00000000
808944f8 00000002
808944fc 86033000
80894500 85cc6000
80894504 00000000
80894508 00022030
8089450c 00000000
80894510 8605c42c
80894514 860d7870
80894518 00000000
8089451c 8089453c nt!KiDoubleFaultStack+0x2c8c
80894520 00000000
80894524 000005ea
80894528 00000002
8089452c 860d77f0
80894530 85cec800
80894534 860302a8
80894538 00cc6000
8089453c 8089459c nt!KiDoubleFaultStack+0x2cec
80894540 f77a2864 vmxnet+0x3864
80894544 8615ba60
80894548 8605c42c
8089454c 00000002
80894550 8615ba60
80894554 8605c2bc
80894558 f7154108 NDIS!ndisMDpcX
8089455c 00000072
=====================================================
In the output of kd, I see two symbols from my own module but they point to the very beginning and the very end of the TCP receive event handler function for whatever reason.
Any help offered here will be greatly appreciated! Thanks!