Crash while removing flow context

WINDBG
1

Hi ,

In my WFP driver, I am getting BSOD while unloading driver. Following is the dump with driver verifier:

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000018, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff802c9c517b0, address which referenced memory

Debugging Details:

READ_ADDRESS: unable to get nt!MmSpecialPoolStart
unable to get nt!MmSpecialPoolEnd
unable to get nt!MmPagedPoolEnd
unable to get nt!MmNonPagedPoolStart
unable to get nt!MmSizeOfNonPagedPoolInBytes
0000000000000018

CURRENT_IRQL: 2

FAULTING_IP:
tcpip!TcpValidateReceive+64
fffff802`c9c517b0 837f1814 cmp dword ptr [rdi+18h],14h

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

BUGCHECK_STR: AV

PROCESS_NAME: System

TAG_NOT_DEFINED_c000000f: FFFFF80194081FB0

TRAP_FRAME: fffff801940809a0 – (.trap 0xfffff801940809a0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=ffffdb0c07b68030 rbx=0000000000000000 rcx=0000000000000002
rdx=ffffdb0c03251000 rsi=0000000000000000 rdi=0000000000000000
rip=fffff802c9c517b0 rsp=fffff80194080b30 rbp=ffffdb0c0310b8f0
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000
r11=fffff80194080990 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
tcpip!TcpValidateReceive+0x64:
fffff802c9c517b0 837f1814 cmp dword ptr [rdi+18h],14h ds:0000000000000018=???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80191e16529 to fffff80191e03430

STACK_TEXT:

fffff80194080b30 fffff802c9c2c6dc : 0000000000056069 0000000000000001 ffffdb0c03251000 0000000000000000 : tcpip!TcpValidateReceive+0x64
fffff80194080bd0 fffff802c9c587b2 : ffffdb0c0310b8f0 0000000000000000 0000000000000000 ffffdb0c0310b890 : tcpip!TcpReceive+0x42c
fffff80194080ca0 fffff802c9c097fe : 0000000000000000 fffff80191c12356 0000000000000000 0000000000000000 : tcpip!TcpNlClientReceiveDatagrams+0x22
fffff80194080ce0 fffff802c9c09453 : 0000000000000000 0000000000000000 fffff80194080e70 fffff802c9da4000 : tcpip!IppDeliverListToProtocol+0x6a
fffff80194080da0 fffff802c9c29c8f : 0000000000000000 fffff80191e0455d ffffdb0c03ec4180 0000000000000000 : tcpip!IppProcessDeliverList+0x63
fffff80194080e10 fffff802c9c29767 : fffff802c9da4000 ffffdb0c0314a940 0000000000000000 ffffdb0c08188a00 : tcpip!IppReceiveHeaderBatch+0x25f
fffff80194080f10 fffff802c9be6b7c : ffffdb0c08189530 ffffdb0c07b68030 0000000000000001 0000000000000000 : tcpip!IppFlcReceivePacketsCore+0x317
fffff80194081030 fffff802c9be68b6 : ffffdb0c07b68030 0000000000000000 fffff80194081148 0000000000000000 : tcpip!IpFlcReceivePackets+0xc
fffff80194081060 fffff802c9c2a798 : ffffdb0c08180002 fffff80100000001 fffff802c9c44c70 0000000000000001 : tcpip!FlpReceiveNonPreValidatedNetBufferListChain+0x256
fffff80194081140 fffff80191d1e7fb : ffffdb0c03129860 ffffdb0c039c2700 fffff802c9c2a640 fffff80194081410 : tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x158
fffff80194081270 fffff80191d1e75d : ffffdb0c03132980 0000000000000000 0000000000000002 0000000000000000 : nt!KeExpandKernelStackAndCalloutInternal+0x8b
fffff801940812c0 fffff802c9c44907 : 0000000000000000 fffff802c904652d ffffdb0c05aecb70 ffffdb0c04bfffc0 : nt!KeExpandKernelStackAndCalloutEx+0x1d
fffff80194081300 fffff802c9c43f83 : 0000000000000001 fffff80194081460 ffffdb0c08187010 0000000000000000 : tcpip!NetioExpandKernelStackAndCallout+0x87
fffff80194081360 fffff802c8f14fae : ffffdb0c041051a0 ffffdb0c08194561 0000000000000001 0000000000000001 : tcpip!FlReceiveNetBufferListChain+0x243
fffff80194081590 fffff802c8f14c81 : ffffdb0c08157a01 fffff802c83f0000 0000000000000000 fffff80200000001 : ndis!ndisMIndicateNetBufferListsToOpen+0x13e
fffff80194081660 fffff802c8f15583 : ffffdb0c041051a0 fffff80194081801 fffff802c8f14a50 ffffdb0c041051a0 : ndis!ndisMTopReceiveNetBufferLists+0x231
fffff80194081760 fffff802c8f1498e : 000a307830206e6f 0000000000000000 0000000000000000 0000000000000000 : ndis!ndisCallReceiveHandler+0x43
fffff801940817b0 fffff802cad4f979 : ffffdb0c04eb71c0 0000000000000001 ffffdb0c05603750 fffff802cab93cf4 : ndis!NdisMIndicateReceiveNetBufferLists+0x5ae
fffff80194081920 fffff802cacdbc71 : ffffdb0c04ec8120 0000000000000000 0000000000000000 ffffdb0c0558a000 : NETwew01!doApiIndicateReceiveNbl+0x9d
fffff80194081960 fffff802cab94515 : ffffdb0c03229930 000000003534a77c 0000000000000001 ffffdb0c009fd1ff : NETwew01!mStatFldConstructor+0x161
fffff801940819d0 fffff802cab8f662 : 0000000000000001 fffff80191d4c596 fffff80191ffc5ff 0000000000000000 : NETwew01!rfdQueueProcessFragments+0x1e5
fffff80194081a70 fffff802cab8ace0 : fffff80191b7e100 ffffdb0c05652b90 ffffdb0c02250000 fffff80191d42054 : NETwew01!isrHandlerRoutineInta+0x1f2
fffff80194081af0 fffff802cad56076 : ffffdb0c05825000 fffff80191d422df 0000000000000002 ffffdb0c01bfb040 : NETwew01!alonExInterruptHandlerRoutine+0x1c
fffff80194081b20 fffff802c8f0a4cd : ffffdb0c01bfb040 fffff80191d41e9c ffffdb0c0582500e ffffdb0c06155001 : NETwew01!oscHandleInterrupt+0x12
fffff80194081b50 fffff80191d53ee2 : 0000000000000000 ffffdb0c014df000 ffffdb0c0614f5e0 fffff80100000002 : ndis!ndisInterruptDpc+0x17d
fffff80194081c70 fffff80191d535df : 000000000000001c 0000000000000000 000000000026a7c7 fffff80191b7e180 : nt!KiExecuteAllDpcs+0x1d2
fffff80194081db0 fffff80191e0a725 : 0000000000000000 fffff80191b7e180 ffff9201f52371f0 0000000000000000 : nt!KiRetireDpcList+0xdf
fffff80194081fb0 fffff80191e0a530 : ffffa3e47397a97f ffff99081af17000 fffffecc840f75a0 0000000000006a8b : nt!KxRetireDpcList+0x5
ffff9201f5237140 fffff80191e07a56 : 0000000000006a8b 0000000000000000 0000000000000000 0000000000000000 : nt!KiDispatchInterruptContinue
ffff9201f5237170 fffff80191cf132c : 0000000000000002 fffff801920146a0 3fffffffffffffff ffff9201f52376c0 : nt!KiDpcInterrupt+0x3a6
ffff9201f5237300 fffff80191cf1670 : 0000000000000000 fffff801920146a0 0000000000000000 0000000000000000 : nt!MiWalkPageTablesRecursively+0x31c
ffff9201f52373c0 fffff80191cf1670 : 0000000000000000 fffff801920146a0 0000000000000000 0000000000000000 : nt!MiWalkPageTablesRecursively+0x660
ffff9201f5237480 fffff80191cf1670 : 0000000000000000 0000000000000001 0000000000000001 0000000000000000 : nt!MiWalkPageTablesRecursively+0x660
ffff9201f5237540 fffff80191cf0f4f : fffff80192014300 0000000000000300 ffff960000007000 ffff9201f5237620 : nt!MiWalkPageTablesRecursively+0x660
ffff9201f5237600 fffff80191c9c8c9 : fffff80100000000 fffff80191c9f010 fffff801920142b0 fffff80191eb4b0c : nt!MiWalkPageTables+0x20f
ffff9201f52376a0 fffff80191ea81ae : 0000000000000001 fffff801920146a0 fffff801920146a0 0000000000000001 : nt!MiEmptyWorkingSet+0x105
ffff9201f5237840 fffff80191ea92ad : ffffb08200000000 000000000000c99c 0000000000000000 fffff80191d31474 : nt!MiEmptyTargetedWorkingSet+0x6e
ffff9201f5237890 fffff8019242ea3a : 0000000000000000 fffff80100000000 fffff801922035a0 ffffdb0c0084be38 : nt!MiTrimAllSystemPagableMemory+0xdd
ffff9201f52378e0 fffff80192443555 : ffff9201f5232000 ffff9201f5238000 fffff8018f413202 00000ae35fb30f8a : nt!MmVerifierTrimMemory+0x6a
ffff9201f5237910 fffff80192443206 : fffff8018f413210 ffffdb0c0a9cbe60 0000000000000000 0000000000000600 : nt!ViKeRaiseIrqlSanityChecks+0xa5
ffff9201f5237950 fffff80192441dd2 : 0000000000000000 fffff8019242f4ea ffffb082cb52ef70 fffff8018f410000 : nt!ViKeAcquireSpinLockRaiseToDpcCommon+0x2a
ffff9201f5237980 fffff8018f40de33 : 0000000000000000 ffffdb0c0a9cbe60 fffff8018f4119f8 fffff8018f3f0016 : nt!VerifierKeAcquireSpinLockRaiseToDpc+0x12
ffff9201f52379c0 fffff8018f40f90b : fffff8018f4113c8 fffff8010000010f fffff8018f4119f8 0000000000000043 : mydriver!EmptyFlowsContextQ+0xb3
ffff9201f5237a20 fffff8018f40daa6 : 0000000700020018 0000000000000043 fffff8018f4119f8 0000000000000000 : mydriver!UnregisterCallouts+0x9b
ffff9201f5237a60 fffff802c90fc43b : ffffdb0c0a9cbe60 0000000000000000 ffffdb0c039c2700 fffff80191cc98a4 : mydriver!DriverUnload+0x86
ffff9201f5237ad0 fffff8019244b661 : ffffdb0c039c2700 0000000000000000 ffffdb0c0088e210 ffff9201f8efb770 : VerifierExt!xdv_DriverUnload_wrapper+0x7b
ffff9201f5237b00 fffff801922eb96b : 0000000000000010 fffff801922035a0 0000000000000010 0000000000210246 : nt!ViGenericDriverUnload+0x31
ffff9201f5237b40 fffff80191cc94d5 : ffffdb0c0088e210 ffffdb0c039c2700 fffff802c985cf00 ffffdb0c039c2700 : nt!IopLoadUnloadDriver+0xe83cb
ffff9201f5237b80 fffff80191da4b87 : ffffdb0c039c2700 0000000000000080 ffffdb0c0089b040 ffffdb0c039c2700 : nt!ExpWorkerThread+0xf5
ffff9201f5237c10 fffff80191e0abe6 : ffff8000d87e9180 ffffdb0c039c2700 fffff80191da4b40 0000000000000000 : nt!PspSystemThreadStartup+0x47
ffff9201f5237c60 0000000000000000 : ffff9201f5238000 ffff9201f5232000 0000000000000000 0000000000000000 : nt!KiStartSystemThread+0x16.

Can any one suggest what could be the possible reason?

2

"xxxxx@gmail.com windbg"@lists.osr.com wrote:

In my WFP driver, I am getting BSOD while unloading driver. Following is the dump with driver verifier:

You are causing tcpip.sys to dereference a null pointer.  My guess is
that you have allowed yourself to be unloaded without unhooking yourself
from whatever callbacks you registered.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

I have verified that all the callouts are unregistered before unloading driver.
In the driver i have registered flow establish callout for “Stream layer” and “Datagram layer”. So in driver unload before unregistering these callouts I am removing and freeing the previously associated flow context. And this is the point where crash happens.
The line at which dump occurs is “KeAquireSpinlock()” . This is the lock acquired before accessing flow context list.

In the dump following part of call stack is from my driver:
fffff8018f4119f8 0000000000000043 : mydriver!EmptyFlowsContextQ+0xb3
ffff9201f5237a20 fffff8018f40daa6 : 0000000700020018 0000000000000043
fffff8018f4119f8 0000000000000000 : mydriver!UnregisterCallouts+0x9b
ffff9201f5237a60 fffff802c90fc43b : ffffdb0c0a9cbe60 0000000000000000
ffffdb0c039c2700 fffff80191cc98a4 : mydriver!DriverUnload+0x86

4

Enable Driver Verifier for your driver and tcpip.sys. You might be
corrupting something that Verifier can catch earlier.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@windbg…

I have verified that all the callouts are unregistered before unloading
driver.
In the driver i have registered flow establish callout for “Stream layer”
and “Datagram layer”. So in driver unload before unregistering these
callouts I am removing and freeing the previously associated flow context.
And this is the point where crash happens.
The line at which dump occurs is “KeAquireSpinlock()” . This is the lock
acquired before accessing flow context list.

In the dump following part of call stack is from my driver:
fffff8018f4119f8 0000000000000043 : mydriver!EmptyFlowsContextQ+0xb3
ffff9201f5237a20 fffff8018f40daa6 : 0000000700020018 0000000000000043
fffff8018f4119f8 0000000000000000 : mydriver!UnregisterCallouts+0x9b
ffff9201f5237a60 fffff802c90fc43b : ffffdb0c0a9cbe60 0000000000000000
ffffdb0c039c2700 fffff80191cc98a4 : mydriver!DriverUnload+0x86

5

On Feb 20, 2018, at 9:54 PM, xxxxx@gmail.com xxxxx@lists.osr.com wrote:

I have verified that all the callouts are unregistered before unloading driver.
In the driver i have registered flow establish callout for “Stream layer” and “Datagram layer”. So in driver unload before unregistering these callouts I am removing and freeing the previously associated flow context. And this is the point where crash happens.

I don’t know the flow of a filter driver like this, but don’t you need to guarantee that the contexts remain valid until the callouts are unregistered? Otherwise, you might get a callback, and here you’ve freed all of the structures.
—
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

6

Thanks for the helpful suggestions.

I reviewed the code and found there was synchronization issue between “Emptyflowcontext” and “Flowdelete” function. So we corrected that and issue got solved.

Thanks