Crash in netio.sys - Bug Check DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1) - NetioDereferenceNetBufferList

WINDBG
1

Hi,

We have seen a crash on one of our customer setup. We have a volume filter
driver(no FS drivers, no network drivers) and some applications
that transfer some files over LAN/WAN. The system is Windows 2008 R2,
standard. Accidentally when our product runs, crash happens. This is the
output from !analyze -v.

Crash is seen in the stack of tcpip and netio.sys. Is anything suspicious
from below analysis? anybody seen this kind of crash? please suggest.

kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck
Analysis *
*
*
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff880018897a0, address which referenced memory
Debugging Details:

Page 12f5c7 not present in the dump file. Type “.hh dbgerr004” for details
PEB is paged out (Peb.Ldr = 000007fffffd3018). Type ".hh dbgerr001" for details PEB is paged out (Peb.Ldr = 000007fffffd3018). Type “.hh dbgerr001” for
details
READ_ADDRESS: 0000000000000000
CURRENT_IRQL: 2
FAULTING_IP:
tcpip! ?? ::FNODOBFM::string'+56f4 fffff880018897a0 488b01 mov rax,qword ptr [rcx]
DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: vacp.exe
TRAP_FRAME: fffff800014202c0 – (.trap 0xfffff800014202c0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa8003d46d40 rbx=fffffa8004680600 rcx=0000000000000000
rdx=fffffa8003d46d41 rsi=0000000000000001 rdi=fffff8800185cbb4
rip=fffff880018897a0 rsp=fffff80001420450 rbp=0000000000000000
r8=fffffa8003d46d40 r9=00000000000000d0 r10=fffff80001835b80
r11=fffffa8004680540 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
tcpip! ?? ::FNODOBFM::string'+0x56f4: fffff880018897a0 488b01 mov rax,qword ptr [rcx]
ds:07ff:0000=???
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff800016b9ca9 to fffff800016ba740
STACK_TEXT:
fffff80001420178 fffff800016b9ca9 : 000000000000000a 0000000000000000
0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff80001420180 fffff800016b8920 : 0000000000000001 fffffa8004802a00
fffffa8004842010 fffffa80040c2000 : nt!KiBugCheckDispatch+0x69
fffff800014202c0 fffff880018897a0 : fffffa8004802a00 fffff88000e67c08
00000000206c644d fffffa800650fc90 : nt!KiPageFault+0x260
fffff80001420450 fffff88000e626a6 : fffffa8004802a00 0000000001834e80
0000000000000000 0000000000000000 : tcpip! ?? ::FNODOBFM::string'+0x56f4 fffff800014204a0 fffff88000e6035d : fffffa80066d2e20 fffffa800650fc90 0000000000000000 fffff80001420400 : NETIO!NetioDereferenceNetBufferList+0x86 fffff800014204d0 fffff8800183bae6 : fffff80001834e80 0000000000000000 0000000000000000 fffffa8004802a00 : NETIO!NetioDereferenceNetBufferListChain+0x2dd fffff80001420550 fffff88001829f47 : fffffa800650fc90 0000000000000000 fffffa8003fe1b40 fffffa80065a1b80 : tcpip!IppCompleteAndFreePacketList+0xc6 fffff80001420580 fffff88001829cc0 : 0000000000000000 fffffa80040cd860 0000000000000001 0000000000000000 : tcpip!IppCleanupMfe+0x77 fffff800014205b0 fffff88001859918 : fffffa80040cd860 0000000000000000 fffff80001420658 fffffa8003fe1b40 : tcpip!IppDereferenceMfe+0x20 fffff800014205e0 fffff880018597aa : fffff80001420830 0000000000000001 fffffa8003fe1b58 0000000000000000 : tcpip!IppMfeSetTimeOut+0xf8 fffff80001420700 fffff8800185861a : 0000000000000000 fffff80001420830 0000000000000001 fffff800016bf1fa : tcpip!IppCompartmentSetTimeout+0x9a fffff80001420770 fffff800016c629e : fffff80001420860 fffff80000000000 0000000040aa0000 0000000000000000 : tcpip!IppTimeout+0x5a fffff800014207a0 fffff800016c5dd6 : fffffa8003d3d3f0 fffffa8003d3d3f0 0000000000000000 0000000000000000 : nt!KiProcessTimerDpcTable+0x66 fffff80001420810 fffff800016c64be : 000000ccf14d88de fffff80001420e88 000000000055f58b fffff800018383e8 : nt!KiProcessExpiredTimerList+0xc6 fffff80001420e60 fffff800016c5cb7 : fffffa8004823ac4 fffff8000055f58b 0000000000000000 000000000000008b : nt!KiTimerExpiration+0x1be fffff80001420f00 fffff800016c0865 : 0000000000000000 fffffa8006852680 0000000000000000 fffff88000ec0c50 : nt!KiRetireDpcList+0x277 fffff80001420fb0 fffff800016c067c : 000000000002625a fffff80001615090 0000000000000000 fffff88004466ca0 : nt!KxRetireDpcList+0x5 fffff88004466be0 fffff80001704113 : fffff800016b6c60 fffff800016b6ccc 00000000001e99f8 fffff800016303c0 : nt!KiDispatchInterruptContinue fffff88004466c10 fffff800016b6ccc : 00000000001e99f8 fffff800016303c0 0000000000000001 fffffa8006616070 : nt!KiDpcInterruptBypass+0x13 fffff88004466c20 000007fefddeebed : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiInterruptDispatchNoLock+0x1fc 00000000030dd220 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x7fefddeebed

STACK_COMMAND: kb
FOLLOWUP_IP:
NETIO!NetioDereferenceNetBufferList+86
fffff880`00e626a6 4885ff test rdi,rdi
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: NETIO!NetioDereferenceNetBufferList+86
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: NETIO
IMAGE_NAME: NETIO.SYS
DEBUG_FLR_IMAGE_TIMESTAMP: 4a5bc18a
FAILURE_BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86
BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86
Followup: MachineOwner

kd> .trap 0xfffff800014202c0
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa8003d46d40 rbx=fffffa8004680600 rcx=0000000000000000
rdx=fffffa8003d46d41 rsi=0000000000000001 rdi=fffff8800185cbb4
rip=fffff880018897a0 rsp=fffff80001420450 rbp=0000000000000000
r8=fffffa8003d46d40 r9=00000000000000d0 r10=fffff80001835b80
r11=fffffa8004680540 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
tcpip! ?? ::FNODOBFM::string'+0x56f4: fffff880018897a0 488b01 mov rax,qword ptr [rcx]
ds:07ff:0000=???

2

>Crash is seen in the stack of tcpip and netio.sys. Is anything suspicious

from below analysis? anybody seen this kind of crash? please >suggest.

This is a NULL pointer dereference. Because your driver is not anywhere in
the crashing stack, there are two likely theories that come to mind:

  1. You have corrupted memory (buffer overrun, use after free, etc.). Does
    your testing pass with Driver Verifier’s Special Pool option enabled on your
    driver?

  2. You have a memory leak that is exhausting memory. Does !vm 21 show
    anything interesting?

-scott

–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

“suresh chepuri” wrote in message
news:xxxxx@windbg…
Hi,

We have seen a crash on one of our customer setup. We have a volume filter
driver(no FS drivers, no network drivers) and some applications that
transfer some files over LAN/WAN. The system is Windows 2008 R2, standard.
Accidentally when our product runs, crash happens. This is the output from
!analyze -v.

Crash is seen in the stack of tcpip and netio.sys. Is anything suspicious
from below analysis? anybody seen this kind of crash? please suggest.

kd> !analyze -v

Bugcheck Analysis



******
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff880018897a0, address which referenced memory
Debugging Details:
------------------
Page 12f5c7 not present in the dump file. Type “.hh dbgerr004” for details
PEB is paged out (Peb.Ldr = 000007fffffd3018). Type ".hh dbgerr001" for <br>details<br>PEB is paged out (Peb.Ldr = 000007fffffd3018). Type “.hh dbgerr001” for
details
READ_ADDRESS: 0000000000000000
CURRENT_IRQL: 2
FAULTING_IP:
tcpip! ?? ::FNODOBFM::string'+56f4<br>fffff880018897a0 488b01 mov rax,qword ptr [rcx]
DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: vacp.exe
TRAP_FRAME: fffff800014202c0 – (.trap 0xfffff800014202c0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa8003d46d40 rbx=fffffa8004680600 rcx=0000000000000000
rdx=fffffa8003d46d41 rsi=0000000000000001 rdi=fffff8800185cbb4
rip=fffff880018897a0 rsp=fffff80001420450 rbp=0000000000000000
r8=fffffa8003d46d40 r9=00000000000000d0 r10=fffff80001835b80
r11=fffffa8004680540 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
tcpip! ?? ::FNODOBFM::string'+0x56f4:<br>fffff880018897a0 488b01 mov rax,qword ptr [rcx]
ds:07ff:0000=???
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff800016b9ca9 to fffff800016ba740
STACK_TEXT:
fffff80001420178 fffff800016b9ca9 : 000000000000000a 0000000000000000
0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff80001420180 fffff800016b8920 : 0000000000000001 fffffa8004802a00
fffffa8004842010 fffffa80040c2000 : nt!KiBugCheckDispatch+0x69
fffff800014202c0 fffff880018897a0 : fffffa8004802a00 fffff88000e67c08
00000000206c644d fffffa800650fc90 : nt!KiPageFault+0x260
fffff80001420450 fffff88000e626a6 : fffffa8004802a00 0000000001834e80
0000000000000000 0000000000000000 : tcpip! ?? ::FNODOBFM::string'+0x56f4<br>fffff800014204a0 fffff88000e6035d : fffffa80066d2e20 fffffa800650fc90 <br>0000000000000000 fffff80001420400 : <br>NETIO!NetioDereferenceNetBufferList+0x86<br>fffff800014204d0 fffff8800183bae6 : fffff80001834e80 0000000000000000 <br>0000000000000000 fffffa8004802a00 : <br>NETIO!NetioDereferenceNetBufferListChain+0x2dd<br>fffff80001420550 fffff88001829f47 : fffffa800650fc90 0000000000000000 <br>fffffa8003fe1b40 fffffa80065a1b80 : <br>tcpip!IppCompleteAndFreePacketList+0xc6<br>fffff80001420580 fffff88001829cc0 : 0000000000000000 fffffa80040cd860 <br>0000000000000001 0000000000000000 : tcpip!IppCleanupMfe+0x77<br>fffff800014205b0 fffff88001859918 : fffffa80040cd860 0000000000000000 <br>fffff80001420658 fffffa8003fe1b40 : tcpip!IppDereferenceMfe+0x20<br>fffff800014205e0 fffff880018597aa : fffff80001420830 0000000000000001 <br>fffffa8003fe1b58 0000000000000000 : tcpip!IppMfeSetTimeOut+0xf8<br>fffff80001420700 fffff8800185861a : 0000000000000000 fffff80001420830 <br>0000000000000001 fffff800016bf1fa : tcpip!IppCompartmentSetTimeout+0x9a<br>fffff80001420770 fffff800016c629e : fffff80001420860 fffff80000000000 <br>0000000040aa0000 0000000000000000 : tcpip!IppTimeout+0x5a<br>fffff800014207a0 fffff800016c5dd6 : fffffa8003d3d3f0 fffffa8003d3d3f0 <br>0000000000000000 0000000000000000 : nt!KiProcessTimerDpcTable+0x66<br>fffff80001420810 fffff800016c64be : 000000ccf14d88de fffff80001420e88 <br>000000000055f58b fffff800018383e8 : nt!KiProcessExpiredTimerList+0xc6<br>fffff80001420e60 fffff800016c5cb7 : fffffa8004823ac4 fffff8000055f58b <br>0000000000000000 000000000000008b : nt!KiTimerExpiration+0x1be<br>fffff80001420f00 fffff800016c0865 : 0000000000000000 fffffa8006852680 <br>0000000000000000 fffff88000ec0c50 : nt!KiRetireDpcList+0x277<br>fffff80001420fb0 fffff800016c067c : 000000000002625a fffff80001615090 <br>0000000000000000 fffff88004466ca0 : nt!KxRetireDpcList+0x5<br>fffff88004466be0 fffff80001704113 : fffff800016b6c60 fffff800016b6ccc <br>00000000001e99f8 fffff800016303c0 : nt!KiDispatchInterruptContinue<br>fffff88004466c10 fffff800016b6ccc : 00000000001e99f8 fffff800016303c0 <br>0000000000000001 fffffa8006616070 : nt!KiDpcInterruptBypass+0x13<br>fffff88004466c20 000007fefddeebed : 0000000000000000 0000000000000000 <br>0000000000000000 0000000000000000 : nt!KiInterruptDispatchNoLock+0x1fc<br>00000000030dd220 0000000000000000 : 0000000000000000 0000000000000000 <br>0000000000000000 0000000000000000 : 0x7fefddeebed

STACK_COMMAND: kb
FOLLOWUP_IP:
NETIO!NetioDereferenceNetBufferList+86
fffff88000e626a6 4885ff test rdi,rdi<br>SYMBOL_STACK_INDEX: 4<br>SYMBOL_NAME: NETIO!NetioDereferenceNetBufferList+86<br>FOLLOWUP_NAME: MachineOwner<br>MODULE_NAME: NETIO<br>IMAGE_NAME: NETIO.SYS<br>DEBUG_FLR_IMAGE_TIMESTAMP: 4a5bc18a<br>FAILURE_BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86<br>BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86<br>Followup: MachineOwner<br>---------<br>kd&gt; .trap 0xfffff800014202c0<br>NOTE: The trap frame does not contain all registers.<br>Some register values may be zeroed or incorrect.<br>rax=fffffa8003d46d40 rbx=fffffa8004680600 rcx=0000000000000000<br>rdx=fffffa8003d46d41 rsi=0000000000000001 rdi=fffff8800185cbb4<br>rip=fffff880018897a0 rsp=fffff80001420450 rbp=0000000000000000<br>r8=fffffa8003d46d40 r9=00000000000000d0 r10=fffff80001835b80<br>r11=fffffa8004680540 r12=0000000000000000 r13=0000000000000000<br>r14=0000000000000000 r15=0000000000000000<br>iopl=0 nv up ei ng nz na pe nc<br>tcpip! ?? ::FNODOBFM::string’+0x56f4:
fffff880`018897a0 488b01 mov rax,qword ptr [rcx]
ds:07ff:0000=???