BSOD in NDIS driver ( logging related?)

We had a BSOD caused by our NDIS driver.
I’d be much obliged if someone could have a more experienced look at the minidump.
I registered already for the OSR debug course in Amherst in April, but in the mean time, looking at the minidump analysis in Windbg is still in novice mode for us.

At first sight, it looks like the crash occurs when logging a string ?
I am aware that this can be the cause when logging ( or rather: string buffer manipulation ) is done at an elevated IRQL.
However, when we do a release build, all debug logging should be disabled.

Here is the minidump analysis:
–START OF MINIDUMP--------------------------------------------
WARNING: Whitespace at end of path element

Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [F:\MEMORY.DMP\010416-6754-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

WARNING: Whitespace at end of path element
Symbol search path is: SRV*d:\localsymbols*http://msdl.microsoft.com/download/symbols;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\AVStream;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Bus;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network
Executable search path is: D:\Personal\Projects\Compositor\software\drivers\DriverInstall\AVStream;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Bus;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network

Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer EmbeddedNT SingleUserTS
Built by: 7601.19045.amd64fre.win7sp1_gdr.151019-1254
Machine Name:
Kernel base = 0xfffff80002e1f000 PsLoadedModuleList = 0xfffff80003066730
Debug session time: Wed Dec 30 01:20:01.914 2015 (UTC + 1:00)
System Uptime: 6 days 23:36:26.630
Loading Kernel Symbols



Loading User Symbols
Loading unloaded module list

*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {fffffa802b600000, 2, 0, fffff88004999200}

Probably caused by : MNA-180_NDIS.sys ( MNA_180_NDIS!memcpy+b0 )

Followup: MachineOwner

0: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: fffffa802b600000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff88004999200, address which referenced memory

Debugging Details:

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800030d0100
fffffa802b600000

CURRENT_IRQL: 2

FAULTING_IP:
MNA_180_NDIS!memcpy+b0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xD1

PROCESS_NAME: java.exe

TRAP_FRAME: fffff8800d80c480 – (.trap 0xfffff8800d80c480)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=da650073da650073 rbx=0000000000000000 rcx=fffff880067c8aa0
rdx=0000020024e3755a rsi=0000000000000000 rdi=0000000000000000
rip=fffff88004999200 rsp=fffff8800d80c618 rbp=0000000019f69206
r8=0000000000001000 r9=000000000000002b r10=da650073da650073
r11=fffff880067c8000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
MNA_180_NDIS!memcpy+0xb0:
fffff88004999200 488b040a mov rax,qword ptr [rdx+rcx] ds:4bf0:fffffa802b5ffffa=???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80002e921e9 to fffff80002e92c40

STACK_TEXT:
fffff8800d80c338 fffff80002e921e9 : 000000000000000a fffffa802b600000 0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff8800d80c340 fffff80002e90e60 : fffffa802b2e955a fffffa800f637570 fffffa800f637570 fffffa802b5ff55a : nt!KiBugCheckDispatch+0x69
fffff8800d80c480 fffff88004999200 : fffff88004995e2a 5ecde45c4cb9fc6d dbe848baa8e5e194 fffffa802b5fe55a : nt!KiPageFault+0x260
fffff8800d80c618 fffff88004995e2a : 5ecde45c4cb9fc6d dbe848baa8e5e194 fffffa802b5fe55a fffff880067c7000 : MNA_180_NDIS!memcpy+0xb0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff8800d80c620 fffff88004994343 : fffffa8000001000 fffffa800f645d00 fffffa8000000001 fffffa8011085b10 : MNA_180_NDIS!HwSubmitTcb+0x162 [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\mphal.c @ 1317]
fffff8800d80c6a0 fffff88004994197 : 0000000000000000 fffffa800f637570 fffffa800f63e040 0000000000000001 : MNA_180_NDIS!ProcessTxNbls+0x17b [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\datapath.c @ 280]
fffff8800d80c700 fffff8800154c4f1 : fffffa800f3a01a0 fffffa8013684d30 0000000000000000 00000000000000a9 : MNA_180_NDIS!MPSendNetBufferLists+0x113 [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\datapath.c @ 200]
fffff8800d80c730 fffff8800148f4b4 : 0000000000000000 fffffa800f654130 0000000000000000 0000000000000000 : ndis!ndisMSendNBLToMiniport+0xb1
fffff8800d80c790 fffff88004046199 : 9610a945cbf5e062 fffffa800f3a01a0 0000000000000000 0000000000000000 : ndis!NdisFSendNetBufferLists+0x64
fffff8800d80c7d0 fffff8800148f3f9 : fffffa800e656938 0000000000000002 fffff80002e9d3b5 fffff88002f6d180 : pacer!PcFilterSendNetBufferLists+0x29
fffff8800d80c8d0 fffff8800154c5d5 : 0000000000000002 0000000000000000 fffffa800f3a01a0 0000000000000000 : ndis!ndisSendNBLToFilter+0x69
fffff8800d80c930 fffff8800165f5ce : 0000000000000000 000000000000000e fffffa800f6c2010 0000000000000000 : ndis!NdisSendNetBufferLists+0x85
fffff8800d80c990 fffff8800165d1c7 : fffff8800176e9a0 0000000000000000 fffffa800f6c0000 fffffa8001760800 : tcpip!IppFragmentPackets+0x39e
fffff8800d80cab0 fffff8800165ebf5 : 0000000000000000 0000000000000000 fffffa800e7bc140 fffffa800e653cec : tcpip!IppDispatchSendPacketHelper+0x87
fffff8800d80cb70 fffff8800165de7e : fffffa800e653c06 fffff8800d80cf00 0000000000000014 fffffa8000000000 : tcpip!IppPacketizeDatagrams+0x2d5
fffff8800d80cc90 fffff8800166079e : 0000000000000000 0000000000004007 0000000072cb9b67 fffffa80112d1ae0 : tcpip!IppSendDatagramsCommon+0x87e
fffff8800d80ce30 fffff88001668aad : fffffa800f658080 0000000000000000 0000000000000000 0000000000004035 : tcpip!IpNlpSendDatagrams+0x3e
fffff8800d80ce70 fffff88001669450 : 0000000000000000 fffff80003027588 0000000000000000 0000000000004800 : tcpip!TcpTcbSend+0x6ad
fffff8800d80d0f0 fffff880016681a8 : 0000000000000000 0000057cde61b655 fffff8800d80d310 fffff8800d80d420 : tcpip!TcpEnqueueTcbSendOlmNotifySendComplete+0xa0
fffff8800d80d120 fffff8800166836b : 00000000001f0000 0000000000000000 0000000000004800 c2646641223b0271 : tcpip!TcpEnqueueTcbSend+0x258
fffff8800d80d1d0 fffff80002e9f188 : 5ecde45c4cb9fc6d dbe848baa8e5e194 9610a945cbf5e062 0ec36a0f8230e3f1 : tcpip!TcpTlConnectionSendCalloutRoutine+0x1b
fffff8800d80d200 fffff8800166922a : fffff88001668350 00000000000042b0 0000000000004000 fffff88004183601 : nt!KeExpandKernelStackAndCalloutEx+0xd8
fffff8800d80d2e0 fffff8800419b9cb : fffffa80108e9c00 fffff8800d80db60 0000000000004035 fffffa801308f040 : tcpip!TcpTlConnectionSend+0x7a
fffff8800d80d350 fffff88004182469 : 0000000000000000 00000000a000000c fffffa8013381d30 fffff80002ea7dec : afd!AfdFastConnectionSend+0x38b
fffff8800d80d510 fffff800031b2690 : 0000000000004035 0000000000000000 0000000000000000 0000000000000000 : afd!AfdFastIoDeviceControl+0x459
fffff8800d80d880 fffff800031b2dc6 : 0000000000000000 0000000000001c0c 0000000000000000 0000000000000000 : nt!IopXxxControlFile+0x520
fffff8800d80da00 fffff80002e91ed3 : 0000000000000000 fffff8000317d6f4 000000000d79c800 0000000017a2c720 : nt!NtDeviceIoControlFile+0x56
fffff8800d80da70 0000000076d1da2a : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiSystemServiceCopyEnd+0x13
0000000017a2ce18 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x76d1da2a

STACK_COMMAND: kb

FOLLOWUP_IP:
MNA_180_NDIS!memcpy+b0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]

FAULTING_SOURCE_CODE:
No source found for ‘d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm’

SYMBOL_STACK_INDEX: 3

SYMBOL_NAME: MNA_180_NDIS!memcpy+b0

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: MNA_180_NDIS

IMAGE_NAME: MNA-180_NDIS.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 566956d5

FAILURE_BUCKET_ID: X64_0xD1_MNA_180_NDIS!memcpy+b0

BUCKET_ID: X64_0xD1_MNA_180_NDIS!memcpy+b0

Followup: MachineOwner

0: kd> lmvm MNA_180_NDIS
start end module name
fffff88004991000 fffff880049ad000 MNA_180_NDIS (private pdb symbols) d:\personal\projects\compositor\software\drivers\driverinstall\network\MNA-180_NDIS.pdb
Loaded symbol image file: MNA-180_NDIS.sys
Mapped memory image file: D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network\MNA-180_NDIS.sys
Image path: MNA-180_NDIS.sys
Image name: MNA-180_NDIS.sys
Timestamp: Thu Dec 10 11:41:25 2015 (566956D5)
CheckSum: 0001B386
ImageSize: 0001C000
File version: 1.0.0.1
Product version: 1.0.0.1
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 1.0 App
File date: 00000000.00000000
Translations: 0009.04b0
CompanyName: BARCO N.V.
ProductName: MNA-180 Network Adapter (NDIS 6.20 Miniport)
InternalName: MNA-180_NDIS.sys
OriginalFilename: MNA-180_NDIS.sys
ProductVersion: 01.00.00.01
FileVersion: 01.00.00.01
FileDescription: MNA-180 NDIS Driver (Build 34105:50923)
LegalCopyright: Copyright (C) 2014 BARCO N.V.

–END OF MINIDUMP -----------------------------------------------------------------------------

This is the code line in mphal.c@1317:
if (LengthInCurrentMdl > PAGE_SIZE - DestinationPageOffset) // Followed by a page jump
{
RtlCopyMemory(PacketDataVa, header + SourceMdlOffset, PAGE_SIZE - DestinationPageOffset);
---->line 1317 LOG(LOG_INFO, “\t=> LOCAL COPY with PAGE JUMP AFTER - Source: %p, Dest= %p, Length: 0x%X”, header + SourceMdlOffset, PacketDataVa, PAGE_SIZE - DestinationPageOffset);
LengthInCurrentMdl = LengthInCurrentMdl - PAGE_SIZE - DestinationPageOffset;
SourceMdlOffset = SourceMdlOffset + PAGE_SIZE - DestinationPageOffset;


This is the LOG function:

---- LOG CODE START -----------------------------
void LOG(int level, char *msg_fmt, …)
{
#if DBG
if ((level <= currentLogLevel) && (KeGetCurrentIrql() == PASSIVE_LEVEL) )
{
va_list arg_list;
va_start( arg_list, msg_fmt );
RtlStringCbVPrintfA(msg_buffer, MSG_LEN, msg_fmt, arg_list);
va_end(arg_list);
KdPrintEx((DPFLTR_DEFAULT_ID, level, “[MNA-180 %s] %s\n”, logPrefix, msg_buffer));
return;
}
#else
UNREFERENCED_PARAMETER (level);
UNREFERENCED_PARAMETER (msg_fmt);

#endif
}
---- LOG CODE END ------------------------------------------

Thanks in advance for any help !

  • Bernard Willaert
    Barco, Belgium
    Healthcare Division
  1. What’s the fail rate for the issue?2. Are you using page pool in memcpy?3. Can you collect a complete dump to analyze futher?

Date: Tue, 5 Jan 2016 05:06:11 -0500
From: xxxxx@barco.com
To: xxxxx@lists.osr.com
Subject: [ntdev] BSOD in NDIS driver ( logging related?)

We had a BSOD caused by our NDIS driver.
I’d be much obliged if someone could have a more experienced look at the minidump.
I registered already for the OSR debug course in Amherst in April, but in the mean time, looking at the minidump analysis in Windbg is still in novice mode for us.

At first sight, it looks like the crash occurs when logging a string ?
I am aware that this can be the cause when logging ( or rather: string buffer manipulation ) is done at an elevated IRQL.
However, when we do a release build, all debug logging should be disabled.

Here is the minidump analysis:
–START OF MINIDUMP--------------------------------------------
WARNING: Whitespace at end of path element

Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [F:\MEMORY.DMP\010416-6754-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

WARNING: Whitespace at end of path element
Symbol search path is: SRV*d:\localsymbols*http://msdl.microsoft.com/download/symbols;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\AVStream;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Bus;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network
Executable search path is: D:\Personal\Projects\Compositor\software\drivers\DriverInstall\AVStream;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Bus;D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network

Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer EmbeddedNT SingleUserTS
Built by: 7601.19045.amd64fre.win7sp1_gdr.151019-1254
Machine Name:
Kernel base = 0xfffff80002e1f000 PsLoadedModuleList = 0xfffff80003066730
Debug session time: Wed Dec 30 01:20:01.914 2015 (UTC + 1:00)
System Uptime: 6 days 23:36:26.630
Loading Kernel Symbols



Loading User Symbols
Loading unloaded module list

*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck D1, {fffffa802b600000, 2, 0, fffff88004999200}

Probably caused by : MNA-180_NDIS.sys ( MNA_180_NDIS!memcpy+b0 )

Followup: MachineOwner

0: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: fffffa802b600000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff88004999200, address which referenced memory

Debugging Details:

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800030d0100
fffffa802b600000

CURRENT_IRQL: 2

FAULTING_IP:
MNA_180_NDIS!memcpy+b0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xD1

PROCESS_NAME: java.exe

TRAP_FRAME: fffff8800d80c480 – (.trap 0xfffff8800d80c480)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=da650073da650073 rbx=0000000000000000 rcx=fffff880067c8aa0
rdx=0000020024e3755a rsi=0000000000000000 rdi=0000000000000000
rip=fffff88004999200 rsp=fffff8800d80c618 rbp=0000000019f69206
r8=0000000000001000 r9=000000000000002b r10=da650073da650073
r11=fffff880067c8000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
MNA_180_NDIS!memcpy+0xb0:
fffff88004999200 488b040a mov rax,qword ptr [rdx+rcx] ds:4bf0:fffffa802b5ffffa=???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff80002e921e9 to fffff80002e92c40

STACK_TEXT:
fffff8800d80c338 fffff80002e921e9 : 000000000000000a fffffa802b600000 0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffff8800d80c340 fffff80002e90e60 : fffffa802b2e955a fffffa800f637570 fffffa800f637570 fffffa802b5ff55a : nt!KiBugCheckDispatch+0x69
fffff8800d80c480 fffff88004999200 : fffff88004995e2a 5ecde45c4cb9fc6d dbe848baa8e5e194 fffffa802b5fe55a : nt!KiPageFault+0x260
fffff8800d80c618 fffff88004995e2a : 5ecde45c4cb9fc6d dbe848baa8e5e194 fffffa802b5fe55a fffff880067c7000 : MNA_180_NDIS!memcpy+0xb0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff8800d80c620 fffff88004994343 : fffffa8000001000 fffffa800f645d00 fffffa8000000001 fffffa8011085b10 : MNA_180_NDIS!HwSubmitTcb+0x162 [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\mphal.c @ 1317]
fffff8800d80c6a0 fffff88004994197 : 0000000000000000 fffffa800f637570 fffffa800f63e040 0000000000000001 : MNA_180_NDIS!ProcessTxNbls+0x17b [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\datapath.c @ 280]
fffff8800d80c700 fffff8800154c4f1 : fffffa800f3a01a0 fffffa8013684d30 0000000000000000 00000000000000a9 : MNA_180_NDIS!MPSendNetBufferLists+0x113 [e:\workspace\mna-1x0-pcie_packagemnc_x64_v1.0\drivers\network\windows\datapath.c @ 200]
fffff8800d80c730 fffff8800148f4b4 : 0000000000000000 fffffa800f654130 0000000000000000 0000000000000000 : ndis!ndisMSendNBLToMiniport+0xb1
fffff8800d80c790 fffff88004046199 : 9610a945cbf5e062 fffffa800f3a01a0 0000000000000000 0000000000000000 : ndis!NdisFSendNetBufferLists+0x64
fffff8800d80c7d0 fffff8800148f3f9 : fffffa800e656938 0000000000000002 fffff80002e9d3b5 fffff88002f6d180 : pacer!PcFilterSendNetBufferLists+0x29
fffff8800d80c8d0 fffff8800154c5d5 : 0000000000000002 0000000000000000 fffffa800f3a01a0 0000000000000000 : ndis!ndisSendNBLToFilter+0x69
fffff8800d80c930 fffff8800165f5ce : 0000000000000000 000000000000000e fffffa800f6c2010 0000000000000000 : ndis!NdisSendNetBufferLists+0x85
fffff8800d80c990 fffff8800165d1c7 : fffff8800176e9a0 0000000000000000 fffffa800f6c0000 fffffa8001760800 : tcpip!IppFragmentPackets+0x39e
fffff8800d80cab0 fffff8800165ebf5 : 0000000000000000 0000000000000000 fffffa800e7bc140 fffffa800e653cec : tcpip!IppDispatchSendPacketHelper+0x87
fffff8800d80cb70 fffff8800165de7e : fffffa800e653c06 fffff8800d80cf00 0000000000000014 fffffa8000000000 : tcpip!IppPacketizeDatagrams+0x2d5
fffff8800d80cc90 fffff8800166079e : 0000000000000000 0000000000004007 0000000072cb9b67 fffffa80112d1ae0 : tcpip!IppSendDatagramsCommon+0x87e
fffff8800d80ce30 fffff88001668aad : fffffa800f658080 0000000000000000 0000000000000000 0000000000004035 : tcpip!IpNlpSendDatagrams+0x3e
fffff8800d80ce70 fffff88001669450 : 0000000000000000 fffff80003027588 0000000000000000 0000000000004800 : tcpip!TcpTcbSend+0x6ad
fffff8800d80d0f0 fffff880016681a8 : 0000000000000000 0000057cde61b655 fffff8800d80d310 fffff8800d80d420 : tcpip!TcpEnqueueTcbSendOlmNotifySendComplete+0xa0
fffff8800d80d120 fffff8800166836b : 00000000001f0000 0000000000000000 0000000000004800 c2646641223b0271 : tcpip!TcpEnqueueTcbSend+0x258
fffff8800d80d1d0 fffff80002e9f188 : 5ecde45c4cb9fc6d dbe848baa8e5e194 9610a945cbf5e062 0ec36a0f8230e3f1 : tcpip!TcpTlConnectionSendCalloutRoutine+0x1b
fffff8800d80d200 fffff8800166922a : fffff88001668350 00000000000042b0 0000000000004000 fffff88004183601 : nt!KeExpandKernelStackAndCalloutEx+0xd8
fffff8800d80d2e0 fffff8800419b9cb : fffffa80108e9c00 fffff8800d80db60 0000000000004035 fffffa801308f040 : tcpip!TcpTlConnectionSend+0x7a
fffff8800d80d350 fffff88004182469 : 0000000000000000 00000000a000000c fffffa8013381d30 fffff80002ea7dec : afd!AfdFastConnectionSend+0x38b
fffff8800d80d510 fffff800031b2690 : 0000000000004035 0000000000000000 0000000000000000 0000000000000000 : afd!AfdFastIoDeviceControl+0x459
fffff8800d80d880 fffff800031b2dc6 : 0000000000000000 0000000000001c0c 0000000000000000 0000000000000000 : nt!IopXxxControlFile+0x520
fffff8800d80da00 fffff80002e91ed3 : 0000000000000000 fffff8000317d6f4 000000000d79c800 0000000017a2c720 : nt!NtDeviceIoControlFile+0x56
fffff8800d80da70 0000000076d1da2a : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiSystemServiceCopyEnd+0x13
0000000017a2ce18 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x76d1da2a

STACK_COMMAND: kb

FOLLOWUP_IP:
MNA_180_NDIS!memcpy+b0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]

FAULTING_SOURCE_CODE:
No source found for ‘d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm’

SYMBOL_STACK_INDEX: 3

SYMBOL_NAME: MNA_180_NDIS!memcpy+b0

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: MNA_180_NDIS

IMAGE_NAME: MNA-180_NDIS.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 566956d5

FAILURE_BUCKET_ID: X64_0xD1_MNA_180_NDIS!memcpy+b0

BUCKET_ID: X64_0xD1_MNA_180_NDIS!memcpy+b0

Followup: MachineOwner

0: kd> lmvm MNA_180_NDIS
start end module name
fffff88004991000 fffff880049ad000 MNA_180_NDIS (private pdb symbols) d:\personal\projects\compositor\software\drivers\driverinstall\network\MNA-180_NDIS.pdb
Loaded symbol image file: MNA-180_NDIS.sys
Mapped memory image file: D:\Personal\Projects\Compositor\software\drivers\DriverInstall\Network\MNA-180_NDIS.sys
Image path: MNA-180_NDIS.sys
Image name: MNA-180_NDIS.sys
Timestamp: Thu Dec 10 11:41:25 2015 (566956D5)
CheckSum: 0001B386
ImageSize: 0001C000
File version: 1.0.0.1
Product version: 1.0.0.1
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 1.0 App
File date: 00000000.00000000
Translations: 0009.04b0
CompanyName: BARCO N.V.
ProductName: MNA-180 Network Adapter (NDIS 6.20 Miniport)
InternalName: MNA-180_NDIS.sys
OriginalFilename: MNA-180_NDIS.sys
ProductVersion: 01.00.00.01
FileVersion: 01.00.00.01
FileDescription: MNA-180 NDIS Driver (Build 34105:50923)
LegalCopyright: Copyright (C) 2014 BARCO N.V.

–END OF MINIDUMP -----------------------------------------------------------------------------

This is the code line in mphal.c@1317:
if (LengthInCurrentMdl > PAGE_SIZE - DestinationPageOffset) // Followed by a page jump
{
RtlCopyMemory(PacketDataVa, header + SourceMdlOffset, PAGE_SIZE - DestinationPageOffset);
---->line 1317 LOG(LOG_INFO, “\t=> LOCAL COPY with PAGE JUMP AFTER - Source: %p, Dest= %p, Length: 0x%X”, header + SourceMdlOffset, PacketDataVa, PAGE_SIZE - DestinationPageOffset);
LengthInCurrentMdl = LengthInCurrentMdl - PAGE_SIZE - DestinationPageOffset;
SourceMdlOffset = SourceMdlOffset + PAGE_SIZE - DestinationPageOffset;


This is the LOG function:

---- LOG CODE START -----------------------------
void LOG(int level, char *msg_fmt, …)
{
#if DBG
if ((level <= currentLogLevel) && (KeGetCurrentIrql() == PASSIVE_LEVEL) )
{
va_list arg_list;
va_start( arg_list, msg_fmt );
RtlStringCbVPrintfA(msg_buffer, MSG_LEN, msg_fmt, arg_list);
va_end(arg_list);
KdPrintEx((DPFLTR_DEFAULT_ID, level, “[MNA-180 %s] %s\n”, logPrefix, msg_buffer));
return;
}
#else
UNREFERENCED_PARAMETER (level);
UNREFERENCED_PARAMETER (msg_fmt);

#endif
}
---- LOG CODE END ------------------------------------------

Thanks in advance for any help !

  • Bernard Willaert
    Barco, Belgium
    Healthcare Division

NTDEV is sponsored by OSR

Visit the list online at: http:
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:>

  1. What’s the fail rate for the issue?
    We saw this only once - as you can see in the dump, the system was up for almost 7 days.

  2. Are you using page pool in memcpy?
    ?? Do you mean the memcpy used for logging ?

  3. Can you collect a complete dump to analyze futher?
    We could set the system to generate a large dump, but as said before, the occurrence is very, very rare.

We still would like to know the root cause of this glitch.
What is particular strange to me is the reference to “java.exe” ?

Debugging Details:

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800030d0100
fffffa802b600000

CURRENT_IRQL: 2

FAULTING_IP:
MNA_180_NDIS!memcpy+b0 [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xD1

===> PROCESS_NAME: java.exe

What type of pointer is the variabile header?
Maybe when you’re adding the SourceMdlOffset value in
RtlCopyMemory(PacketDataVa, *header *+ SourceMdlOffset, PAGE_SIZE -
DestinationPageOffset) you’re advancing the pointer to much.

On 5 January 2016 at 14:46, wrote:

> 1. What’s the fail rate for the issue?
> We saw this only once - as you can see in the dump, the system was up for
> almost 7 days.
>
> 2. Are you using page pool in memcpy?
> ?? Do you mean the memcpy used for logging ?
>
> 3. Can you collect a complete dump to analyze futher?
> We could set the system to generate a large dump, but as said before, the
> occurrence is very, very rare.
>
> We still would like to know the root cause of this glitch.
> What is particular strange to me is the reference to “java.exe” ?
>
>
> Debugging Details:
> ------------------
>
>
> READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800030d0100
> fffffa802b600000
>
> CURRENT_IRQL: 2
>
> FAULTING_IP:
> MNA_180_NDIS!memcpy+b0
> [d:\winmain\minkernel\crts\crtw32\string\amd64\memcpy.asm @ 199]
> fffff880`04999200 488b040a mov rax,qword ptr [rdx+rcx]
>
> CUSTOMER_CRASH_COUNT: 1
>
> DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
>
> BUGCHECK_STR: 0xD1
>
> ===> PROCESS_NAME: java.exe
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

What type of pointer is the variabile header?

PUCHAR header = NULL;

But the faulting instruction points to the source line after this memcpy, in this case the LOG

Well, actually the fault happens in RtlCopyMemory, you can look on the call
stack and see that code from memcpy.asm is executed (which certainly
belongs to RtlCopyMemory).

The reason you see the next line when you look through the functions on the
stack is because of the way function calls work: when a function is called
the return address to the* instruction following the call* is saved on the
stack, hence seeing a pointer to the next piece of code on the stack.

On 5 January 2016 at 15:27, wrote:

> What type of pointer is the variabile header?
>
> PUCHAR header = NULL;
>
> But the faulting instruction points to the source line after this memcpy,
> in this case the LOG
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

On Tue, Jan 5, 2016 at 7:46 AM, wrote:

> What is particular strange to me is the reference to “java.exe” ?

Why is that strange? Your network stack driver is going to executing in
many different process contexts.

Mark Roddy

Thanks everyone for Your answers so far.

So, the context in which the NDIS driver fails is java.exe.
The faulting instruction is in the line before the logging:
RtlCopyMemory(PacketDataVa, header + SourceMdlOffset, PAGE_SIZE - DestinationPageOffset);

We will review this particular piece of code again to see why / how it may fail.
It is obvious that we overlooked a spurious condition that causes this crash.

Thanks again for your time and effort !

  • Bernard Willaert

xxxxx@barco.com wrote:

The faulting instruction is in the line before the logging:
RtlCopyMemory(PacketDataVa, header + SourceMdlOffset, PAGE_SIZE - DestinationPageOffset);

We will review this particular piece of code again to see why / how it may fail.
It is obvious that we overlooked a spurious condition that causes this crash.

The registers really tell the story. Look at the failing address:
fffffa802b600000. When you see a nice, round number like that, you have
to think that you ran off the end of a buffer into empty space. And
look at rcx and rdx, , which hold the source base address and the copy
counter:

rax=da650073da650073 rbx=0000000000000000 rcx=fffff880067c8aa0
rdx=0000020024e3755a rsi=0000000000000000 rdi=0000000000000000

My conclusion would be that the length you provided was spurious, and
you copied many many megabytes before crashing.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Thanks, Tim , for this detailed analysis.

We’ll have a look on how the length could go wrong.

  • Bernard Willaert