Unable to free transmit data packets in TDI CLient driver Bugcheck 0x4e

Hi all,
I am transmitting a data packet from my driver to a remote server over TCP. Now I allocate packet data space from PagedPool and Lock the pages in memory using “MmProbeAndLockPages” call. Then i form an mdl from the buffer and send the data attaching the mdl to the IRP. Later on in the IRP completion routine. I try to free the memory of the packet.

Now the code is working perfectly fine (with Verifier enabled) for windows XP however it gives a bugcheck on Windows2003 with the same code.This bugcheck occurs when i try to free the packet buffer. Now as far as i could get from the bugcheck analysis is that someone is still referencing the pages while i am trying to free them. And i see no other suspect than TCP. Following is the bugcheck analysis.

BugCheck 4E, {9a, 18d9, 6, 2}

kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

PFN_LIST_CORRUPT (4e)
Typically caused by drivers passing bad memory descriptor lists (ie: calling
MmUnlockPages twice with the same list, etc). If a kernel debugger is
available get the stack trace.
Arguments:
Arg1: 0000009a,
Arg2: 000018d9
Arg3: 00000006
Arg4: 00000002

Debugging Details:

BUGCHECK_STR: 0x4E_9a

DEFAULT_BUCKET_ID: DRIVER_FAULT

PROCESS_NAME: services.exe

CURRENT_IRQL: 2

LAST_CONTROL_TRANSFER: from 80825729 to 8086d6bc

STACK_TEXT:
f78ae084 80825729 00000003 0000009a 00000000 nt!RtlpBreakWithStatusInstruction
f78ae0d0 8082661d 00000003 c040dbd0 8102b7bc nt!KiBugCheckDebugBreak+0x19
f78ae468 808269b5 0000004e 0000009a 000018d9 nt!KeBugCheck2+0x5d1
f78ae488 8086291b 0000004e 0000009a 000018d9 nt!KeBugCheckEx+0x1b
f78ae4a4 8088c751 8102b7bc 808a8b60 00912ea8 nt!MiBadRefCount+0x33
f78ae4dc 8088d5ad 81b7a000 826a6f68 f78ae5cc nt!MiFreePoolPages+0x5cf
f78ae530 ba247445 646e6573 646e6573 81b7a000 nt!ExFreePoolWithTag+0x277
f78ae544 809ad23e 00000000 826a6f68 826a6f80 TDIClient!NBDSendIoCompletion+0x25 [l:\asyncworkingcode\scsi_nbd\tdiclient\main.cpp @ 2072]
f78ae568 8081d745 00000000 826a6f68 f78ae5cc nt!IovpLocalCompletionRoutine+0xb4
f78ae598 809ad77e 826a6f68 81fe3008 00000000 nt!IopfCompleteRequest+0xcd
f78ae604 babec080 f78ae6d0 82350a30 f78ae6d0 nt!IovCompleteRequest+0x9a
f78ae61c babedecb 826a6f68 00000000 0000101c tcpip!TCPDataRequestComplete+0xa6
f78ae63c babed841 f78ae6d0 f78ae788 820eb80e tcpip!CompleteSends+0x2b
f78ae6f0 babebf9f 822c1170 121b11ac 0b1b11ac tcpip!TCPRcv+0x34c
f78ae750 babeb9e8 00000020 822c1170 babed54d tcpip!DeliverToUser+0x189
f78ae804 babebc66 822c1170 820eb822 00000024 tcpip!IPRcvPacket+0x66c
f78ae844 babebd68 00000000 820c0508 820eb800 tcpip!ARPRcvIndicationNew+0x149
f78ae880 f71311d9 823342b8 00000000 82064000 tcpip!ARPRcvPacket+0x68
f78ae8d4 bad6d79d 822f0130 f78ae8fc 00000001 NDIS!ethFilterDprIndicateReceivePacket+0x318
f78ae8f0 bad6d52c 82064000 820c0508 820c04d0 wlbs!Nic_recv_packet+0x28
f78ae90c f7131550 01064000 820c0508 8224400c wlbs!Prot_packet_recv+0xa1
f78ae960 f7759da4 822a8238 f78ae980 00000002 NDIS!ethFilterDprIndicateReceivePacket+0x1d2
WARNING: Stack unwind information not available. Following frames may be wrong.
f78aef90 f71264da 00000046 ffdffa40 822452cc vmxnet+0x2da4
f78aefa4 80830062 822452cc 822452b8 00000000 NDIS!ndisMDpcX+0x1f
f78aeff4 80888258 ba40f120 00000000 00000000 nt!KiRetireDpcList+0xca
f78aeff8 ba40f120 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x38
80888258 00000000 0000000a bb835b75 00000128 0xba40f120

STACK_COMMAND: kb

FOLLOWUP_IP:
TDIClient!NBDSendIoCompletion+25 [l:\tdiclient\main.cpp @ 2072]
ba247445 33c0 xor eax,eax

FAULTING_SOURCE_CODE:
2068: ExFreePoolWithTag(BufferPointer, ‘dnes’);
2069: return STATUS_SUCCESS;
2070: }
2071:

2072:

SYMBOL_STACK_INDEX: 7

SYMBOL_NAME: TDIClient!NBDSendIoCompletion+25

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: TDIClient

IMAGE_NAME: TDIClient.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 49918d40

FAILURE_BUCKET_ID: 0x4E_9a_VRF_TDIClient!NBDSendIoCompletion+25

BUCKET_ID: 0x4E_9a_VRF_TDIClient!NBDSendIoCompletion+25

Followup: MachineOwner

kd> !pfn 000018d9
PFN 000018D9 at address 8102B7BC
flink 00000000 blink / share count 00000001 pteaddress C040DBD0
reference count 0002 Cached color 0
restore pte 00000000 containing page 002385 Active R
ReadInProgress

kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks…

Resource @ nt!CmpRegistryLock (0x8089e9a0) Shared 1 owning threads
Contention Count = 4
Threads: 81be0020-01<*>
KD: Scanning for held locks…

Loading symbols for f713f000 Ntfs.sys -> Ntfs.sys
Resource @ Ntfs!NtfsData (0xf7162590) Exclusively owned
Threads: 8247a020-01<*>
KD: Scanning for held locks…

Resource @ 0x8238fde0 Shared 1 owning threads
Threads: 81be0020-01<*>
KD: Scanning for held locks…

Resource @ 0x8219eb88 Exclusively owned
Threads: 8247a020-01<*>

Resource @ 0x8219eb20 Exclusively owned
Threads: 8247a020-01<*>

Resource @ 0x821b5838 Exclusively owned
Threads: 8247a020-01<*>

Resource @ 0x821b57d0 Exclusively owned
Threads: 8247a020-01<*>
KD: Scanning for held locks…

Resource @ 0x81b80534 Exclusively owned
Threads: 8247a020-01<*>
KD: Scanning for held locks.

Resource @ 0x8236c5f0 Exclusively owned
Threads: 8247a020-01<*>
KD: Scanning for held locks.

Resource @ 0x81c0afb8 Shared 1 owning threads
Threads: 8247a020-01<*>

Resource @ 0x821efdb0 Shared 1 owning threads
Threads: 8247a020-01<*>

Resource @ 0x822db448 Shared 1 owning threads
Threads: 8247a020-01<*>
KD: Scanning for held locks.
2562 total locks, 12 locks currently held

Now I have following queries

  1. Does TCP still access the packet buffer after i have transmitted it and have received a callback for the IOcompletion routine I registered prior to sending ?(PS I am using TDI interface calls)

  2. I have read that the MDL (formed from the packet buffer ) which i am attaching along with the IRP gets freed by TCP. Also the pages from the PAGED POOL which i had locked in earlier should get unlocked by TCP. Is this correct ? If yes then I am left with only the responsibility of freeing the buffer allocated.

Please do advice on the same.

Thanks in advance,

Imtiaz

[quote]
2. I have read that the MDL (formed from the packet buffer ) which i am
attaching along with the IRP gets freed by TCP. Also the pages from the
PAGED POOL which i had locked in earlier should get unlocked by TCP. Is this
correct ? If yes then I am left with only the responsibility of freeing the
buffer allocated.

[quote]

Where did you read that?

Your driver allocated the memory, the MDL, locked the pages, and allocated
the IRP. When the IRP comes back in your CRTN from TCPIP then your driver
should:

Free the IRP
Unlock the pages.
Free the MDL.
Free memory.

return STATUS_MORE_PROCESSING_REQUIRED;

The idea that the transport should cleanup any of the IRP seems like
nonsense to me. The idea of expecting the IRP to be cleaned up by the I/O
Manager also seems wrong to me (not that you said your were doing that).

And why paged memory? Why not NP memory and skip the lockdown step all
together?

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

> PFN_LIST_CORRUPT (4e)

The usual “MDL issue” error code.

You must MmUnlockPages and then IoFreeMdl before ExFreeing the memory itself.


Maxim S. Shatskih
Windows DDK MVP

xxxxx@storagecraft.com

http://www.storagecraft.com