WFP Callout bug

Im quite new at driver developmen and im making a small WFP callout driver which intercepts packets on the transport layer. Upon recieving the packets i clone them and place the pointers in a linked list. I reinject the packets from a worker thread at set timer interval. My code seems to have a serous bug that i dont understand. It seems im trying to access memory which im not allowed to access.

Queing Code:
KeAcquireInStackQueuedSpinLock(
&editLock,
&lockHandle
);
tempNode = (NODE*)MmAllocateNonCachedMemory(sizeof(NODE));

tempNode->compartmentId = compartmentId;
tempNode->interfaceIndex = interfaceIndex;
tempNode->subInterfaceIndex = subInterfaceIndex;
tempNode->next = NULL;
tempNode->clonedNetBufferList = NULL;

NdisRetreatNetBufferDataStart(
NET_BUFFER_LIST_FIRST_NB(netBufferList),
ipHeaderSize + transportHeaderSize,
0,
NULL
);

status = FwpsAllocateCloneNetBufferList0(
netBufferList,
NULL,
NULL,
0,
&tempNode->clonedNetBufferList
);

NdisAdvanceNetBufferDataStart(
NET_BUFFER_LIST_FIRST_NB(netBufferList),
ipHeaderSize + transportHeaderSize,
FALSE,
NULL);

if (!NT_SUCCESS(status))
{
KdPrint(("CalloutDriver: Error: Bufferlist Cloning Failed\n"));
goto exit;
}
else
counter++;

if(NodeListHead == NULL)
{
NodeListHead = tempNode;
NodeListTail = NodeListHead;
KdPrint(("CalloutDriver: DriverEntry: Clone netbuffer to head\n"));
}
else
{
NodeListTail->next = tempNode;
NodeListTail = NodeListTail->next;
KdPrint(("CalloutDriver: DriverEntry: Clone netbuffer to tail\n"));
}
KeReleaseInStackQueuedSpinLock(&lockHandle);

Injection Code:

while (1)
{
KeWaitForSingleObject(
&editEvent,
Executive,
KernelMode,
FALSE,
&timeOut
);

while(NodeListHead != NULL)
{
KeAcquireInStackQueuedSpinLock(
&editLock,
&lockHandle
);
status = FwpsInjectTransportReceiveAsync0(
injectionHandle,
NULL,
NULL,
0,
AF_INET,
NodeListHead->compartmentId,
NodeListHead->interfaceIndex,
NodeListHead->subInterfaceIndex,
NodeListHead->clonedNetBufferList,
completionFn,
NULL
);

if (!NT_SUCCESS(status))
{
KdPrint(("CalloutDriver: Error: Packet Injection Failed\n"));
}
else
{
KdPrint(("CalloutDriver: DriverEntry: Injection complete\n"));
}

if(NodeListHead == NodeListTail)
{
FwpsFreeCloneNetBufferList0(NodeListHead->clonedNetBufferList, 0);
MmFreeNonCachedMemory(NodeListHead,sizeof(NODE));
NodeListHead = NULL;
NodeListTail = NULL;
}
else
{
FwpsFreeCloneNetBufferList0(NodeListHead->clonedNetBufferList, 0);
tempNode = NodeListHead;
NodeListHead = NodeListHead->next;
MmFreeNonCachedMemory(tempNode,sizeof(NODE));
}

counter--;

KeReleaseInStackQueuedSpinLock(&lockHandle);
}
if(injectionState == OOB_INJECTION_SHUT_DOWN)
break;
}

*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 50002397, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: 8414efc8, address which referenced memory

Debugging Details:

READ_ADDRESS: 50002397

CURRENT_IRQL: 2

FAULTING_IP:
ndis!NdisFreeCloneNetBufferList+35
8414efc8 f6470620 test byte ptr [edi+6],20h

DEFAULT_BUCKET_ID: VISTA_RC

BUGCHECK_STR: 0xD1

PROCESS_NAME: System

TRAP_FRAME: 89348cb4 -- (.trap ffffffff89348cb4)
ErrCode = 00000000
eax=82f3b388 ebx=00000000 ecx=f9fa0047 edx=f9f90046 esi=82f3b388 edi=50002391
eip=8414efc8 esp=89348d28 ebp=89348d38 iopl=0 nv up ei pl nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010202
ndis!NdisFreeCloneNetBufferList+0x35:
8414efc8 f6470620 test byte ptr [edi+6],20h ds:0023:50002397=??
Resetting default scope

LAST_CONTROL_TRANSFER: from 81cad13f to 81c35688

STACK_TEXT:
89348894 81cad13f 00000003 893433a4 00000000 nt!RtlpBreakWithStatusInstruction
893488e4 81cadbac 00000003 50002397 8414efc8 nt!KiBugCheckDebugBreak+0x1c
89348c94 81c494d4 0000000a 50002397 00000002 nt!KeBugCheck2+0x5f4
89348c94 8414efc8 0000000a 50002397 00000002 nt!KiTrap0E+0x2ac
89348d38 88eb8ed1 82e4c8a8 00000000 00000000 ndis!NdisFreeCloneNetBufferList+0x35
89348d54 883cbd64 badbadfa 00000000 00000000 fwpkclnt!FwpsFreeCloneNetBufferList0+0x7c
89348d7c 81dafbfd 883ce8f0 89343680 00000000 CalloutDriver!FlowInjectionWorker+0x154 [c:\projects\wfpcallout\mycalloutdriver\oob_injection.c @ 171]
89348dc0 81c9a396 883cbc10 883ce8f0 00000000 nt!PspSystemThreadStartup+0x9d
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
fwpkclnt!FwpsFreeCloneNetBufferList0+7c
88eb8ed1 eb06 jmp fwpkclnt!FwpsFreeCloneNetBufferList0+0x84 (88eb8ed9)

SYMBOL_STACK_INDEX: 5

SYMBOL_NAME: fwpkclnt!FwpsFreeCloneNetBufferList0+7c

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: fwpkclnt

IMAGE_NAME: fwpkclnt.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 4549b2f6

FAILURE_BUCKET_ID: 0xD1_fwpkclnt!FwpsFreeCloneNetBufferList0+7c

BUCKET_ID: 0xD1_fwpkclnt!FwpsFreeCloneNetBufferList0+7c

Followup: MachineOwner

Hope Someone can help me out.

I think i might know why… I need to Free the cloned netbuffer from completionFn.

Jesper Kragh

Yup. That did the trick.

Sorry for the fast post…

Jesper

A quick look at your code shows you are calling MmAllocateNonCachedMemory at
DISPATCH_LEVEL. Although this allocates frlom the non paged pool, the doc
says it is not allowed. You must make sure all memory you touch is non paged
while holding the spin lock. Put on driver verifier to catch errors like
this in an early stage. BTW is it necessary you use MmAllocate… instead of
the the ndis or pool allocation routines ?

/Daniel

wrote in message news:xxxxx@ntdev…
> Im quite new at driver developmen and im making a small WFP callout driver
> which intercepts packets on the transport layer. Upon recieving the
> packets i clone them and place the pointers in a linked list. I reinject
> the packets from a worker thread at set timer interval. My code seems to
> have a serous bug that i dont understand. It seems im trying to access
> memory which im not allowed to access.
>
> Queing Code:
> KeAcquireInStackQueuedSpinLock(
> &editLock,
> &lockHandle
> );
> tempNode = (NODE*)MmAllocateNonCachedMemory(sizeof(NODE));
>
> tempNode->compartmentId = compartmentId;
> tempNode->interfaceIndex = interfaceIndex;
> tempNode->subInterfaceIndex = subInterfaceIndex;
> tempNode->next = NULL;
> tempNode->clonedNetBufferList = NULL;
>
> NdisRetreatNetBufferDataStart(
> NET_BUFFER_LIST_FIRST_NB(netBufferList),
> ipHeaderSize + transportHeaderSize,
> 0,
> NULL
> );
>
> status = FwpsAllocateCloneNetBufferList0(
> netBufferList,
> NULL,
> NULL,
> 0,
> &tempNode->clonedNetBufferList
> );
>
> NdisAdvanceNetBufferDataStart(
> NET_BUFFER_LIST_FIRST_NB(netBufferList),
> ipHeaderSize + transportHeaderSize,
> FALSE,
> NULL);
>
> if (!NT_SUCCESS(status))
> {
> KdPrint((“CalloutDriver: Error: Bufferlist Cloning Failed\n”));
> goto exit;
> }
> else
> counter++;
>
> if(NodeListHead == NULL)
> {
> NodeListHead = tempNode;
> NodeListTail = NodeListHead;
> KdPrint((“CalloutDriver: DriverEntry: Clone netbuffer to head\n”));
> }
> else
> {
> NodeListTail->next = tempNode;
> NodeListTail = NodeListTail->next;
> KdPrint((“CalloutDriver: DriverEntry: Clone netbuffer to tail\n”));
> }
> KeReleaseInStackQueuedSpinLock(&lockHandle);
>
>
>
>
> Injection Code:
>
> while (1)
> {
> KeWaitForSingleObject(
> &editEvent,
> Executive,
> KernelMode,
> FALSE,
> &timeOut
> );
>
> while(NodeListHead != NULL)
> {
> KeAcquireInStackQueuedSpinLock(
> &editLock,
> &lockHandle
> );
> status = FwpsInjectTransportReceiveAsync0(
> injectionHandle,
> NULL,
> NULL,
> 0,
> AF_INET,
> NodeListHead->compartmentId,
> NodeListHead->interfaceIndex,
> NodeListHead->subInterfaceIndex,
> NodeListHead->clonedNetBufferList,
> completionFn,
> NULL
> );
>
> if (!NT_SUCCESS(status))
> {
> KdPrint((“CalloutDriver: Error: Packet Injection Failed\n”));
> }
> else
> {
> KdPrint((“CalloutDriver: DriverEntry: Injection complete\n”));
> }
>
> if(NodeListHead == NodeListTail)
> {
> FwpsFreeCloneNetBufferList0(NodeListHead->clonedNetBufferList, 0);
> MmFreeNonCachedMemory(NodeListHead,sizeof(NODE));
> NodeListHead = NULL;
> NodeListTail = NULL;
> }
> else
> {
> FwpsFreeCloneNetBufferList0(NodeListHead->clonedNetBufferList, 0);
> tempNode = NodeListHead;
> NodeListHead = NodeListHead->next;
> MmFreeNonCachedMemory(tempNode,sizeof(NODE));
> }
>
> counter–;
>
> KeReleaseInStackQueuedSpinLock(&lockHandle);
> }
> if(injectionState == OOB_INJECTION_SHUT_DOWN)
> break;
> }
>
> ***
> *
>
> * Bugcheck Analysis
>
> *
>
>

>
> DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
> An attempt was made to access a pageable (or completely invalid) address
> at an
> interrupt request level (IRQL) that is too high. This is usually
> caused by drivers using improper addresses.
> If kernel debugger is available get stack backtrace.
> Arguments:
> Arg1: 50002397, memory referenced
> Arg2: 00000002, IRQL
> Arg3: 00000000, value 0 = read operation, 1 = write operation
> Arg4: 8414efc8, address which referenced memory
>
> Debugging Details:
> ------------------
>
>
> READ_ADDRESS: 50002397
>
> CURRENT_IRQL: 2
>
> FAULTING_IP:
> ndis!NdisFreeCloneNetBufferList+35
> 8414efc8 f6470620 test byte ptr [edi+6],20h
>
> DEFAULT_BUCKET_ID: VISTA_RC
>
> BUGCHECK_STR: 0xD1
>
> PROCESS_NAME: System
>
> TRAP_FRAME: 89348cb4 – (.trap ffffffff89348cb4)
> ErrCode = 00000000
> eax=82f3b388 ebx=00000000 ecx=f9fa0047 edx=f9f90046 esi=82f3b388
> edi=50002391
> eip=8414efc8 esp=89348d28 ebp=89348d38 iopl=0 nv up ei pl nz na po
> nc
> cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
> efl=00010202
> ndis!NdisFreeCloneNetBufferList+0x35:
> 8414efc8 f6470620 test byte ptr [edi+6],20h
> ds:0023:50002397=??
> Resetting default scope
>
> LAST_CONTROL_TRANSFER: from 81cad13f to 81c35688
>
> STACK_TEXT:
> 89348894 81cad13f 00000003 893433a4 00000000
> nt!RtlpBreakWithStatusInstruction
> 893488e4 81cadbac 00000003 50002397 8414efc8 nt!KiBugCheckDebugBreak+0x1c
> 89348c94 81c494d4 0000000a 50002397 00000002 nt!KeBugCheck2+0x5f4
> 89348c94 8414efc8 0000000a 50002397 00000002 nt!KiTrap0E+0x2ac
> 89348d38 88eb8ed1 82e4c8a8 00000000 00000000
> ndis!NdisFreeCloneNetBufferList+0x35
> 89348d54 883cbd64 badbadfa 00000000 00000000
> fwpkclnt!FwpsFreeCloneNetBufferList0+0x7c
> 89348d7c 81dafbfd 883ce8f0 89343680 00000000
> CalloutDriver!FlowInjectionWorker+0x154
> [c:\projects\wfpcallout\mycalloutdriver\oob_injection.c @ 171]
> 89348dc0 81c9a396 883cbc10 883ce8f0 00000000
> nt!PspSystemThreadStartup+0x9d
> 00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16
>
>
> STACK_COMMAND: kb
>
> FOLLOWUP_IP:
> fwpkclnt!FwpsFreeCloneNetBufferList0+7c
> 88eb8ed1 eb06 jmp fwpkclnt!FwpsFreeCloneNetBufferList0+0x84
> (88eb8ed9)
>
> SYMBOL_STACK_INDEX: 5
>
> SYMBOL_NAME: fwpkclnt!FwpsFreeCloneNetBufferList0+7c
>
> FOLLOWUP_NAME: MachineOwner
>
> MODULE_NAME: fwpkclnt
>
> IMAGE_NAME: fwpkclnt.sys
>
> DEBUG_FLR_IMAGE_TIMESTAMP: 4549b2f6
>
> FAILURE_BUCKET_ID: 0xD1_fwpkclnt!FwpsFreeCloneNetBufferList0+7c
>
> BUCKET_ID: 0xD1_fwpkclnt!FwpsFreeCloneNetBufferList0+7c
>
> Followup: MachineOwner
>
>
> Hope Someone can help me out.
>