NETIO.sys - Null pointer dereference and crash in NETIO!FeApplyModifiedLayerData

Hello OSR Community, I have been observing crashes on few laptops, but not all, when running a WFP driver I am developing. I am attaching a PDF document that explains the purpose of the driver, an overview of the sequence of API calls made, my analysis of the memory dump file and also kernel debug excerpts of the same code path from a machine where no crash is seen. You will find the following attachments

A crash report document containing details of the crash dump analysis I did on the kernel dump file and comparison with another instance where the driver never crashes, windows system information etc

I have the output of netsh wfp show filters, netsh wfp show state and list of all active drivers on the machine where the crash was seen

I hope someone from the community can take a look at the report and the logs, and advice me on where the fault is and how it can be addressed. If needed I can upload the kernel dump file which is about 1.3 GB

Thank you

Regards,
WThomas

[Edited by Mods: Attached PDF removed – I’m not even sure how/why attachments are allowed]

Yeah? Just FYI: I am not opening a PDF file provided by somebody in their first post to this community (or, probably, ever). You know that there are plenty of PDF exploits, right?

There are, perhaps, other members who think differently. So, maybe you’ll get some help. However, I would respectfully suggest that the way to maximize the chance of getting help here will be to do what everyone else on this forum does and post the !analyze -v output to the forum, along with whatever question you have.

Have you seen anyone, ever, post a PDF file here?

ETA: I removed the PDF from the OP.

Thank you Peter

Win7_10_64bit_TestDriver is a driver I am working on which sets a callout function at FWPM_LAYER_ALE_BIND_REDIRECT_V4 and calls on FwpsApplyModifiedLayerData0 after pending a packet for classification. In the “packet pending” state the driver prints the name of the application that initiates a TCP or UDP session via a workitem. FwpsApplyModifiedLayerData0 is called in the workitem. This is a stripped down version of what I actually do and I know there are better places to get the application name. But for now please assume I have to do certain things at FWPM_LAYER_ALE_BIND_REDIRECT_V4 and I need the application name there.
The driver causes a crash on certain machines. Here is the “analyze -v” on the kernel memory dump taken from the machine where the crash was seen. This is not seen on all machines, but just a few. The “analyze -v” output shows the cause of the crash is a null pointer dereference in NETIO.sys. I have tried using the WFP API’s according to the MS documentation. I would like to know if there is a problem in the way I am using the WFP APi’s or as per the output below, this is an issue in NETIO.sys. I validate the handle and buffer passed into FwpsApplyModifiedLayerData0. As for the 3rd parameter I experimented with both FWPS_CLASSIFY_FLAG_REAUTHORIZE_IF_MODIFIED_BY_OTHERS and setting it to 0.

1: kd> !analyze -v


  •                                                                         *
    
  •                    Bugcheck Analysis                                    *
    
  •                                                                         *
    

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck. Usually the exception address pinpoints
the driver/function that caused the problem. Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: ffffffffc0000005, The exception code that was not handled
Arg2: fffff80487d9e6ec, The address that the exception occurred at
Arg3: fffff90d1eac7708, Exception Record Address
Arg4: fffff90d1eac6f40, Context Record Address

Debugging Details:

KEY_VALUES_STRING: 1

Key  : AV.Dereference
Value: NullClassPtr

Key  : AV.Fault
Value: Read

Key  : Analysis.CPU.mSec
Value: 2140

Key  : Analysis.DebugAnalysisManager
Value: Create

Key  : Analysis.Elapsed.mSec
Value: 6153

Key  : Analysis.Init.CPU.mSec
Value: 11327

Key  : Analysis.Init.Elapsed.mSec
Value: 345006

Key  : Analysis.Memory.CommitPeak.Mb
Value: 91

Key  : WER.OS.Branch
Value: vb_release

Key  : WER.OS.Timestamp
Value: 2019-12-06T14:06:00Z

Key  : WER.OS.Version
Value: 10.0.19041.1

BUGCHECK_CODE: 7e

BUGCHECK_P1: ffffffffc0000005

BUGCHECK_P2: fffff80487d9e6ec

BUGCHECK_P3: fffff90d1eac7708

BUGCHECK_P4: fffff90d1eac6f40

EXCEPTION_RECORD: fffff90d1eac7708 – (.exr 0xfffff90d1eac7708)
ExceptionAddress: fffff80487d9e6ec (NETIO!FeApplyModifiedLayerData+0x000000000000002c)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 0000000000000038
Attempt to read from address 0000000000000038

CONTEXT: fffff90d1eac6f40 – (.cxr 0xfffff90d1eac6f40)
rax=0000000000000000 rbx=0000000000000000 rcx=ffffd681c17a8180
rdx=0000000000000000 rsi=0000000000000000 rdi=0000432f00003319
rip=fffff80487d9e6ec rsp=fffff90d1eac7940 rbp=ffffc409885035a0
r8=ffffc40964cd5340 r9=4121890e4b3f792b r10=fffff804832bf8e0
r11=fffff80487dca4f8 r12=0000000000000300 r13=0000000000000000
r14=0000000000000000 r15=fffff80483d24440
iopl=0 nv up ei ng nz na pe nc
cs=0010 ss=0000 ds=002b es=002b fs=0053 gs=002b efl=00050282
NETIO!FeApplyModifiedLayerData+0x2c:
fffff80487d9e6ec 80783800 cmp byte ptr [rax+38h],0 ds:002b:0000000000000038=??
Resetting default scope

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXNTFS: 1 (!blackboxntfs)

BLACKBOXPNP: 1 (!blackboxpnp)

BLACKBOXWINLOGON: 1

PROCESS_NAME: System

READ_ADDRESS: 0000000000000038

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE_STR: c0000005

EXCEPTION_PARAMETER1: 0000000000000000

EXCEPTION_PARAMETER2: 0000000000000038

EXCEPTION_STR: 0xc0000005

STACK_TEXT:
fffff90d1eac7940 fffff8048813793a : ffffc409866db060 ffffc40984c56510 ffffc4097fb9d080 0100000000100000 : NETIO!FeApplyModifiedLayerData+0x2c
fffff90d1eac7970 fffff8047f101c78 : fffff8047f103180 ffffc4097f2ef020 ffffc4097dd49710 0000000000000000 : fwpkclnt!FwpsApplyModifiedLayerData0+0x1a
fffff90d1eac79a0 fffff80483274835 : ffffc40984c56510 ffffc40981834920 ffffc40984c56510 ffffc4097a584080 : Win7_10_64bit_TestDriver+0x1c78
fffff90d1eac7a00 fffff804832b86d5 : ffffc40980d5a040 ffffc40980d5a040 fffff80483274700 ffffc40900000000 : nt!IopProcessWorkItem+0x135
fffff90d1eac7a70 fffff80483355a15 : ffffc40980d5a040 0000000000000080 ffffc409640a3180 0000000000000001 : nt!ExpWorkerThread+0x105
fffff90d1eac7b10 fffff804833fec78 : ffffd681c17a8180 ffffc40980d5a040 fffff804833559c0 0000000000000006 : nt!PspSystemThreadStartup+0x55
fffff90d1eac7b60 0000000000000000 : fffff90d1eac8000 fffff90d1eac1000 0000000000000000 0000000000000000 : nt!KiStartSystemThread+0x28

SYMBOL_NAME: NETIO!FeApplyModifiedLayerData+2c

MODULE_NAME: NETIO

IMAGE_NAME: NETIO.SYS

STACK_COMMAND: .cxr 0xfffff90d1eac6f40 ; kb

BUCKET_ID_FUNC_OFFSET: 2c

FAILURE_BUCKET_ID: AV_NETIO!FeApplyModifiedLayerData

OS_VERSION: 10.0.19041.1

BUILDLAB_STR: vb_release

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

FAILURE_ID_HASH: {30594c57-1544-e0e5-206d-d31719fbe7f2}

Followup: MachineOwner

Analyze is pointing you right at the line of code in your driver that is indicted in the crash:

fffff90d1eac79a0 fffff80483274835 : ffffc40984c56510 ffffc40981834920 ffffc40984c56510 ffffc4097a584080 : Win7_10_64bit_TestDriver+0x1c78

but you don’t have your driver symbols loaded, which makes debug analysis difficult.

if (pktCtx->layerData != NULL && pktCtx->classifyHandle != 0)
FwpsApplyModifiedLayerData(pktCtx->classifyHandle, pktCtx->layerData, 0);

This is the location which you see in the stack trace.
I ensure the passed in parameters are valid. Why is the crash in NETIO.sys. Why not fwpkclnt. The code details are in the attached PDF. On a different laptop the same code path works fine for days without any issues. I have been advised about PDFs. So if you could take look I can post the complete code snippet in the report over here. The txt files have the active WFP filters, running driver etc

Thank you Mark

@WThomas… The PDF has been removed from your post.

@“Peter_Viscarola_(OSR)” before my first post I read the following link.

https://community.osr.com/discussion/290621/community-rules-and-guidelines-please-read-before-posting

Though, like you pointed out, I have not seen any PDF postings when I looked for answers in the past, I never saw it as a “dont post PDF” indication. I posted the output of the debug logs and left the PDF there because I hoped someone else would read it and help. Are you afraid of just PDF based exploits? Because you should know that any file format thats parsed by a software could trigger a bug in that software that renders the file format, **right? **
PNG, BMP, DOC, XLS, VS solution files and even TXT files. Include it in the link about posting.

If that’s the line of code, then “pktCtx” is NULL. There is no alternative.

Hi Tim, if pktCtx is NULL then wouldnt the access violation happen in the if statement? There are two dereferencing that I do in the “if” statement. Further I only posted a small snippet of the code. Early on in the WorkItem I access fields in this structure and that hows I pull out the application name. So if pktCtx was NULL I would have seen a crash early on. I can confirm pktCtx is not NULL and the call to FwpsApplyModifiedLayerData happens as you see in the call stack.

Thank you Tim

@WThomas … Railing at me for removing your PDF is not a good way for you to start in this community. When you post here, you’re a guest in my house. I suggest that you conduct yourself appropriately.

Since you seem to want some sort of explication: we had disabled the ability to upload files to the forums quite a long time ago. It seems that a software revision changed this setting, so we were surprised that you could upload a PDF or anything else. That setting has now been restored (I think).

So, stop arguing with your host, please.

And now to your crash:

The code is attempting to reference address 0x38… that sure looks like an offset, with a null pointer, to me. Set a breakpoint in the debugger and walk through your driver to the point where you call into FwpsApplyModifiedLayerData0. I bet you find the zero pointer rather quickly.

@“Peter_Viscarola_(OSR)” I am sorry for being impolite
I am copying the contents of the original PDF below which has four sections. It has the driver code excerpts, the crash analysis, comparison on a laptop where things work fine (setting break points like Peter suggested) and system information where the crash was seen.

Crash In NetIO.sys When Using A WFP Based Driver

Section 1: Driver Information
The purpose of this driver is to identify at the kernel layer user space applications that initiate a session. In certain installations we observed crashes pointing to NETIO.sys. To get to the root of the problem a stripped-down driver was created which does not have all the functionality of the original driver. In this driver sample the callout is added to the layer FWPM_LAYER_ALE_CONNECT_REDIRECT_V4. The API call sequence is roughly as follows.

InitializationCode() {
NtStatus = FwpmTransactionBegin0(hFilterEngine, 0);
NtStatus = FwpmSubLayerAdd(hFilterEngine, &fwpmSublayer, NULL);
NtStatus = FwpsCalloutRegister(DeviceObject, &fwpsCallout, calloutID);
NtStatus = FwpmCalloutAdd0(hFilterEngine, &fwpmCallout, NULL, calloutID);
condition[0].fieldKey = FWPM_CONDITION_IP_LOCAL_PORT;
condition[0].matchType = FWP_MATCH_LESS_OR_EQUAL;
condition[0].conditionValue.type = FWP_UINT16;
condition[0].conditionValue.uint16 = 65535;
fwpmFilter.action.type = FWP_ACTION_CALLOUT_UNKNOWN;
fwpmFilter.weight.uint32 = 50000;
fwpmFilter.numFilterConditions = 1;
fwpmFilter.filterCondition = condition;fwpmFilter.action.calloutKey = calloutGUID;
NtStatus = FwpmFilterAdd0(hFilterEngine, &fwpmFilter, NULL, filterID);
NtStatus = FwpmTransactionCommit0(hFilterEngine);
NtStatus = FwpsInjectionHandleCreate(AF_INET,
FWPS_INJECTION_TYPE_TRANSPORT | FWPS_INJECTION_TYPE_FORWARD |
FWPS_INJECTION_TYPE_NETWORK, &hInjectionHandle);
}
void ClassifyFn (


{
NtStatus = FwpsAcquireClassifyHandle((void*)classifyContext, 0, &(pktCtx-

classifyHandle));
NtStatus = FwpsAcquireWritableLayerDataPointer(pktCtx->classifyHandle, filter-
filterId, 0, &writableLayerData, classifyOut);
NtStatus = FwpsPendClassify(pktCtx->classifyHandle, filter->filterId, 0, pktCtx-
classifyOut);
//Allocate a WorkItem to get the application name.
IoQueueWorkItem(workItem, GetApplicationName, DelayedWorkQueue,
(void*)pktCtx);classifyOut->actionType = FWP_ACTION_BLOCK;
classifyOut->rights &= ~FWPS_RIGHT_ACTION_WRITE;
classifyOut->flags |= FWPS_CLASSIFY_OUT_FLAG_ABSORB;
}

void
GetApplicationName(
In PDEVICE_OBJECT DevObject,
In PVOID ctx
)
//After all processing the code goes to exit where the pend operation is complete and the packet is let
//through. The crash is seen when invoking FwpsApplyModifiedLayerData.
Exit:
if (ctx != NULL) {
if (pktCtx->workItem != NULL) {
IoFreeWorkItem(pktCtx->workItem);
}
pktCtx->classifyOut->actionType = FWP_ACTION_PERMIT;
pktCtx->classifyOut->rights &= ~FWPS_RIGHT_ACTION_WRITE;
/* I also experimented with pktCtx->classifyOut->rights |=
FWPS_RIGHT_ACTION_WRITE; */
if (pktCtx->layerData != NULL && pktCtx->classifyHandle != 0)
FwpsApplyModifiedLayerData(pktCtx->classifyHandle, pktCtx->layerData, 0);
if (pktCtx->classifyHandle != 0) {
FwpsCompleteClassify(pktCtx->classifyHandle, 0, pktCtx->classifyOut);
FwpsReleaseClassifyHandle(pktCtx->classifyHandle);
}
if (pktCtx->classifyOut)ExFreePool(pktCtx->classifyOut);
ExFreePoolWithTag(ctx, PKT_TAG);
}
}

Section 2: Crash Analysis On Memory Dump File
1: kd> !analyze -v
SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)

Arg1: ffffffffc0000005, The exception code that was not handled => Access violation code c0000005
raised when invalid memory is access
Arg2: fffff80487d9e6ec, The address that the exception occurred at => This address is the code
location which has the bug
Arg3: fffff90d1eac7708, Exception Record Address
Arg4: fffff90d1eac6f40, Context Record Address
1: kd> u fffff80487d9e6ec => Disassembly of code location where exception is seen
NETIO!FeApplyModifiedLayerData+0x2c:fffff80487d9e6ec 80783800 cmp byte ptr [rax+38h],0 ** REDACTED ** 1: kd> dd rax fffff90d1eac6f40 00000000 00000000 00000000 00000000

BUGCHECK_CODE: 7e
BUGCHECK_P1: ffffffffc0000005
BUGCHECK_P2: fffff80487d9e6ec
BUGCHECK_P3: fffff90d1eac7708
BUGCHECK_P4: fffff90d1eac6f40
EXCEPTION_RECORD: fffff90d1eac7708 – (.exr 0xfffff90d1eac7708)
ExceptionAddress: fffff80487d9e6ec (NETIO!FeApplyModifiedLayerData+0x000000000000002c)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 0000000000000038
Attempt to read from address 0000000000000038
CONTEXT: fffff90d1eac6f40 – (.cxr 0xfffff90d1eac6f40)
rax=0000000000000000 rbx=0000000000000000 rcx=ffffd681c17a8180rdx=0000000000000000 rsi=0000000000000000 rdi=0000432f00003319
rip=fffff80487d9e6ec rsp=fffff90d1eac7940 rbp=ffffc409885035a0
r8=ffffc40964cd5340 r9=4121890e4b3f792b r10=fffff804832bf8e0
r11=fffff80487dca4f8 r12=0000000000000300 r13=0000000000000000
r14=0000000000000000 r15=fffff80483d24440
iopl=0
nv up ei ng nz na pe nc
cs=0010 ss=0000 ds=002b es=002b fs=0053 gs=002b
efl=00050282
NETIO!FeApplyModifiedLayerData+0x2c:
fffff80487d9e6ec 80783800 cmp byte ptr [rax+38h],0 ds:002b:0000000000000038=??
Resetting default scope
BLACKBOXBSD: 1 (!blackboxbsd)
BLACKBOXNTFS: 1 (!blackboxntfs)
BLACKBOXPNP: 1 (!blackboxpnp)
BLACKBOXWINLOGON: 1
PROCESS_NAME: System
READ_ADDRESS: 0000000000000038
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The
memory could not be %s.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: 0000000000000038
EXCEPTION_STR: 0xc0000005
STACK_TEXT:
fffff90d1eac7940 fffff8048813793a : ffffc409866db060 ffffc40984c56510 ffffc4097fb9d080 0100000000100000 : NETIO!FeApplyModifiedLayerData+0x2c
fffff90d1eac7970 fffff8047f101c78 : fffff8047f103180 ffffc4097f2ef020 ffffc4097dd49710 0000000000000000 : fwpkclnt!FwpsApplyModifiedLayerData0+0x1afffff90d1eac79a0 fffff80483274835 : ffffc40984c56510 ffffc40981834920 ffffc40984c56510 ffffc4097a584080 : TestDriver!GetApplicationInformation+0x1d8
[C:\Users\WinnyThomas\Desktop\TestDriver\TestDriver\TestDriver.cpp @ 176]
fffff90d1eac7a00 fffff804832b86d5 : ffffc40980d5a040 ffffc40980d5a040 fffff80483274700 ffffc40900000000 : nt!IopProcessWorkItem+0x135
fffff90d1eac7a70 fffff80483355a15 : ffffc40980d5a040 0000000000000080 ffffc409640a3180 0000000000000001 : nt!ExpWorkerThread+0x105
fffff90d1eac7b10 fffff804833fec78 : ffffd681c17a8180 ffffc40980d5a040 fffff804833559c0 0000000000000006 : nt!PspSystemThreadStartup+0x55
fffff90d1eac7b60 0000000000000000 : fffff90d1eac8000 fffff90d1eac1000 0000000000000000 0000000000000000 : nt!KiStartSystemThread+0x28
SYMBOL_NAME: NETIO!FeApplyModifiedLayerData+2c
MODULE_NAME: NETIO
IMAGE_NAME: NETIO.SYS
STACK_COMMAND: .cxr 0xfffff90d1eac6f40 ; kb
BUCKET_ID_FUNC_OFFSET: 2c
FAILURE_BUCKET_ID: AV_NETIO!FeApplyModifiedLayerData
OS_VERSION: 10.0.19041.1
BUILDLAB_STR: vb_release
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
FAILURE_ID_HASH: {30594c57-1544-e0e5-206d-d31719fbe7f2}
Followup:
MachineOwner

Inside TestDriver!GetApplicationName at the offset shown (0x1d8) I call the fwpkclnt API FwpsApplyModifiedLayerData. This API is called when I hold onto a packet for a while to make changes and a policy decision. The parameters to this function are an injection handle, a buffer and a flag. Early in the driver initialization I verify the handle is valid, if not the driver wont start because this handle is crucial. Next, I do verify I have valid buffer data to work on.If not, which can happen sometimes, the driver never enters the code path which calls FwpsApplyModifiedLayerData and finally the flag is set to the recommended FWPS_CLASSIFY_FLAG_REAUTHORIZE_IF_MODIFIED_BY_OTHERS. So, the parameters I pass in when I invoke the API are valid. When the call passes on from my driver to fwpkclnt and then to NETIO.sys, a null pointer deference is seen in NETIO.sys

Section 3: Analysis Of Same Code Path On A Separate Laptop Where No Crash Is Seen
Comparison with a laptop where I am running the same driver and recent windows update
0: kd> kb
00 fffff808dee15544 : 0000000000000001 0000000000000554 0000000000000000
0000000000000554 : NETIO!FeApplyModifiedLayerData+0x2c => this is the location where a pointer dereference happens, same as in your case, but in my case, there is a valid pointer in rax as you see below 01 fffff80038c92591 : ffffb48869a04010 fffff80038c92000 ffffb48800000213 ffffffff434f5250 : fwpkclnt!FwpsApplyModifiedLayerData0+0x14 => this offset location is
different because I am using an updated windows version 20H2
02 fffff800396fe14b : ffffb488671999b0 00000000000002a0 0000000000000554
ffffb4886244a3d0 : TestDriver!GetApplicationInformation +0x1d8 => You see the same offset inside my driver calling fwpkclnt. 03 fffff8003963ee05 : ffffb4886244a3d0 ffffb4886308a700 fffff800396fe050 000000000000000c : nt!IopProcessWorkItem+0xfb
04 fffff8003962af87 : 0000000000000000 0000000000000080 ffffb488624cd040
ffffb4886308a700 : nt!ExpWorkerThread+0xf5 05 fffff8003976b676 : ffffe101a9c20180 ffffb4886308a700 fffff8003962af40 0000000000000246 : nt!PspSystemThreadStartup+0x47
06 0000000000000000 : fffff50f9c540000 fffff50f9c539000 0000000000000000
0000000000000000 : nt!KiStartSystemThread+0x16 0: kd> r raxrax=ffffb488693ea9f0 0: kd> db rax+38h => unlike in the memory dump where the memory evaluates to 0x00000000 + 38h in my case it evaluates to ffffb488693ea9f0 + 38h. Which is a valid memory ffffb488693eaa28 01 00 01 00 00 00 00 00-00 00 00 00 00 00 00 00 … => The
contents of the dereference memory is highlighted in green. Only one byte is used in the
comparison operation in NETIO.SYS
ffffb488`693eaa38 00 00 00 00 00 00 00 00-af 05 01 00 00 00 00 00 …

Section 4: System Information Where The Crash Is Seen
NETIO.sys Version: 10.0.19041.1566
Edition Windows 10 Enterprise
Version 21H1

Are you assigning writeableLayerData to pktCtx->layerData somewhere?

Your code has a possible race condition here:
`//Allocate a WorkItem to get the application name.
IoQueueWorkItem(workItem, GetApplicationName, DelayedWorkQueue,
(void*)pktCtx);

// shouldn't these be done before queueing the work item?
classifyOut->actionType = FWP_ACTION_BLOCK;
classifyOut->rights &= ~FWPS_RIGHT_ACTION_WRITE;
classifyOut->flags |= FWPS_CLASSIFY_OUT_FLAG_ABSORB;

}`

Also you call a bunch of fwps functions that return status values but your code segment doesn’t show any error checking. I’ll assume you do check the return values.

I will move the block of code you highlighted, so its executed before the call to the workitem. But just curious, I see only two possible outcomes here

  1. The workitem get invoked first before the code you highlighted. Anyway in the workitem I set the action to FWP_ACTION_BLOCK and permit it, and later the “FWPS_CLASSIFY_OUT_FLAG_ABSORB” action informs WFP that the original packet’s decision should be left to the driver
  2. Code executes in the reverse order to what i describe in point 1. But this scenario is exactly what I want. The driver absorbs the packet and the decision is made in the workitem.

I see your point. If its 2 that I want then the block of code should come first before the workitem. But why would crash happen in case code executes as described in 1.

I check return values for all API calls. I did not want to bloat up this posting so I removed it while posting, including some additional code which I felt may not be relevant to this discussion.

Thank you very much Mark

@Tim_Roberts Yes Tim I assign that structure field pktCtx->layerData and before I do I ensure the address I assign is a valid. This is how I do it

            NtStatus = FwpsAcquireWritableLayerDataPointer(pktCtx->classifyHandle, filter->filterId, 0, &writableLayerData, classifyOut);
            if (!NT_SUCCESS(NtStatus)) {
                    DbgPrint("AppTunnelClassifyCallback (FWPS_LAYER_ALE_BIND_REDIRECT_V4): FwpsAcquireWritableLayerDataPointer failed: 0x%08x\n", NtStatus);
                    FwpsReleaseClassifyHandle(pktCtx->classifyHandle);
                    goto Exit;
            }
             
            if (writableLayerData == NULL) {
                    DbgPrint("CRITICAL (FWPS_LAYER_ALE_BIND_REDIRECT_V4) -  NULL WriteAbleLayerData\n");
                    goto Exit;
            }
            pktCtx->layerData = writableLayerData;

@Mark just an after thought. Even if I move the block before the workitem there would still be a race condition because that block of code wont be effective until I call “return” and during this time the workitem could execute. I agree the code would be better if I move the block before the workitem, but I am wondering is this is really the cause why NETIO.sys is dereferencing a NULL pointer.

Thank you Mark and Tim

Have you tried working through what you want to achieve using inline classification first? (Is there a reason you’re using pend/out of band?)

The callout function executes at DISPATCH_LEVEL. There is some processing I need to do on the first packet in a session and make some API calls which wont work at that IRQL, for example PsLookupProcessByProcessId. So I have this processing deferred inside a WorkItem where I eventually release the packet.