I'm currently writing a passthrough (reference-absorb-insert_dpc-clone-reinject for every packet in every udp/tcp flow) WFP driver for testing purposes.
I noticed that for the outbound transport layer, most of the metadata fields for the TCP protocol have zero values in two cases:
- Client send TCP reset
Stack
01 fffff8035d69eb60 fffff8035cca870b NETIO!ProcessCallout+0x83e
02 fffff8035d69ece0 fffff8035cea1788 NETIO!KfdClassify+0x8bb
03 fffff8035d69f0e0 fffff8035ce3e03b tcpip!ProcessOutboundTransportLayerClassify+0x8ac
04 fffff8035d69f220 fffff8035cdb879d tcpip!WfpProcessOutTransportStackIndication+0x8932b
05 fffff8035d69f550 fffff8035cdb6fda tcpip!IppInspectLocalDatagramsOut+0x10bd
06 fffff8035d69f840 fffff8035cdb6bb4 tcpip!IppSendDatagramsCommon+0x41a
07 fffff8035d69f9c0 fffff8035ce22faf tcpip!IpNlpSendDatagrams+0x44
08 fffff8035d69fa00 fffff8035ce7e620 tcpip!InetSendDatagramsAf+0xbf
09 fffff8035d69fb00 fffff8035cdef98f tcpip!TcpResetSend+0x308
0a fffff8035d69fbc0 fffff8035cdedf64 tcpip!TcpMatchReceive+0xf7f
0b fffff8035d69fe70 fffff8035ce01732 tcpip!TcpReceive+0x2b4
0c fffff8035d69ff60 fffff8035cdbd95c tcpip!TcpNlClientReceiveDatagrams+0x22
0d fffff8035d69ffa0 fffff8035cdba03a tcpip!IppProcessDeliverList+0xcc
0e fffff8035d6a0080 fffff8035cdbe82b tcpip!IppReceiveHeaderBatch+0x3aa
0f fffff8035d6a0180 fffff8035cdc05ec tcpip!IppFlcReceivePacketsCore+0x48b
10 fffff8035d6a0280 fffff8035ce08ea0 tcpip!IpFlcReceivePackets+0xc
11 fffff8035d6a02b0 fffff8035ce0849c tcpip!FlpReceiveNonPreValidatedNetBufferListChain+0x270
12 fffff8035d6a03b0 fffff8035acaa6f8 tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x17c
13 fffff8035d6a0500 fffff8035acaa66d nt!KeExpandKernelStackAndCalloutInternal+0x78
14 fffff8035d6a0570 fffff8035cdfa7ed nt!KeExpandKernelStackAndCalloutEx+0x1d
15 fffff8035d6a05b0 fffff8035cdf9ecd tcpip!NetioExpandKernelStackAndCallout+0x8d
16 fffff8035d6a0610 fffff8035cb31eb1 tcpip!FlReceiveNetBufferListChain+0x46d
17 fffff8035d6a08c0 fffff8035cb31ccb ndis!ndisMIndicateNetBufferListsToOpen+0x141
18 fffff8035d6a09a0 fffff8035cb37fe1 ndis!ndisMTopReceiveNetBufferLists+0x22b
19 fffff8035d6a0a20 fffff8035cb6e3af ndis!ndisCallReceiveHandler+0x61
1a fffff8035d6a0a70 fffff8035cb34b84 ndis!ndisInvokeNextReceiveHandler+0x1df
1b fffff8035d6a0b40 fffff8035e2a2745 ndis!NdisMIndicateReceiveNetBufferLists+0x104
1c fffff8035d6a0bd0 fffff8035ac47582 kdnic!RxReceiveIndicateDpc+0x1e5
1d fffff8035d6a0c30 fffff8035ac465fd nt!KiProcessExpiredTimerList+0x172
1e fffff8035d6a0d20 fffff8035ae05dc5 nt!KiRetireDpcList+0x5dd
1f fffff8035d6a0fb0 fffff8035ae05bb0 nt!KxRetireDpcList+0x5
20 ffff8485d15170a0 fffff8035ae052f5 nt!KiDispatchInterruptContinue
21 ffff8485d15170d0 fffff8035adffdf1 nt!KiDpcInterruptBypass+0x25
22 ffff8485d15170e0 fffff8035ad8d916 nt!KiInterruptDispatchNoLockNoEtw+0xb1
23 ffff8485d1517270 fffff8035ad8dbcb nt!HvlGetReferenceTimeUsingTscPage+0x36
24 ffff8485d15172a0 fffff8035ac2bd85 nt!HalpHvCounterQueryCounter+0x1b
25 ffff8485d15172d0 ffff8908c8a3717e nt!KeQueryPerformanceCounter+0xd5
26 ffff8485d1517300 ffff8908c8a3766a win32kbase!EtwTraceReleaseUserCrit+0x4a
27 ffff8485d15173d0 ffff8908c8dd1ba3 win32kbase!UserSessionSwitchLeaveCrit+0x6a
28 ffff8485d1517440 ffff8908c98348b6 win32kfull!NtUserGetQueueStatus+0x153
29 ffff8485d1517490 fffff8035ae11505 win32k!NtUserGetQueueStatus+0x16
2a ffff8485d15174c0 00007ff95ea48614 nt!KiSystemServiceCopyEnd+0x25
2b 000000000503e588 00007ff95ea33a2b 0x00007ff95ea48614 2c 000000000503e590 0000000077404660 0x00007ff95ea33a2b
2d 000000000503e598 000000000594e170 0x77404660
2e 000000000503e5a0 000000000503fda0 0x594e170
2f 000000000503e5a8 0000000004e1b000 0x503fda0
30 000000000503e5b0 0000000004e1d000 0x4e1b000
31 000000000503e5b8 00007ff95f7290da 0x4e1d000
32 000000000503e5c0 0000000000000000 0x00007ff9`5f7290da
Metadata fields
0: kd> dx -r1 pMetaValues
pMetaValues : 0xffff9f06655c3c70 [Type: FWPS_INCOMING_METADATA_VALUES0_ *]
[+0x000] currentMetadataValues : 0x8cc0 [Type: unsigned int]
[+0x004] flags : 0x1 [Type: unsigned int]
[+0x008] reserved : 0xfffff8035d69f178 [Type: unsigned _int64]
[+0x010] discardMetadata [Type: FWPS_DISCARD_METADATA0]
[+0x020] flowHandle : 0x0 [Type: unsigned _int64]
[+0x028] ipHeaderSize : 0x0 [Type: unsigned int]
[+0x02c] transportHeaderSize : 0x14 [Type: unsigned int]
[+0x030] processPath : 0x0 [Type: FWP_BYTE_BLOB *]
[+0x038] token : 0x0 [Type: unsigned __int64]
[+0x040] processId : 0x0 [Type: unsigned _int64]
[+0x048] sourceInterfaceIndex : 0x0 [Type: unsigned int]
[+0x04c] destinationInterfaceIndex : 0x0 [Type: unsigned int]
[+0x050] compartmentId : 0x1 [Type: unsigned long]
[+0x054] fragmentMetadata [Type: FWPS_INBOUND_FRAGMENT_METADATA0]
[+0x060] pathMtu : 0x0 [Type: unsigned long]
[+0x068] completionHandle : 0x0 [Type: void *]
[+0x070] transportEndpointHandle : 0x0 [Type: unsigned _int64]
[+0x078] remoteScopeId [Type: SCOPE_ID]
[+0x080] controlData : 0x0 [Type: cmsghdr *]
[+0x088] controlDataLength : 0x0 [Type: unsigned long]
[+0x08c] packetDirection : FWP_DIRECTION_OUTBOUND (0) [Type: FWP_DIRECTION]
[+0x090] headerIncludeHeader : 0x0 [Type: void *]
[+0x098] headerIncludeHeaderLength : 0x0 [Type: unsigned long]
[+0x09c] destinationPrefix [Type: _IP_ADDRESS_PREFIX]
[+0x0bc] frameLength : 0x0 [Type: unsigned short]
[+0x0c0] parentEndpointHandle : 0x0 [Type: unsigned __int64]
[+0x0c8] icmpIdAndSequence : 0x0 [Type: unsigned int]
[+0x0cc] localRedirectTargetPID : 0x0 [Type: unsigned long]
[+0x0d0] originalDestination : 0x0 [Type: sockaddr *]
[+0x0d8] redirectRecords : 0x0 [Type: void *]
[+0x0e0] currentL2MetadataValues : 0x0 [Type: unsigned int]
[+0x0e4] l2Flags : 0x0 [Type: unsigned int]
[+0x0e8] ethernetMacHeaderSize : 0x0 [Type: unsigned int]
[+0x0ec] wiFiOperationMode : 0x0 [Type: unsigned int]
[+0x0f0] vSwitchSourcePortId : 0x0 [Type: unsigned int]
[+0x0f4] vSwitchSourceNicIndex : 0x0 [Type: unsigned short]
[+0x0f8] vSwitchDestinationPortId : 0x0 [Type: unsigned int]
[+0x100] vSwitchPacketContext : 0x0 [Type: void *]
[+0x108] subProcessTag : 0x0 [Type: void *]
[+0x110] reserved1 : 0x0 [Type: unsigned __int64]
- Client send ACK on server FIN (time-wait)
Stack
01 fffff8035d69eb00 fffff8035cca870b NETIO!ProcessCallout+0x83e
02 fffff8035d69ec80 fffff8035cea1788 NETIO!KfdClassify+0x8bb
03 fffff8035d69f080 fffff8035ce3e03b tcpip!ProcessOutboundTransportLayerClassify+0x8ac
04 fffff8035d69f1c0 fffff8035cdb879d tcpip!WfpProcessOutTransportStackIndication+0x8932b
05 fffff8035d69f4f0 fffff8035cdb6fda tcpip!IppInspectLocalDatagramsOut+0x10bd
06 fffff8035d69f7e0 fffff8035cdb6bb4 tcpip!IppSendDatagramsCommon+0x41a
07 fffff8035d69f960 fffff8035ce22faf tcpip!IpNlpSendDatagrams+0x44
08 fffff8035d69f9a0 fffff8035ce22c62 tcpip!InetSendDatagramsAf+0xbf
09 fffff8035d69faa0 fffff8035ce22df4 tcpip!TcpTimeWaitTcbSend+0x14a
0a fffff8035d69fb30 fffff8035cdef03a tcpip!TcpTimeWaitTcbReceive+0x164
0b fffff8035d69fbc0 fffff8035cdee0fd tcpip!TcpMatchReceive+0x62a
0c fffff8035d69fe70 fffff8035ce01732 tcpip!TcpReceive+0x44d
0d fffff8035d69ff60 fffff8035cdbd95c tcpip!TcpNlClientReceiveDatagrams+0x22
0e fffff8035d69ffa0 fffff8035cdba03a tcpip!IppProcessDeliverList+0xcc
0f fffff8035d6a0080 fffff8035cdbe82b tcpip!IppReceiveHeaderBatch+0x3aa
10 fffff8035d6a0180 fffff8035cdc05ec tcpip!IppFlcReceivePacketsCore+0x48b
11 fffff8035d6a0280 fffff8035ce08ea0 tcpip!IpFlcReceivePackets+0xc
12 fffff8035d6a02b0 fffff8035ce0849c tcpip!FlpReceiveNonPreValidatedNetBufferListChain+0x270
13 fffff8035d6a03b0 fffff8035acaa6f8 tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x17c
14 fffff8035d6a0500 fffff8035acaa66d nt!KeExpandKernelStackAndCalloutInternal+0x78
15 fffff8035d6a0570 fffff8035cdfa7ed nt!KeExpandKernelStackAndCalloutEx+0x1d
16 fffff8035d6a05b0 fffff8035cdf9ecd tcpip!NetioExpandKernelStackAndCallout+0x8d
17 fffff8035d6a0610 fffff8035cb31eb1 tcpip!FlReceiveNetBufferListChain+0x46d
18 fffff8035d6a08c0 fffff8035cb31ccb ndis!ndisMIndicateNetBufferListsToOpen+0x141
19 fffff8035d6a09a0 fffff8035cb37fe1 ndis!ndisMTopReceiveNetBufferLists+0x22b
1a fffff8035d6a0a20 fffff8035cb6e3af ndis!ndisCallReceiveHandler+0x61
1b fffff8035d6a0a70 fffff8035cb34b84 ndis!ndisInvokeNextReceiveHandler+0x1df
1c fffff8035d6a0b40 fffff8035e2a2745 ndis!NdisMIndicateReceiveNetBufferLists+0x104
1d fffff8035d6a0bd0 fffff8035ac47582 kdnic!RxReceiveIndicateDpc+0x1e5
1e fffff8035d6a0c30 fffff8035ac465fd nt!KiProcessExpiredTimerList+0x172
1f fffff8035d6a0d20 fffff8035ae05dc5 nt!KiRetireDpcList+0x5dd
20 fffff8035d6a0fb0 fffff8035ae05bb0 nt!KxRetireDpcList+0x5
21 ffff8485d04d7190 fffff8035ae04fad nt!KiDispatchInterruptContinue
22 ffff8485d04d71c0 fffff8035acc41c4 nt!KiDpcInterrupt+0x2fd
23 ffff8485d04d7350 fffff8035acc3d1a nt!KiExitDispatcher+0x74
24 ffff8485d04d73c0 fffff8035b0625cd nt!KeSetEvent+0xea
25 ffff8485d04d7450 fffff8035ae11505 nt!NtSetEvent+0xbd
26 ffff8485d04d74c0 0000000077401cfc nt!KiSystemServiceCopyEnd+0x25
27 000000000ab0f068 0000000077401cbb 0x77401cfc
28 000000000ab0f070 000000237748322c 0x77401cbb
29 000000000ab0f078 0000000000000000 0x00000023`7748322c
Metadata fields
0: kd> dx -r1 pMetaValues
pMetaValues : 0xffff9f06655c3c70 [Type: FWPS_INCOMING_METADATA_VALUES0_ *]
[+0x000] currentMetadataValues : 0x8cc0 [Type: unsigned int]
[+0x004] flags : 0x1 [Type: unsigned int]
[+0x008] reserved : 0xfffff8035d69f118 [Type: unsigned _int64]
[+0x010] discardMetadata [Type: FWPS_DISCARD_METADATA0]
[+0x020] flowHandle : 0x0 [Type: unsigned _int64]
[+0x028] ipHeaderSize : 0x0 [Type: unsigned int]
[+0x02c] transportHeaderSize : 0x14 [Type: unsigned int]
[+0x030] processPath : 0x0 [Type: FWP_BYTE_BLOB *]
[+0x038] token : 0x0 [Type: unsigned __int64]
[+0x040] processId : 0x0 [Type: unsigned _int64]
[+0x048] sourceInterfaceIndex : 0x0 [Type: unsigned int]
[+0x04c] destinationInterfaceIndex : 0x0 [Type: unsigned int]
[+0x050] compartmentId : 0x1 [Type: unsigned long]
[+0x054] fragmentMetadata [Type: FWPS_INBOUND_FRAGMENT_METADATA0]
[+0x060] pathMtu : 0x0 [Type: unsigned long]
[+0x068] completionHandle : 0x0 [Type: void *]
[+0x070] transportEndpointHandle : 0x0 [Type: unsigned _int64]
[+0x078] remoteScopeId [Type: SCOPE_ID]
[+0x080] controlData : 0x0 [Type: cmsghdr *]
[+0x088] controlDataLength : 0x0 [Type: unsigned long]
[+0x08c] packetDirection : FWP_DIRECTION_OUTBOUND (0) [Type: FWP_DIRECTION]
[+0x090] headerIncludeHeader : 0x0 [Type: void *]
[+0x098] headerIncludeHeaderLength : 0x0 [Type: unsigned long]
[+0x09c] destinationPrefix [Type: _IP_ADDRESS_PREFIX]
[+0x0bc] frameLength : 0x0 [Type: unsigned short]
[+0x0c0] parentEndpointHandle : 0x0 [Type: unsigned __int64]
[+0x0c8] icmpIdAndSequence : 0x0 [Type: unsigned int]
[+0x0cc] localRedirectTargetPID : 0x0 [Type: unsigned long]
[+0x0d0] originalDestination : 0x0 [Type: sockaddr *]
[+0x0d8] redirectRecords : 0x0 [Type: void *]
[+0x0e0] currentL2MetadataValues : 0x0 [Type: unsigned int]
[+0x0e4] l2Flags : 0x0 [Type: unsigned int]
[+0x0e8] ethernetMacHeaderSize : 0x0 [Type: unsigned int]
[+0x0ec] wiFiOperationMode : 0x0 [Type: unsigned int]
[+0x0f0] vSwitchSourcePortId : 0x0 [Type: unsigned int]
[+0x0f4] vSwitchSourceNicIndex : 0x0 [Type: unsigned short]
[+0x0f8] vSwitchDestinationPortId : 0x0 [Type: unsigned int]
[+0x100] vSwitchPacketContext : 0x0 [Type: void *]
[+0x108] subProcessTag : 0x0 [Type: void *]
[+0x110] reserved1 : 0x0 [Type: unsigned __int64]
In my case, the problem was the absence (zero value) of the transportEndpointHandle (FWPS_IS_METADATA_FIELD_PRESENT(pMetaValues,
FWPS_METADATA_FIELD_TRANSPORT_ENDPOINT_HANDLE) is true) field. Because of this, the FwpsInjectTransportSendAsync function returns error in InjectionComplete. This results in a situation where the packet was absorbed, and than reinjected with error
The absence of other fields can also create a problem. For example, drivers using FWP_CALLOUT_FLAG_CONDITIONAL_ON_FLOW can skip these packets, since flowHandle is also empty.
Also, in popular examples, such as dd_proxy and WFPSampler, this situation is not handled in any way.
This looks like a bug. How to handle such situations correctly? Is it possible to correctly pend such packets? Or perhaps it just doesn't make sense?