Packet fragments in MiniportSend()

NTDEV
1

Hello all,

I have developed a VNIC driver (TAP adapter) (NDIS5) based on the e100bex sample in the 3790.1830 DDK. This driver is used to “hook” packets, generated locally on the machine, to an application. The application specifices the destination of the packets it is interested in and any matching packets are copied to the queue (just a linked list of buffers which read using IRP_MJ_READ) instead of being sent to their appropriate destination.

For single fragment packets, everything works fine. However, when a packet goes above the MTU specified by the driver, only the last fragment of the packet is received in MiniportSend(), without any calls to MiniportSend() for any of the other fragments. Increasing the MTU in the driver allows for the packet to be received in one chunk, but this is not a valid fix.

Am I missing some special way to handle fragmentation if IP packets? Perhaps, am I skipping some specific OID required to allow a miniport driver to receive packet fragments?

Additional info:
From what I have managed to understand, NDIS_PACKET only represents a single packet and does not need to be walked through to obtain any subsequent packets. I have attempted using MiniportSendPackets() but it was also being called for one packet, the last fragment.

Regards,
Matt

2

I think you need to back up and verify that what you think you are seeing is what is actually happening.

TCPIP.SYS (via its ARP layer) will present each ‘fragment’ of a fragmented IP datagram as individual IP packets encapsulated in Ethernet framing at MiniportSendPackets or at MiniportSend depending on which you have declared support for.

At the MAC layer where your [virtual] Miniport is operating, it is always MAC packets with a payload of an IP packet (not datagram) that have been prepared specifically for that links framing and send constraints. If your application expects IP datagrams then something has to perform ‘reassembly’. Perhaps your application does that.

There are no OIDs involved and it surely is not reasonable to set the link MTU so large as to always be able to carry a maximum sized IP datagram.

Something else is likely amiss in your driver, diagnostics, or perception (measurement) of what is going on.

Have you tried capturing network traffic sent to your adapter by using NetMon or Wireshark? Do they show that only the last fragment of a fragmented IP datagram are being sent to your adapter?

Good Luck,
Dave Cattley

3

Thanks for your reply.

I am already handling IP packet reassemly at userspace. The application expects Ethernet frames, which it decapsulates, checks for fragmentation, and reassembles the fragments into a single IP packet.

What I am expecting is multiple Ethernet frames, each containing a fragment of the IP packet that was generated by the system, but alas, only the last such frame is being received (only one call to MiniportSend()).

I have checked this using Wireshark, and all fragments were visable.

4

Build NETVMINI without changes form the WDK. Install it, and run you test
with it. I have very high confidence that you will see all fragments of a
fragmented IP datagram in the send path.

If Wireshark sees them all then they are all going from the protocol (TCPIP)
to the MAC (your miniport). And since NDIS does not really provide much in
the way of discrimination in the send path on what shows up in your MAC, I
would expect them to be there.

You don’t have some third-party firewall based on a IM driver installed do
you? One that might, for example, have a policy of blocking fragments?
Because that policy is *implemented* by only allowing the last fragment
through.

Rule #1 in Network Driver development and testing: Kick everyone out of the
pool until you can swim, let them back in one at a time.

Good Luck,
Dave Cattley

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, May 19, 2011 5:24 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Packet fragments in MiniportSend()

Thanks for your reply.

I am already handling IP packet reassemly at userspace. The application
expects Ethernet frames, which it decapsulates, checks for fragmentation,
and reassembles the fragments into a single IP packet.

What I am expecting is multiple Ethernet frames, each containing a fragment
of the IP packet that was generated by the system, but alas, only the last
such frame is being received (only one call to MiniportSend()).

I have checked this using Wireshark, and all fragments were visable.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

5

I’ll try NETVMINI like you suggested.

There are no other drivers or applications that should be interfering and Windows Firewall is turned off. I have noticed this behaviour on two Windows XP machines (SP2 and SP3). On Windows 7 it did work without any problems.

6

Windows Firewall is not what I am wondering about. Since Wireshark shows the packets being sent, Windows Firewall is not an issue as it operates inside the TCP/IP stack. Antivirus products with filtering for IDP or firewalling are what I am wondering about.

Break into your system with WinDBG/KD and type:

!ndiskd.protocols

and determine if your adapter is directly bound to TCPIP or not.

Good Luck,
Dave Cattley

Date: Thu, 19 May 2011 08:22:05 -0400
From: xxxxx@gmail.com
To: xxxxx@lists.osr.com
Subject: RE:[ntdev] Packet fragments in MiniportSend()

I’ll try NETVMINI like you suggested.

There are no other drivers or applications that should be interfering and Windows Firewall is turned off. I have noticed this behaviour on two Windows XP machines (SP2 and SP3). On Windows 7 it did work without any problems.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

Thanks for the replies. The driver does seem to be directly bound.

I executed the tests from ndistester and everything worked fine. I still can’t figure out what is wrong. Currently , my major concern is that it works no Windows Vista/2k8/7 but not XP (might be something with the NDIS5 to NDIS6 conversion layer that allows the driver to work?).

8

I have installed a Windows XP checked build (SP2) and now, SendHandler is only being called for ARP requests (no other packets). !ndiskd.miniport gave the following:
Miniport 81ddb410 : ADP, v0.0

AdapterContext : 81d5c210
Flags : 28452000
IGNORE_TOKEN_RING_ERRORS, DESERIALIZED, RESOURCES_AVAILABLE
DOES_NOT_DO_LOOPBACK, MEDIA_CONNECTED,
NOT_BUS_MASTER, NOT_SUPPORTS_MEDIA_SENSE,
PnPFlags : 00018000
NO_HALT_ON_SUSPEND, RECEIVED_START,
MiniportState : STATE_UNDEFINED
IfIndex : 0
Ndis5MiniportInNdis6Mode : 0
InternalResetCount : 0000
MiniportResetCount : 0000
References : 2
UserModeOpenReferences: 0
PnPDeviceState : PNP_DEVICE_STARTED
CurrentDevicePowerState : PowerDeviceD0
Bus PM capabilities
DeviceD1: 0
DeviceD2: 0
WakeFromD0: 0
WakeFromD1: 0
WakeFromD2: 0
WakeFromD3: 0

SystemState DeviceState
PowerSystemUnspecified PowerDeviceUnspecified
S0 D0
S1 D3
S2 D3
S3 D3
S4 D3
S5 D3
SystemWake: PowerSystemUnspecified
DeviceWake: PowerDeviceUnspecified
Current PnP and PM Settings: : 00000030
DISABLE_WAKE_UP, DISABLE_WAKE_ON_RECONNECT,
No Resources Allocated
MediaType : 802.3
DeviceObject : 81ddb310, PhysDO : 8219ba60 Next DO: 8219ba60
MapRegisters : 00000000
FirstPendingPkt: 00000000
DriverVerifyFlags : 00000000
Miniport Interrupt : 00000000
Miniport version 0.0
Miniport Filter List:
Miniport Open Block Queue:
81c67da8: Protocol 82112700 = PSCHED, ProtocolBindingContext 81e76d20, v5.0

I’m hopelessly stuck, any help is greatly appreciated.

9

By SendHandler do you mean MiniportSend or MiniportSendPackets?

Of course either should work but I would favor having a MiniportSendPackets
based send-path.

Only ARP ‘requests’? Meaning you decoded the packet and found it to be an
ARP-REQ?

Back to an ealier post… So, what about NETVMINI? Does that work for
you in its unmodified state?

Good Luck,
Dave Cattley

10

By SendHandler I mean MiniportSend. I will give a shot at MiniportSendPackets once again and see what happens with the checked build.

“Only ARP ‘requests’? Meaning you decoded the packet and found it to be an ARP-REQ?” That is what I meant. Sorry for not being clear.

NETVMINI and other drivers, including a driver similar to the one I’m testing, work fine. It’s just this one that is having troubles.

Thansk for your help,
Matt

11

It was clear but I wanted to be sure you mean “ARP Request” by content and
not a request from the ARP module of TCPIP (say be looking at a stack
backtrace). All sends from TCPIP that go over Ethernet originate from the
“ARP” interface layer of TCPIP. I just wanted to be sure I was not over
reading detail into what you were saying.

RE: NETVMINI - ok, the good news is that your environment is fine. The bad
news is that you driver is not. At this point you have an A-B comparison
exercise between NETVMINI’s very simple but operational miniport edge and
your driver’s miniport edge.

Make sure that you have advertised reasonable values for buffer size and max
buffers, that you are returning packets when completed, etc. In other
words, copy the send path from NETVMINI and attach it to your send
processing.

Good Luck,
Dave Cattley

12

I’m looking into that, nothing obvious was noted yet but we’ll see.

On a sidenote, I noted that OID_GEN_LINK_SPEED is being called each time I expect a packet to be MiniportSendPackets to be called with a packet that does not contain ARP-REQ (say it contains a ping). Is this normal? (I am setting this value to 1,000,000 and is reported fine in the adapter properties).

13

OID_GEN_LINK_SPEED and other ‘statistics’ related OIDs will get called quite
often. I cannot say I have ever noted any correlation in time between
OID_GEN_LINK_SPEED and sending IP packets.

Cheers,
Dave Cattley

14

It turns out that this problem was caused by improperly setting data for OID_GEN_CURRENT_PACKET_FILTER (the appropriate filter was not being set). I’m not going to try to understand why on different machines it had different effects, but now it works.

Thanks for your help!
Matt