Indicating packets from WSK to NetAdapterCx

lstipakov

Hello,

I have already asked this question on Microsoft forums (https://docs.microsoft.com/en-us/answers/questions/541601/indicating-packets-from-wsk-to-netadaptercx.html, but I feel I could get a (faster) answer here.

I am working on the virtual network adapter driver which uses NetAdapterCx, WSK and CNG.

Here is what driver does on Tx path:

• iterate over packets in "post" subsection of NET_RING

• for each packet, get the first fragment and fragment's MDL. If MDL is NULL (packet was bounced by NetAdapter), allocate MDL from fragment's VA.

• encrypt MDL chain with CNG using chaining mode (account for plaintext length must be multiplier of block size)

• send MDL chain with WskSendTo/WskSend, pass NET_PACKET as completion routine context

• move sent fragments and packets to "drain" subsection by adjusting NextIndex

• in WSK completion routine, set NET_PACKET::Scratch to 1 to indicate that packet has been sent

• iterate over packets in "drain" subsection and drain them to OS by adjusting BeginIndex if Scratch is set to 1

The code could be found here: https://github.com/lstipakov/ovpn-dco-win/blob/zerocopy/txqueue.cpp#L139

My question is - what would be the proper way to implement the same "zerocopy" approach on Rx path? At the moment it works like this:

• packet is received by WskReceiveFromEvent callback

• packet is decrypted into ciphertext buffer, fetched from pre-allocated "producer" pool (I use DMF_BufferQueue)

• ciphertext buffer is enqueued into "consumer" queue

• call NetRxQueueNotifyMoreReceivedPacketsAvailable() to trigger Rx queue's Advance callback

• in Rx Advance callback, iterate over fragments, dequeue buffer from "consumer" queue and copy buffer content to fragment's VA

• buffer is "reused" by placing into "producer" pool

I can do decryption in-place and make WSK retain data by returning STATUS_PENDING from WskReceiveFromEvent callback, but how do I "indicate" data provided by WSK to NetAdapter without memcpying? Can I somehow tell NET_FRAGMENT "hey use this MDL which I got from WSK and decrypted in-place" ?

msr

Hi

See if AllocModeDriver helps

typedef enum _NET_RX_FRAGMENT_BUFFER_ALLOCATION_MODE {
NetRxFragmentBufferAllocationModeSystem = ,
**NetRxFragmentBufferAllocationModeDriver = **
} NET_RX_FRAGMENT_BUFFER_ALLOCATION_MODE;

Thanks

lstipakov

Hi,

See if AllocModeDriver helps

Yeah thanks, that was the case indeed. I also figured it out by reading NetAdapter's code. Here is how I made it work in my driver:

• When setting datapath capabilities, we use NET_ADAPTER_RX_CAPABILITIES_INIT_DRIVER_MANAGED macro. We also specify EvtAdapterReturnRxBuffer callback.

• In WskReceiveFromEvent callback we iterate over WSK_DATAGRAM_INDICATION list and enqueue each item into cosumer pool (from DMF_BufferQueue)

    // each datagram indication is one UDP datagram
    for (PWSK_DATAGRAM_INDICATION next; dataIndication != NULL; dataIndication = next) {
        next = dataIndication->Next;
  
        // break list so that we can pass individual WSK_DATAGRAM_INDICATION to WskRelease
        dataIndication->Next = NULL;
  
        // fetch buffer from producer
        OVPN_RX_BUFFER* rxBuffer;
        LOG_IF_NOT_NT_SUCCESS(OvpnBufferQueueFetch(device->DataRxBufferQueue, &rxBuffer));
        rxBuffer->DatagramIndication = dataIndication;
  
        // enqueue buffer to consumer, which is consumed by NetAdapter's RX Advance callback
        OvpnBufferQueueEnqueue(device->DataRxBufferQueue, rxBuffer);
    }
  
    // tell NetAdapter that we have something to consume
    OvpnAdapterNotifyRx(device->Adapter);
  

• In RX queue's Advance callback we iterate over queue's fragments. We dequeue datagram indication and set fragment properties based on datagram indication's buffer and MDL:

  while (NetFragmentIteratorHasAny(&fi)) {
      OVPN_RX_BUFFER* buffer;
      // nothing has arrived and decrypted yet?
      if (!NT_SUCCESS(OvpnBufferQueueDequeue(bufferQueue, &buffer))) {
          break;
      }
  
      fragment = NetFragmentIteratorGetFragment(&fi);
  
      PWSK_DATAGRAM_INDICATION datagramIndication = buffer->DatagramIndication;
      // return buffer to producer queue
      OvpnBufferQueueReuse(bufferQueue, buffer);
  
      PMDL mdl = datagramIndication->Buffer.Mdl;
  
      // TODO: correctly adjust crypto/ovpn overhead
      fragment->ValidLength = datagramIndication->Buffer.Length - 8;
      fragment->Offset = datagramIndication->Buffer.Offset + 8;
      fragment->Capacity = MmGetMdlByteCount(mdl);
  
      NET_FRAGMENT_VIRTUAL_ADDRESS* virtualAddr = NetExtensionGetFragmentVirtualAddress(&queue->VirtualAddressExtension, NetFragmentIteratorGetIndex(&fi));
      virtualAddr->VirtualAddress = (PUCHAR)(MmGetSystemAddressForMdlSafe(mdl, LowPagePriority));
  
      // TODO: handle case when packet (DataIndication) contains multiple fragments
      NET_PACKET* packet = NetPacketIteratorGetPacket(&pi);
      packet->FragmentIndex = NetFragmentIteratorGetIndex(&fi);
      packet->FragmentCount = 1;
  
      packet->Layout = {};
  
      // NetAdapter will call ReturnRxBuffer callback when it is done with buffers, there we return datagramIndication back to WSK
      returnCtx = NetExtensionGetFragmentReturnContext(&queue->ReturnContextExtension, NetFragmentIteratorGetIndex(&fi));
      returnCtx->Handle = (NET_FRAGMENT_RETURN_CONTEXT_HANDLE)datagramIndication;
  
      NetFragmentIteratorAdvance(&fi);
      NetPacketIteratorAdvance(&pi);
  }
  NetFragmentIteratorSet(&fi);
  NetPacketIteratorSet(&pi);
  

• In NetAdapter's EvtAdapterReturnRxBuffer callback we call WskRelease on a datagram indication, which is passed as NET_FRAGMENT_RETURN_CONTEXT_HANDLE by NetAdapter:

  _Use_decl_annotations_
  void
  OvpnEvtAdapterReturnRxBuffer(NETADAPTER netAdapter, NET_FRAGMENT_RETURN_CONTEXT_HANDLE rxReturnContext)
  {
      POVPN_ADAPTER adapter = OvpnGetAdapterContext(netAdapter);
      POVPN_DEVICE device = OvpnGetDeviceContext(adapter->WdfDevice);
  
      PWSK_DATAGRAM_INDICATION dataIndication = (PWSK_DATAGRAM_INDICATION)rxReturnContext;
  
      LOG_IF_NOT_NT_SUCCESS(((WSK_PROVIDER_DATAGRAM_DISPATCH*)device->Socket.Socket->Dispatch)->WskRelease(device->Socket.Socket, dataIndication));
  }
  

Not sure if this is the "best practices", but it works and doesn't crash on iperf3 tests.

msr

Hi

NetAdapterCx has some double dozen DV checks. You can enable verifier once and check all o.k. as well.

One thing I hope NetAdapterCx does is burst returning of return conetxts. Right now it returns 1 by 1 thru EvtAdapterReturnRxBuffer. Maybe it can use a ring abstraction (if it doesn't want to bloatNET_FRAGMENT_RETURN_CONTEXT_HANDLE abstraction) here as well, to burst return bunch of return-contexts insetad of making EvtAdapterReturnRxBuffer callback into driver for every rx-completion.

Thanks