How to send cloned NBLs?

Dear all,

first of all thanks for the valuable information provided by OSR Online and this forum threads.

I am a newbie in Windows driver development and I am trying to learn how to writer a NDIS 6.0 filter driver that drops those packets on the send-to-network path which match certain criterias.

I started using the NDIS filter driver sample project provided by VS2013. Next, I modified the functions FilterSendNetBufferLists(…) and FilterSendNetBufferListsComplete(…). I started with creating cloned NET_BUFFER_LISTs in FilterSendNetBufferLists() which are freed in FilterSendNetBufferListsComplete(). The cloned NET_BUFFER_LISTs are then sent. The driver currently does not drop any packet since in the first step I just wanted to create a clone of all NET_BUFFER_LISTs (NBL) and send these cloned NBLs in order to see if it works. Unfortunately, as soon as the driver is installed, the machine “looses” its IP address and acquires an IP address from 169.254.149.x. So I assume that the driver does not send any network packets to the network and thus, the machine has no connectivity…

The code of FilterSendNetBufferLists(…) is as follows:

Use_decl_annotations
VOID
FilterSendNetBufferLists(
NDIS_HANDLE FilterModuleContext,
PNET_BUFFER_LIST NetBufferLists,
NDIS_PORT_NUMBER PortNumber,
ULONG SendFlags
)
{
PMS_FILTER pFilter = (PMS_FILTER)FilterModuleContext;
BOOLEAN DispatchLevel;
BOOLEAN bFalse = FALSE;
PNBL_QUEUE_HEADER NblSendQueue; // NBLs to passthrough to lower-level miniport
PNET_BUFFER_LIST pCurrentNetBufferList;
PNET_BUFFER_LIST pNextNetBufferList;
PNET_BUFFER_LIST CloneNbl;

DEBUGP(DL_TRACE, “===>SendNetBufferList: NBL = %p.\n”, NetBufferLists);

INIT_NBL_QUEUE_HEADER(&NblSendQueue);

do {
DispatchLevel = NDIS_TEST_SEND_AT_DISPATCH_LEVEL(SendFlags);
FILTER_ACQUIRE_LOCK(&pFilter->Lock, DispatchLevel);

pCurrentNetBufferList = NetBufferLists;
while (pCurrentNetBufferList) {
// Locate the Next NetBufferList
pNextNetBufferList = NET_BUFFER_LIST_NEXT_NBL(pCurrentNetBufferList);

// Unlink NBLs Following The Original NBL
NET_BUFFER_LIST_NEXT_NBL(pCurrentNetBufferList) = NULL;

CloneNbl = NdisAllocateCloneNetBufferList(pCurrentNetBufferList,
NULL,
NULL,
NDIS_CLONE_FLAGS_USE_ORIGINAL_MDLS);
if (!CloneNbl) {
DEBUGP(DL_TRACE, “\tCloning the NET_BUFFER_LIST %p failed.\n”, pCurrentNetBufferList);
INSERT_TAIL_NBL_QUEUE(&NblSendQueue, pCurrentNetBufferList);
} else {
DEBUGP(DL_TRACE, “\tSend cloned NET_BUFFER_LIST.\n”);
pCurrentNetBufferList->ChildRefCount++;
CloneNbl->ParentNetBufferList = pCurrentNetBufferList;
CloneNbl->SourceHandle = FilterModuleContext;
INSERT_TAIL_NBL_QUEUE(&NblSendQueue, CloneNbl);
}
}

// Move to the Next NetBufferList
pCurrentNetBufferList = pNextNetBufferList;
}

FILTER_RELEASE_LOCK(&pFilter->Lock, DispatchLevel);

// Send Passthrough NetBufferLists
if (!IS_NBL_QUEUE_EMPTY(&NblSendQueue)) {
NdisFSendNetBufferLists(pFilter->FilterHandle,
GET_NBL_QUEUE_HEAD(&NblSendQueue),
PortNumber,
SendFlags);
}
} while (bFalse);

DEBUGP(DL_TRACE, “<===SendNetBufferList. \n”);
}

And the function FilterSendNetBufferListsComplete(…) looks as follows:

Use_decl_annotations
VOID
FilterSendNetBufferListsComplete(
NDIS_HANDLE FilterModuleContext,
PNET_BUFFER_LIST NetBufferLists,
ULONG SendCompleteFlags
)
{
PMS_FILTER pFilter = (PMS_FILTER)FilterModuleContext;
ULONG NumOfSendCompletes = 0;
BOOLEAN DispatchLevel;
PNET_BUFFER_LIST CurrNbl;

PNET_BUFFER_LIST NblIter;
PNET_BUFFER_LIST PrevNbl;
PNET_BUFFER_LIST NextNbl;
PNET_BUFFER_LIST ParentNbl;

DEBUGP(DL_TRACE, “===>SendNBLComplete, NetBufferList: %p.\n”, NetBufferLists);

PrevNbl = NULL;
NblIter = NetBufferLists;
NextNbl = NULL;
while (NblIter) {
NextNbl = NET_BUFFER_LIST_NEXT_NBL(NblIter);
if (NblIter->SourceHandle == FilterModuleContext) {
ParentNbl = NblIter->ParentNetBufferList;
if (ParentNbl) {
NET_BUFFER_LIST_STATUS(ParentNbl) = NET_BUFFER_LIST_STATUS(NblIter);
ParentNbl->ChildRefCount–;
NET_BUFFER_LIST_NEXT_NBL(ParentNbl) = NET_BUFFER_LIST_NEXT_NBL(NblIter);
if (!PrevNbl) {
NetBufferLists = ParentNbl;
}
else {
NET_BUFFER_LIST_NEXT_NBL(PrevNbl) = ParentNbl;
}

PrevNbl = ParentNbl;
} else {
PrevNbl = NblIter;
}

NblIter->ParentNetBufferList = NULL;
NdisFreeCloneNetBufferList(NblIter, NDIS_CLONE_FLAGS_USE_ORIGINAL_MDLS);
} else {
PrevNbl = NblIter;
}

// Move to the next net buffer listPrevNbl = NblIter;
NblIter = NextNbl;
}

// Send complete the NBLs. If you removed any NBLs from the chain, make
// sure the chain isn’t empty (i.e., NetBufferLists!=NULL).
if (NetBufferLists) {
NdisFSendNetBufferListsComplete(pFilter->FilterHandle, NetBufferLists, SendCompleteFlags);
}

DEBUGP(DL_TRACE, “<===SendNBLComplete.\n”);
}

So, what`s wrong with my code? Any help, ideas and notes are welcome!
Thanks, sk

To support your inquiry I made a similar modification to a baseline NDIS
LWF. Code is below.

The only “unusual” think that you have overlooked is checksum offload
support. You need to study that.

In the send routine I copy the checksum info from the parent into the clone.

Seems to work OK.

Good luck!

Thomas F. Divine
http://www.pcausa.com
NDIS Protocol and Filter Development Consulting

Use_decl_annotations
VOID
FilterSendNetBufferLists(
NDIS_HANDLE ContextHandle,
PNET_BUFFER_LIST NetBufferLists,
NDIS_PORT_NUMBER PortNumber,
ULONG SendFlags
)
{
PADAPTER_CONTEXT adapter = (PADAPTER_CONTEXT)ContextHandle;
PNET_BUFFER_LIST currentNbl;
BOOLEAN dispatchLevel;
BOOLEAN bFalse = FALSE;
NDF_NBL_QUEUE_HEADER nblSendQueue; // NBLs to passthrough to
lower-level miniport
PNET_BUFFER_LIST nextNbl;

NDF_DBGPRINT(DL_TRACE, DBG_INIT, “===>SendNetBufferList: NBL = %p.\n”,
NetBufferLists);

NdfInitializeNblQueueHeader(&nblSendQueue);

do
{
dispatchLevel = NDIS_TEST_SEND_AT_DISPATCH_LEVEL(SendFlags);

if (adapter->TrackSends)
{
NdfObjectAcquireLock(adapter->AdapterObject,dispatchLevel);

currentNbl = NetBufferLists;
while (currentNbl)
{
adapter->OutstandingSends++;
FILTER_LOG_SEND_REF(1, adapter, currentNbl,
adapter->OutstandingSends);

currentNbl = NET_BUFFER_LIST_NEXT_NBL(currentNbl);
}

NdfObjectReleaseLock(adapter->AdapterObject,dispatchLevel);
}

currentNbl = NetBufferLists;
while (currentNbl)
{
PNET_BUFFER_LIST cloneNbl;

// Locate the Next NetBufferList
nextNbl = NET_BUFFER_LIST_NEXT_NBL(currentNbl);

// Unlink NBLs Following The Original NBL
NET_BUFFER_LIST_NEXT_NBL(currentNbl) = NULL;

cloneNbl = NdisAllocateCloneNetBufferList(currentNbl,
NULL,
NULL,
NDIS_CLONE_FLAGS_USE_ORIGINAL_MDLS);

if (!cloneNbl)
{
NDF_DBGPRINT(DL_WARNING, DBG_SEND, “Cloning the
NET_BUFFER_LIST %p failed.\n”, currentNbl);
NdfInsertTailNblQueue(&nblSendQueue, currentNbl);
}
else
{
NDIS_TCP_IP_CHECKSUM_NET_BUFFER_LIST_INFO nblChecksumInfo;

NDF_DBGPRINT(DL_TRACE, DBG_SEND, “Send cloned
NET_BUFFER_LIST.\n”);

nblChecksumInfo.Value = NET_BUFFER_LIST_INFO(currentNbl,
TcpIpChecksumNetBufferListInfo);

NET_BUFFER_LIST_INFO(cloneNbl,
TcpIpChecksumNetBufferListInfo) = nblChecksumInfo.Value;

currentNbl->ChildRefCount++;
cloneNbl->ParentNetBufferList = currentNbl;
cloneNbl->SourceHandle = ContextHandle;
NdfInsertTailNblQueue(&nblSendQueue, cloneNbl);
}

// Move to the Next NetBufferList
currentNbl = nextNbl;
}

} while (bFalse);

// Send Passthrough NetBufferLists
if (!NdfIsNblQueueEmpty(&nblSendQueue))
{
NdisFSendNetBufferLists(adapter->FilterHandle,
NdfGetNblQueueHead(&nblSendQueue),
PortNumber,
SendFlags);
}

NDF_DBGPRINT(DL_TRACE, DBG_INIT, “<===SendNetBufferList. \n”);
}

Use_decl_annotations
VOID
FilterSendNetBufferListsComplete(
NDIS_HANDLE ContextHandle,
PNET_BUFFER_LIST NetBufferLists,
ULONG SendCompleteFlags
)
{
PADAPTER_CONTEXT adapter = (PADAPTER_CONTEXT)ContextHandle;
ULONG numOfSendCompletes = 0;
BOOLEAN dispatchLevel;
PNET_BUFFER_LIST currentNbl;
PNET_BUFFER_LIST nextNbl;
NDF_NBL_QUEUE_HEADER nblCompleteQueue;

NDF_DBGPRINT(DL_TRACE, DBG_INIT, “===>SendNBLComplete, NetBufferList:
%p.\n”, NetBufferLists);

NdfInitializeNblQueueHeader(&nblCompleteQueue);

dispatchLevel = NDIS_TEST_SEND_AT_DISPATCH_LEVEL(SendCompleteFlags);

if (adapter->TrackSends)
{
currentNbl = NetBufferLists;
while (currentNbl)
{
numOfSendCompletes++;
currentNbl = NET_BUFFER_LIST_NEXT_NBL(currentNbl);
}

NdfObjectAcquireLock(adapter->AdapterObject,dispatchLevel);
adapter->OutstandingSends -= numOfSendCompletes;
FILTER_LOG_SEND_REF(2, adapter, PrevNbl, adapter->OutstandingSends);
NdfObjectReleaseLock(adapter->AdapterObject,dispatchLevel);
}

currentNbl = NetBufferLists;
while (currentNbl)
{
PNET_BUFFER_LIST parentNbl;

// Locate the Next NetBufferList
nextNbl = NET_BUFFER_LIST_NEXT_NBL(currentNbl);

// Unlink NBLs Following The Original NBL
NET_BUFFER_LIST_NEXT_NBL(currentNbl) = NULL;

parentNbl = currentNbl->ParentNetBufferList;

if (parentNbl)
{
parentNbl->Status = currentNbl->Status;

currentNbl->ParentNetBufferList = NULL;
currentNbl->ChildRefCount–;

NdisFreeCloneNetBufferList(
currentNbl,
NDIS_CLONE_FLAGS_USE_ORIGINAL_MDLS
);

NdfInsertTailNblQueue(&nblCompleteQueue, parentNbl);
}
else
{
NdfInsertTailNblQueue(&nblCompleteQueue, currentNbl);
}

// Move to the Next NetBufferList
currentNbl = nextNbl;
}

// Complete These NetBufferLists
if (!NdfIsNblQueueEmpty(&nblCompleteQueue))
{
NdisFSendNetBufferListsComplete(
adapter->FilterHandle,
NdfGetNblQueueHead(&nblCompleteQueue),
SendCompleteFlags
);
}

NDF_DBGPRINT(DL_TRACE, DBG_INIT, “<===SendNBLComplete.\n”);
}

Hello Thomas,

that works like a charm. :slight_smile: Thank you very much!

May I ask one further question regarding dropping packets from the cloned NBL? As I read in the thread http://www.osronline.com/showthread.cfm?link=192918 you already explained that packets (NB) that should be dropped should be unlinked from the cloned NBL. You wrote that after cloning the NBL you also create a companion context info that holds an array of all NBs. I am asking myself how you accomplish this since the NBL does not provide the number of NB within the NBL. So, are you first iterating over all NB in order to count them and afterwards allocating the companion context info (respecting that the size of the context info must be a multiple of the value defined by MEMORY_ALLOCATION_ALIGNMENT)? And after allocating the context info (NdisAllocateNetBufferListContext) I now can iterate over all NBs and drop those that fulfill certain criterias? This seems kind of inefficient since I have to iterate the NBL twice. Is there a better way, e.g. using a local NB array, iterating over the NBL (updating this local array and dropping NBs) and afterwards create the context info? But then the question what should be the size of the local NB array? May I ask what you would propose?
Of course, during the SendComplete callback then the originally NBL is reconstructed using this context info from the NBL.

Thanks again for your time and your advice!
sk

Iterating over a linked-list is quick. No penalty there.

OTOT, unless you are implementing an impairment generator it is unlikely
that you will actually want to drop a single NB in a NBL. More likely you
will want to drop all NBs in a certain NBL; that is: drop the NBL entirely
instead of individual NBs.

Depends on what you are actually doing.

Thomas F. Divine

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@siemens.com
Sent: Monday, April 14, 2014 5:10 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send cloned NBLs?

Hello Thomas,

that works like a charm. :slight_smile: Thank you very much!

May I ask one further question regarding dropping packets from the cloned
NBL? As I read in the thread
http://www.osronline.com/showthread.cfm?link=192918 you already explained
that packets (NB) that should be dropped should be unlinked from the cloned
NBL. You wrote that after cloning the NBL you also create a companion
context info that holds an array of all NBs. I am asking myself how you
accomplish this since the NBL does not provide the number of NB within the
NBL. So, are you first iterating over all NB in order to count them and
afterwards allocating the companion context info (respecting that the size
of the context info must be a multiple of the value defined by
MEMORY_ALLOCATION_ALIGNMENT)? And after allocating the context info
(NdisAllocateNetBufferListContext) I now can iterate over all NBs and drop
those that fulfill certain criterias? This seems kind of inefficient since I
have to iterate the NBL twice. Is there a better way, e.g. using a local NB
array, iterating over the NBL (updating this local array and dropping NBs)
and afterwards create the context info? But then the question what should be
the size of the local NB array? May I ask what you would propose?
Of course, during the SendComplete callback then the originally NBL is
reconstructed using this context info from the NBL.

Thanks again for your time and your advice!
sk


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Hi Thomas,

thanks again for your fast reply! I want to implement a driver that policies the outgoing traffic of a computer according to a token bucket. In my first attempt I dropped the whole NBL if not suffienct tokens were available. That worked but the observed bandwidth was much lower than the configured token bucket should allow. I though that I better could approximate the send bandwidth to the configured one if I instead only drop those NBs that exceed the tokens.

For example: max allowed burst: 3072 byte and actual tokens available: 3072 (with 1 token = 1byte), NBL with 4 NBs each 1500 byte data: In my first attempt the whole NBL was dropped and thus all packets. So I though that it woud better to send the first and second packet and drop the third and fourth one.

For testing/estimating the bandwidth I used netperf.

Maybe you know of a bettter way to accomplish a fine-granular token bucket policing which should ensure that a given burst is never exceeded on the “line”.

Thanks,
sk

In this scenario all you need to do is to “unhitch” NB three and save a
pointer to it in your NBL context. NB three includes pointer to NB four, and
so on. In send complete re-hitch NB three to NB 2 and the chain is restored.

Your NBL context may also include a pointer to NB 2 to simplify
“re-hitching” the NB chain.

It is possible that you would not need a NBL clone. Instead, allocate NBL
context in original send NBL.

Good luck,

Thomas F. Divine

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@siemens.com
Sent: Monday, April 14, 2014 6:57 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send cloned NBLs?

Hi Thomas,

thanks again for your fast reply! I want to implement a driver that policies
the outgoing traffic of a computer according to a token bucket. In my first
attempt I dropped the whole NBL if not suffienct tokens were available. That
worked but the observed bandwidth was much lower than the configured token
bucket should allow. I though that I better could approximate the send
bandwidth to the configured one if I instead only drop those NBs that exceed
the tokens.

For example: max allowed burst: 3072 byte and actual tokens available: 3072
(with 1 token = 1byte), NBL with 4 NBs each 1500 byte data: In my first
attempt the whole NBL was dropped and thus all packets. So I though that it
woud better to send the first and second packet and drop the third and
fourth one.

For testing/estimating the bandwidth I used netperf.

Maybe you know of a bettter way to accomplish a fine-granular token bucket
policing which should ensure that a given burst is never exceeded on the
“line”.

Thanks,
sk


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer