NDIS Intermediate Driver - Adding new header

Hi,

I am designing a new layer between TCP/IP and MAC layer using NDIS
intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer
has its own header and few control packets. I wanted to know whether it
is possible to change the packet structure in this intermediate driver
(PassThru drivers) or not. Also, whether this is a good design or not.
If not, what are the other ways in which I can implement this middle
layer between TCP/IP and MAC layer?

I am confused because at many places, it suggests not to modify packet
data virtual memory or buffer descriptor at the intermediate drivers if
you haven’t allocated it.

I will greatly appreciate any help in this regard. Thanks in advance.

Best Regards,

-Niranjan

“Niranjan” wrote in message news:xxxxx@ntdev…
Hi,

I am designing a new layer between TCP/IP and MAC layer using NDIS intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer has its own header and few control packets. I wanted to know whether it is possible to change the packet structure in this intermediate driver (PassThru drivers) or not. Also, whether this is a good design or not. If not, what are the other ways in which I can implement this middle layer between TCP/IP and MAC layer?

NDIS IM drivers seem to be a good choice.

I am confused because at many places, it suggests not to modify packet data virtual memory or buffer descriptor at the intermediate drivers if you haven’t allocated it.

This is not a suggestion. If you didn’t allocate the VM, then you should not change or modify it. You must invent a method that inserts your header without modifying other VM or other NDIS buffer descriptors.

The simplest approach is to allocate your own VM sufficiently large to accomodate the original packet plus your header. Copy the MAC header, your header and then the original packet payload into your own VM. Then allocate your own NDIS buffer and packet descriptor and send that instead of the original.

If you get more experiance with the idea behind NDIS buffer descriptors and memory descriptor lists you may see a method that involves less copying.

Thomas F. Divine

I will greatly appreciate any help in this regard. Thanks in advance.

Best Regards,

-Niranjan

Hi Thomas,

I was under the impression that one can change data allocated by
tcp/ip, e.g. changing the destination IP address, without having to
allocate your own packet and buffer descriptors. Am I wrong in this ?
Obviously this involves handling checksum and TCP task offload issues.

On the other hand, adding additional data such as Niranjan described
requires the copy and allocation of your own packet and buffer
descriptors ?

Regards

Beyers Cronje

On 8/16/05, Thomas F. Divine wrote:
>
>
>
> “Niranjan” wrote in message news:xxxxx@ntdev…
>
>
> Hi,
>
>
>
> I am designing a new layer between TCP/IP and MAC layer using NDIS
> intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer has
> its own header and few control packets. I wanted to know whether it is
> possible to change the packet structure in this intermediate driver
> (PassThru drivers) or not. Also, whether this is a good design or not. If
> not, what are the other ways in which I can implement this middle layer
> between TCP/IP and MAC layer?
>
>
>
> NDIS IM drivers seem to be a good choice.
>
>
>
>
> I am confused because at many places, it suggests not to modify packet data
> virtual memory or buffer descriptor at the intermediate drivers if you
> haven’t allocated it.
>
>
>
> This is not a suggestion. If you didn’t allocate the VM, then you should not
> change or modify it. You must invent a method that inserts your header
> without modifying other VM or other NDIS buffer descriptors.
>
>
>
> The simplest approach is to allocate your own VM sufficiently large to
> accomodate the original packet plus your header. Copy the MAC header, your
> header and then the original packet payload into your own VM. Then allocate
> your own NDIS buffer and packet descriptor and send that instead of the
> original.
>
>
>
> If you get more experiance with the idea behind NDIS buffer descriptors and
> memory descriptor lists you may see a method that involves less copying.
>
>
>
> Thomas F. Divine
>
>
>
>
>
> I will greatly appreciate any help in this regard. Thanks in advance.
>
>
>
> Best Regards,
>
> -Niranjan
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com

Thanks for the reply.

Do I need to allocate a new packet or just a new NDIS buffer. I was thinking of allocating a new NDIS buffer which can hold the new intermediary layer header information and then link this buffer to the head of the buffer-descriptor chain attached to a packet descriptor (using NdisChainBufferAtFront). This will not require unnecessary copy of original packet payload. Please let me know whether this is correct design or not.

Best Regards,
-Niranjan
“Thomas F. Divine” wrote in message news:xxxxx@ntdev…

“Niranjan” wrote in message news:xxxxx@ntdev…
Hi,

I am designing a new layer between TCP/IP and MAC layer using NDIS intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer has its own header and few control packets. I wanted to know whether it is possible to change the packet structure in this intermediate driver (PassThru drivers) or not. Also, whether this is a good design or not. If not, what are the other ways in which I can implement this middle layer between TCP/IP and MAC layer?

NDIS IM drivers seem to be a good choice.

I am confused because at many places, it suggests not to modify packet data virtual memory or buffer descriptor at the intermediate drivers if you haven’t allocated it.

This is not a suggestion. If you didn’t allocate the VM, then you should not change or modify it. You must invent a method that inserts your header without modifying other VM or other NDIS buffer descriptors.

The simplest approach is to allocate your own VM sufficiently large to accomodate the original packet plus your header. Copy the MAC header, your header and then the original packet payload into your own VM. Then allocate your own NDIS buffer and packet descriptor and send that instead of the original.

If you get more experiance with the idea behind NDIS buffer descriptors and memory descriptor lists you may see a method that involves less copying.

Thomas F. Divine

I will greatly appreciate any help in this regard. Thanks in advance.

Best Regards,

-Niranjan

There are three cases of updating/chaging a NDIS packet:

  1. Updating NDIS packet data without any change in the length of the packet.

  2. Updating NDIS packet data where the length of the packet changes

  3. Adding a entirely new header at the beginning of the received NDIS packet
    (no changes in the existing packet data, just addition of few more bytes)

For case 1, I am not sure whether we need to allocate a new NDIS packet or
we can just change the data in the NDIS buffer as it doesn’t require any
memory allocation.

For case 2, we need to allocate a new NDIS packet.

But for case 3, do I need to allocate a new packet or just a new NDIS
buffer. I was thinking of allocating a new NDIS buffer which can hold the
new intermediary layer header information and then link this buffer to the
head of the buffer-descriptor chain attached to a packet descriptor (using
NdisChainBufferAtFront). This will not require unnecessary copy of original
packet payload. Please let me know whether this is correct design or not.

Best Regards,

-Niranjan

“Beyers Cronje” wrote in message news:xxxxx@ntdev…
Hi Thomas,

I was under the impression that one can change data allocated by
tcp/ip, e.g. changing the destination IP address, without having to
allocate your own packet and buffer descriptors. Am I wrong in this ?
Obviously this involves handling checksum and TCP task offload issues.

On the other hand, adding additional data such as Niranjan described
requires the copy and allocation of your own packet and buffer
descriptors ?

Regards

Beyers Cronje

On 8/16/05, Thomas F. Divine wrote:
>
>
>
> “Niranjan” wrote in message news:xxxxx@ntdev…
>
>
> Hi,
>
>
>
> I am designing a new layer between TCP/IP and MAC layer using NDIS
> intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer
has
> its own header and few control packets. I wanted to know whether it is
> possible to change the packet structure in this intermediate driver
> (PassThru drivers) or not. Also, whether this is a good design or not. If
> not, what are the other ways in which I can implement this middle layer
> between TCP/IP and MAC layer?
>
>
>
> NDIS IM drivers seem to be a good choice.
>
>
>
>
> I am confused because at many places, it suggests not to modify packet
data
> virtual memory or buffer descriptor at the intermediate drivers if you
> haven’t allocated it.
>
>
>
> This is not a suggestion. If you didn’t allocate the VM, then you should
not
> change or modify it. You must invent a method that inserts your header
> without modifying other VM or other NDIS buffer descriptors.
>
>
>
> The simplest approach is to allocate your own VM sufficiently large to
> accomodate the original packet plus your header. Copy the MAC header, your
> header and then the original packet payload into your own VM. Then
allocate
> your own NDIS buffer and packet descriptor and send that instead of the
> original.
>
>
>
> If you get more experiance with the idea behind NDIS buffer descriptors
and
> memory descriptor lists you may see a method that involves less copying.
>
>
>
> Thomas F. Divine
>
>
>
>
>
> I will greatly appreciate any help in this regard. Thanks in advance.
>
>
>
> Best Regards,
>
> -Niranjan
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com

Hi Niranjan,

First off let me just state I’m also new to NDIS so take my comments
at face value.

For case 1 you can change the data without allocating new packet
(given that the packet has free stack locations) or buffer
descriptors.

For case 3 I can’t imagine why one can’t chain your buffer to the
packet and have your sendcomplete handler recycle and unchain your
buffer. Issues with tcp task offload should be handled.

Beyers

On 8/17/05, Niranjan wrote:
> There are three cases of updating/chaging a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
> packet.
>
> 2. Updating NDIS packet data where the length of the packet changes
>
> 3. Adding a entirely new header at the beginning of the received NDIS
> packet
> (no changes in the existing packet data, just addition of few more
> bytes)
>
> For case 1, I am not sure whether we need to allocate a new NDIS packet
> or
> we can just change the data in the NDIS buffer as it doesn’t require any
> memory allocation.
>
> For case 2, we need to allocate a new NDIS packet.
>
> But for case 3, do I need to allocate a new packet or just a new NDIS
> buffer. I was thinking of allocating a new NDIS buffer which can hold
> the
> new intermediary layer header information and then link this buffer to
> the
> head of the buffer-descriptor chain attached to a packet descriptor
> (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
> original
> packet payload. Please let me know whether this is correct design or
> not.
>
> Best Regards,
>
> -Niranjan
>
>
>
> “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> Hi Thomas,
>
> I was under the impression that one can change data allocated by
> tcp/ip, e.g. changing the destination IP address, without having to
> allocate your own packet and buffer descriptors. Am I wrong in this ?
> Obviously this involves handling checksum and TCP task offload issues.
>
> On the other hand, adding additional data such as Niranjan described
> requires the copy and allocation of your own packet and buffer
> descriptors ?
>
> Regards
>
> Beyers Cronje
>
> On 8/16/05, Thomas F. Divine wrote:
> >
> >
> >
> > “Niranjan” wrote in message
> news:xxxxx@ntdev…
> >
> >
> > Hi,
> >
> >
> >
> > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> layer
> has
> > its own header and few control packets. I wanted to know whether it is
> > possible to change the packet structure in this intermediate driver
> > (PassThru drivers) or not. Also, whether this is a good design or not.
> If
> > not, what are the other ways in which I can implement this middle
> layer
> > between TCP/IP and MAC layer?
> >
> >
> >
> > NDIS IM drivers seem to be a good choice.
> >
> >
> >
> >
> > I am confused because at many places, it suggests not to modify packet
> data
> > virtual memory or buffer descriptor at the intermediate drivers if you
> > haven’t allocated it.
> >
> >
> >
> > This is not a suggestion. If you didn’t allocate the VM, then you
> should
> not
> > change or modify it. You must invent a method that inserts your header
> > without modifying other VM or other NDIS buffer descriptors.
> >
> >
> >
> > The simplest approach is to allocate your own VM sufficiently large to
> > accomodate the original packet plus your header. Copy the MAC header,
> your
> > header and then the original packet payload into your own VM. Then
> allocate
> > your own NDIS buffer and packet descriptor and send that instead of
> the
> > original.
> >
> >
> >
> > If you get more experiance with the idea behind NDIS buffer
> descriptors
> and
> > memory descriptor lists you may see a method that involves less
> copying.
> >
> >
> >
> > Thomas F. Divine
> >
> >
> >
> >
> >
> > I will greatly appreciate any help in this regard. Thanks in advance.
> >
> >
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag
> argument:
> ‘’
> > To unsubscribe send a blank email to
> > xxxxx@lists.osr.com
>
>

“Niranjan” wrote in message news:xxxxx@ntdev…
> There are three cases of updating/chaging a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
> packet.
>
> 2. Updating NDIS packet data where the length of the packet changes
>
> 3. Adding a entirely new header at the beginning of the received NDIS
> packet
> (no changes in the existing packet data, just addition of few more bytes)
>
> For case 1, I am not sure whether we need to allocate a new NDIS packet or
> we can just change the data in the NDIS buffer as it doesn’t require any
> memory allocation.
>
> For case 2, we need to allocate a new NDIS packet.
>
> But for case 3, do I need to allocate a new packet or just a new NDIS
> buffer. I was thinking of allocating a new NDIS buffer which can hold the
> new intermediary layer header information and then link this buffer to the
> head of the buffer-descriptor chain attached to a packet descriptor (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
> original
> packet payload. Please let me know whether this is correct design or not.
>
> Best Regards,
>
> -Niranjan
>

“Niranjan” wrote in message
news:%23v4RqC0oFHA.320@TK2MSFTNGP09.phx.gbl…
> Thanks for the reply.
>
> I can think of three cases when we talk about updating a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
> packet.
> 2. Updating NDIS packet data where the length of the packet changes
> 3. Adding a entirely new header at the beginning of the received NDIS
> packet
>
> For case 1 and 2, you need to allocate a new NDIS packet. But for case 3,
> do
> I need to allocate a new packet or just a new NDIS buffer. I was thinking
> of
> allocating a new NDIS buffer which can hold the new intermediary layer
> header information and then link this buffer to the head of the
> buffer-descriptor chain attached to a packet descriptor (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
> original
> packet payload. Please let me know whether this is correct design or not.
>
> Best Regards,
> -Niranjan
>
It is always exciting to think of complicated ways to attempt to improve
performance. The best rule to follow if you are going to change any data is
to make the changes in virtual memory that you allocated yourself and be
done with it. Forget about trying interesting ways to chain the original VM
into your own.

Here is an example of the kind of problem you will encounter if you get too
creative.

If you are modifying a packet that is being received it is easy to overlook
this requirement from the DDK:

“The first buffer in the packet must contain the amount of data specified by
the lookahead setting plus the data size of the MAC header. The miniport
driver receives the lookahead setting in a set to OID_GEN_CURRENT_LOOKAHEAD.
If the total packet size including the MAC header is less than the lookahead
setting plus the MAC header size, the first buffer must contain the entire
packet.”

(I certainly agree with you that this “requirement” seems more like a “bug
workaround”, but that’s beside the point…).

So, in case 3.) above you can’t build a chain consisting of the MAC header,
your new header and then the original payload. This applies to receive only,
of course.

You might be able to use the approach you are considering on send.

In the case of data modifying data being sent, I think that you should
consider VM that you did not allocate yourself to be read-only. It isn’t
actually be read-only, of course, but it should be. In your filter driver
you have absolutely no idea where the VM being sent originated. In some
cases some elements of the send MDL chain may actually be mapped from a user
application. So, modifying data in the original VM is actually modifying
data in a higher-level protocol or perhaps even in user-space.

There is, of course, the approach that I can “temporarily” modify someone
else’s memory - as long as it is restored to it’s original state when you’re
done with it. Change it and then undo the changes. BUT how are you going to
be sure that your changes don’t have an unintended side-effect. Sounds a
little too complicated for me.

If you can find a comment in the DDK that says that you can modify the
packet VM that you did not allocate, then go for it. Enjoy yourself.

Thomas F. Divine, Windows DDK MVP
http://www.pcausa.com

Niranjan,

If you are going to copy the data, you will need to allocate a new
NDIS_PACKET. In fact, most of the time your IM driver passes a received
NDIS_PACKT ‘up’ or a sent NDIS_PACKET ‘down’ it will need to allocate a new
packet. The exception is if you have used the “Packet Stack” mechanism.
Even with “packet stacking” you will need to deal with the possibility that
the packet stack is exhausted and allocate a new NDIS_PACKET anyway.

I recommend you read the DDK section on packet handling for IM drivers very
carefully and take advantage of the resources at www.ndis.com /
www.pcausa.com which address packet handling.

Now your specific scenarios:

  1. You mean changing the data in the buffer right? You must allocate a new
    buffer and a new NDIS_PACKET to reference the NDIS_BUFFER chain (MDL chain).
    Essentially, you must copy the packet. It may be possible to use a packet
    stack approach here and avoid the new NDIS_PACKET but frankly I have not
    ever tried and I don’t recommend it.

  2. Same as above. You changed the data and the ‘description’ of the
    buffers.

  3. If a packet stack is available, you can add the new header although in
    practice the benefit of *not* performing the copy may be thwarted by other
    rules you must follow. The NDIS_PACKET, most particularly the NDIS_BUFFER
    chain, must be restored to the original state before you can return the
    packet to the owner (the protocol that sent it or the NIC that).

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Niranjan
Sent: Wednesday, August 17, 2005 11:45 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header

There are three cases of updating/chaging a NDIS packet:

  1. Updating NDIS packet data without any change in the length of the packet.

  2. Updating NDIS packet data where the length of the packet changes

  3. Adding a entirely new header at the beginning of the received NDIS packet
    (no changes in the existing packet data, just addition of few more bytes)

For case 1, I am not sure whether we need to allocate a new NDIS packet or
we can just change the data in the NDIS buffer as it doesn’t require any
memory allocation.

For case 2, we need to allocate a new NDIS packet.

But for case 3, do I need to allocate a new packet or just a new NDIS
buffer. I was thinking of allocating a new NDIS buffer which can hold the
new intermediary layer header information and then link this buffer to the
head of the buffer-descriptor chain attached to a packet descriptor (using
NdisChainBufferAtFront). This will not require unnecessary copy of original
packet payload. Please let me know whether this is correct design or not.

Best Regards,

-Niranjan

“Beyers Cronje” wrote in message news:xxxxx@ntdev…
Hi Thomas,

I was under the impression that one can change data allocated by
tcp/ip, e.g. changing the destination IP address, without having to
allocate your own packet and buffer descriptors. Am I wrong in this ?
Obviously this involves handling checksum and TCP task offload issues.

On the other hand, adding additional data such as Niranjan described
requires the copy and allocation of your own packet and buffer
descriptors ?

Regards

Beyers Cronje

On 8/16/05, Thomas F. Divine wrote:
>
>
>
> “Niranjan” wrote in message news:xxxxx@ntdev…
>
>
> Hi,
>
>
>
> I am designing a new layer between TCP/IP and MAC layer using NDIS
> intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer
has
> its own header and few control packets. I wanted to know whether it is
> possible to change the packet structure in this intermediate driver
> (PassThru drivers) or not. Also, whether this is a good design or not. If
> not, what are the other ways in which I can implement this middle layer
> between TCP/IP and MAC layer?
>
>
>
> NDIS IM drivers seem to be a good choice.
>
>
>
>
> I am confused because at many places, it suggests not to modify packet
data
> virtual memory or buffer descriptor at the intermediate drivers if you
> haven’t allocated it.
>
>
>
> This is not a suggestion. If you didn’t allocate the VM, then you should
not
> change or modify it. You must invent a method that inserts your header
> without modifying other VM or other NDIS buffer descriptors.
>
>
>
> The simplest approach is to allocate your own VM sufficiently large to
> accomodate the original packet plus your header. Copy the MAC header, your
> header and then the original packet payload into your own VM. Then
allocate
> your own NDIS buffer and packet descriptor and send that instead of the
> original.
>
>
>
> If you get more experiance with the idea behind NDIS buffer descriptors
and
> memory descriptor lists you may see a method that involves less copying.
>
>
>
> Thomas F. Divine
>
>
>
>
>
> I will greatly appreciate any help in this regard. Thanks in advance.
>
>
>
> Best Regards,
>
> -Niranjan
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@msn.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks for the great explanation.

I just want to make the explanation for case 3 more clearer to me. Adding
new buffer (for the intermediary layer header) in front of the chains of
buffer will work for the send operation. I didn’t understand why it will not
work for receive operation. Because, when the intermediary layer receives
the packet, it will strip off the buffer corresponding to its own layer
header and then will forward it to the protocol driver. I will greatly
appreciate if you can explain the problem related to lookahead settings in
more details and whether this can be solved in some way or not?

I have another basic question related to normal packet processing in the
TCP/IP stack. Does each layer (having its own header and control packets) in
the stack creates a new packet (using NdisAllocatePacket and
NdisAllocateBuffer) whenever it receives packet from the upper layer to be
forwarded to the lower layer or from the lower layer to be forwarded to the
upper layer? So, if there are 4 layers, there are 4 times the packet is
created and freed at each layer.

Best Regards,
-Niranjan

“Thomas F. Divine” wrote in message news:xxxxx@ntdev…
>
> “Niranjan” wrote in message news:xxxxx@ntdev…
> > There are three cases of updating/chaging a NDIS packet:
> >
> > 1. Updating NDIS packet data without any change in the length of the
> > packet.
> >
> > 2. Updating NDIS packet data where the length of the packet changes
> >
> > 3. Adding a entirely new header at the beginning of the received NDIS
> > packet
> > (no changes in the existing packet data, just addition of few more
bytes)
> >
> > For case 1, I am not sure whether we need to allocate a new NDIS packet
or
> > we can just change the data in the NDIS buffer as it doesn’t require any
> > memory allocation.
> >
> > For case 2, we need to allocate a new NDIS packet.
> >
> > But for case 3, do I need to allocate a new packet or just a new NDIS
> > buffer. I was thinking of allocating a new NDIS buffer which can hold
the
> > new intermediary layer header information and then link this buffer to
the
> > head of the buffer-descriptor chain attached to a packet descriptor
(using
> > NdisChainBufferAtFront). This will not require unnecessary copy of
> > original
> > packet payload. Please let me know whether this is correct design or
not.
> >
> > Best Regards,
> >
> > -Niranjan
> >
>
> “Niranjan” wrote in message
> news:%23v4RqC0oFHA.320@TK2MSFTNGP09.phx.gbl…
> > Thanks for the reply.
> >
> > I can think of three cases when we talk about updating a NDIS packet:
> >
> > 1. Updating NDIS packet data without any change in the length of the
> > packet.
> > 2. Updating NDIS packet data where the length of the packet changes
> > 3. Adding a entirely new header at the beginning of the received NDIS
> > packet
> >
> > For case 1 and 2, you need to allocate a new NDIS packet. But for case
3,
> > do
> > I need to allocate a new packet or just a new NDIS buffer. I was
thinking
> > of
> > allocating a new NDIS buffer which can hold the new intermediary layer
> > header information and then link this buffer to the head of the
> > buffer-descriptor chain attached to a packet descriptor (using
> > NdisChainBufferAtFront). This will not require unnecessary copy of
> > original
> > packet payload. Please let me know whether this is correct design or
not.
> >
> > Best Regards,
> > -Niranjan
> >
> It is always exciting to think of complicated ways to attempt to improve
> performance. The best rule to follow if you are going to change any data
is
> to make the changes in virtual memory that you allocated yourself and be
> done with it. Forget about trying interesting ways to chain the original
VM
> into your own.
>
> Here is an example of the kind of problem you will encounter if you get
too
> creative.
>
> If you are modifying a packet that is being received it is easy to
overlook
> this requirement from the DDK:
>
> “The first buffer in the packet must contain the amount of data specified
by
> the lookahead setting plus the data size of the MAC header. The miniport
> driver receives the lookahead setting in a set to
OID_GEN_CURRENT_LOOKAHEAD.
> If the total packet size including the MAC header is less than the
lookahead
> setting plus the MAC header size, the first buffer must contain the entire
> packet.”
>
> (I certainly agree with you that this “requirement” seems more like a “bug
> workaround”, but that’s beside the point…).
>
> So, in case 3.) above you can’t build a chain consisting of the MAC
header,
> your new header and then the original payload. This applies to receive
only,
> of course.
>
> You might be able to use the approach you are considering on send.
>
> In the case of data modifying data being sent, I think that you should
> consider VM that you did not allocate yourself to be read-only. It isn’t
> actually be read-only, of course, but it should be. In your filter driver
> you have absolutely no idea where the VM being sent originated. In some
> cases some elements of the send MDL chain may actually be mapped from a
user
> application. So, modifying data in the original VM is actually modifying
> data in a higher-level protocol or perhaps even in user-space.
>
> There is, of course, the approach that I can “temporarily” modify someone
> else’s memory - as long as it is restored to it’s original state when
you’re
> done with it. Change it and then undo the changes. BUT how are you going
to
> be sure that your changes don’t have an unintended side-effect. Sounds a
> little too complicated for me.
>
> If you can find a comment in the DDK that says that you can modify the
> packet VM that you did not allocate, then go for it. Enjoy yourself.
>
> Thomas F. Divine, Windows DDK MVP
> http://www.pcausa.com
>
>
>
>

Does IM driver copies the data or it just allocate and initialize a new
packet descriptor and then chain the existing buffer descriptor (received
from the upper layer having packet data) to the new packet descriptor. IM
driver doesn’t allocate any buffer (using NdisAllocateBuffer) to copy the
data received from the upper layer or lower layer.

Also, when you say to create a new packet at intermediary layer, it means to
create NDIS buffer first (using NdisAllocateBuffer from the buffer pool) and
then set the private field of NDIS packet with this buffer, not just
allocating and initializing a new packet descriptor.

I am new to NDIS programming, so these question might be very naive. Thanks
for all your help.

Best Regards,
-Niranjan

“David R. Cattley” wrote in message news:xxxxx@ntdev…
> Niranjan,
>
> If you are going to copy the data, you will need to allocate a new
> NDIS_PACKET. In fact, most of the time your IM driver passes a received
> NDIS_PACKT ‘up’ or a sent NDIS_PACKET ‘down’ it will need to allocate a
new
> packet. The exception is if you have used the “Packet Stack” mechanism.
> Even with “packet stacking” you will need to deal with the possibility
that
> the packet stack is exhausted and allocate a new NDIS_PACKET anyway.
>
> I recommend you read the DDK section on packet handling for IM drivers
very
> carefully and take advantage of the resources at www.ndis.com /
> www.pcausa.com which address packet handling.
>
> Now your specific scenarios:
>
> 1. You mean changing the data in the buffer right? You must allocate a
new
> buffer and a new NDIS_PACKET to reference the NDIS_BUFFER chain (MDL
chain).
> Essentially, you must copy the packet. It may be possible to use a packet
> stack approach here and avoid the new NDIS_PACKET but frankly I have not
> ever tried and I don’t recommend it.
>
> 2. Same as above. You changed the data and the ‘description’ of the
> buffers.
>
> 3. If a packet stack is available, you can add the new header although in
> practice the benefit of not performing the copy may be thwarted by other
> rules you must follow. The NDIS_PACKET, most particularly the NDIS_BUFFER
> chain, must be restored to the original state before you can return the
> packet to the owner (the protocol that sent it or the NIC that).
>
>
> Good Luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Development
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Niranjan
> Sent: Wednesday, August 17, 2005 11:45 AM
> To: Windows System Software Devs Interest List
> Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
> There are three cases of updating/chaging a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
packet.
>
> 2. Updating NDIS packet data where the length of the packet changes
>
> 3. Adding a entirely new header at the beginning of the received NDIS
packet
> (no changes in the existing packet data, just addition of few more bytes)
>
> For case 1, I am not sure whether we need to allocate a new NDIS packet or
> we can just change the data in the NDIS buffer as it doesn’t require any
> memory allocation.
>
> For case 2, we need to allocate a new NDIS packet.
>
> But for case 3, do I need to allocate a new packet or just a new NDIS
> buffer. I was thinking of allocating a new NDIS buffer which can hold the
> new intermediary layer header information and then link this buffer to the
> head of the buffer-descriptor chain attached to a packet descriptor (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
original
> packet payload. Please let me know whether this is correct design or not.
>
> Best Regards,
>
> -Niranjan
>
>
>
> “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> Hi Thomas,
>
> I was under the impression that one can change data allocated by
> tcp/ip, e.g. changing the destination IP address, without having to
> allocate your own packet and buffer descriptors. Am I wrong in this ?
> Obviously this involves handling checksum and TCP task offload issues.
>
> On the other hand, adding additional data such as Niranjan described
> requires the copy and allocation of your own packet and buffer
> descriptors ?
>
> Regards
>
> Beyers Cronje
>
> On 8/16/05, Thomas F. Divine wrote:
> >
> >
> >
> > “Niranjan” wrote in message
news:xxxxx@ntdev…
> >
> >
> > Hi,
> >
> >
> >
> > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer
> has
> > its own header and few control packets. I wanted to know whether it is
> > possible to change the packet structure in this intermediate driver
> > (PassThru drivers) or not. Also, whether this is a good design or not.
If
> > not, what are the other ways in which I can implement this middle layer
> > between TCP/IP and MAC layer?
> >
> >
> >
> > NDIS IM drivers seem to be a good choice.
> >
> >
> >
> >
> > I am confused because at many places, it suggests not to modify packet
> data
> > virtual memory or buffer descriptor at the intermediate drivers if you
> > haven’t allocated it.
> >
> >
> >
> > This is not a suggestion. If you didn’t allocate the VM, then you should
> not
> > change or modify it. You must invent a method that inserts your header
> > without modifying other VM or other NDIS buffer descriptors.
> >
> >
> >
> > The simplest approach is to allocate your own VM sufficiently large to
> > accomodate the original packet plus your header. Copy the MAC header,
your
> > header and then the original packet payload into your own VM. Then
> allocate
> > your own NDIS buffer and packet descriptor and send that instead of the
> > original.
> >
> >
> >
> > If you get more experiance with the idea behind NDIS buffer descriptors
> and
> > memory descriptor lists you may see a method that involves less copying.
> >
> >
> >
> > Thomas F. Divine
> >
> >
> >
> >
> >
> > I will greatly appreciate any help in this regard. Thanks in advance.
> >
> >
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag argument:
> ‘’
> > To unsubscribe send a blank email to
> > xxxxx@lists.osr.com
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@msn.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

“Niranjan” wrote in message news:xxxxx@ntdev…
> Thanks for the great explanation.
>
> I just want to make the explanation for case 3 more clearer to me. Adding
> new buffer (for the intermediary layer header) in front of the chains of
> buffer will work for the send operation.
> I didn’t understand why it will not
> work for receive operation. Because, when the intermediary layer receives
> the packet, it will strip off the buffer corresponding to its own layer
> header and then will forward it to the protocol driver. I will greatly
> appreciate if you can explain the problem related to lookahead settings in
> more details and whether this can be solved in some way or not?
>
If you can satisfy this DDK requirement when indicating receives:

“The first buffer in the packet must contain the amount of data specified by
the lookahead setting plus the data size of the MAC header. The miniport
driver receives the lookahead setting in a set to OID_GEN_CURRENT_LOOKAHEAD.
If the total packet size including the MAC header is less than the lookahead
setting plus the MAC header size, the first buffer must contain the entire
packet.”

When you strip off your header, then you are all set.

> I have another basic question related to normal packet processing in the
> TCP/IP stack. Does each layer (having its own header and control packets)
> in
> the stack creates a new packet (using NdisAllocatePacket and
> NdisAllocateBuffer) whenever it receives packet from the upper layer to be
> forwarded to the lower layer or from the lower layer to be forwarded to
> the
> upper layer? So, if there are 4 layers, there are 4 times the packet is
> created and freed at each layer.

Do not make any assumption whatsoever about the organization of NDIS buffer
descriptors passed into your NDIS IM driver during send or receive (except
for those requireemnts explicitly set forth in the DDK). You can, of course,
build software that works on your machine based on your observations of NDIS
buffer organization. If you distribute such a driver you can count on it to
fail when a user installs an additional component (perhaps another NDIS IM
driver…) that may coalease buffers or organize them in some other fashion
that you weren’t expecting.

The “NDIS Packet Discussion” on NDIS.com may help. See the URL:

http://www.ndis.com/papers/default.htm

Good luck,

Thomas F. Divine, Windows DDK MVP
http://www.pcausa.com

>
> Best Regards,
> -Niranjan
>
> “Thomas F. Divine” wrote in message
> news:xxxxx@ntdev…
>>
>> “Niranjan” wrote in message
>> news:xxxxx@ntdev…
>> > There are three cases of updating/chaging a NDIS packet:
>> >
>> > 1. Updating NDIS packet data without any change in the length of the
>> > packet.
>> >
>> > 2. Updating NDIS packet data where the length of the packet changes
>> >
>> > 3. Adding a entirely new header at the beginning of the received NDIS
>> > packet
>> > (no changes in the existing packet data, just addition of few more
> bytes)
>> >
>> > For case 1, I am not sure whether we need to allocate a new NDIS packet
> or
>> > we can just change the data in the NDIS buffer as it doesn’t require
>> > any
>> > memory allocation.
>> >
>> > For case 2, we need to allocate a new NDIS packet.
>> >
>> > But for case 3, do I need to allocate a new packet or just a new NDIS
>> > buffer. I was thinking of allocating a new NDIS buffer which can hold
> the
>> > new intermediary layer header information and then link this buffer to
> the
>> > head of the buffer-descriptor chain attached to a packet descriptor
> (using
>> > NdisChainBufferAtFront). This will not require unnecessary copy of
>> > original
>> > packet payload. Please let me know whether this is correct design or
> not.
>> >
>> > Best Regards,
>> >
>> > -Niranjan
>> >
>>
>> “Niranjan” wrote in message
>> news:%23v4RqC0oFHA.320@TK2MSFTNGP09.phx.gbl…
>> > Thanks for the reply.
>> >
>> > I can think of three cases when we talk about updating a NDIS packet:
>> >
>> > 1. Updating NDIS packet data without any change in the length of the
>> > packet.
>> > 2. Updating NDIS packet data where the length of the packet changes
>> > 3. Adding a entirely new header at the beginning of the received NDIS
>> > packet
>> >
>> > For case 1 and 2, you need to allocate a new NDIS packet. But for case
> 3,
>> > do
>> > I need to allocate a new packet or just a new NDIS buffer. I was
> thinking
>> > of
>> > allocating a new NDIS buffer which can hold the new intermediary layer
>> > header information and then link this buffer to the head of the
>> > buffer-descriptor chain attached to a packet descriptor (using
>> > NdisChainBufferAtFront). This will not require unnecessary copy of
>> > original
>> > packet payload. Please let me know whether this is correct design or
> not.
>> >
>> > Best Regards,
>> > -Niranjan
>> >
>> It is always exciting to think of complicated ways to attempt to improve
>> performance. The best rule to follow if you are going to change any data
> is
>> to make the changes in virtual memory that you allocated yourself and be
>> done with it. Forget about trying interesting ways to chain the original
> VM
>> into your own.
>>
>> Here is an example of the kind of problem you will encounter if you get
> too
>> creative.
>>
>> If you are modifying a packet that is being received it is easy to
> overlook
>> this requirement from the DDK:
>>
>> “The first buffer in the packet must contain the amount of data specified
> by
>> the lookahead setting plus the data size of the MAC header. The miniport
>> driver receives the lookahead setting in a set to
> OID_GEN_CURRENT_LOOKAHEAD.
>> If the total packet size including the MAC header is less than the
> lookahead
>> setting plus the MAC header size, the first buffer must contain the
>> entire
>> packet.”
>>
>> (I certainly agree with you that this “requirement” seems more like a
>> “bug
>> workaround”, but that’s beside the point…).
>>
>> So, in case 3.) above you can’t build a chain consisting of the MAC
> header,
>> your new header and then the original payload. This applies to receive
> only,
>> of course.
>>
>> You might be able to use the approach you are considering on send.
>>
>> In the case of data modifying data being sent, I think that you should
>> consider VM that you did not allocate yourself to be read-only. It isn’t
>> actually be read-only, of course, but it should be. In your filter driver
>> you have absolutely no idea where the VM being sent originated. In some
>> cases some elements of the send MDL chain may actually be mapped from a
> user
>> application. So, modifying data in the original VM is actually modifying
>> data in a higher-level protocol or perhaps even in user-space.
>>
>> There is, of course, the approach that I can “temporarily” modify someone
>> else’s memory - as long as it is restored to it’s original state when
> you’re
>> done with it. Change it and then undo the changes. BUT how are you going
> to
>> be sure that your changes don’t have an unintended side-effect. Sounds a
>> little too complicated for me.
>>
>> If you can find a comment in the DDK that says that you can modify the
>> packet VM that you did not allocate, then go for it. Enjoy yourself.
>>
>> Thomas F. Divine, Windows DDK MVP
>> http://www.pcausa.com
>>
>>
>>
>>
>
>
>

Niranjan,

Some IM drivers allocate an entirely new buffer (or list of buffers) from
non-paged pool, some may be clever enough (and architected to deal with)
simply ‘re-describing’ the portions of the existing packet buffer chain (MDL
chain) with new NDIS_BUFFERs to eliminate copying.

As Thomas has indicated, the receive path is often more restrictive than the
send path since minimum look-a-head must be maintained and ultimately
TCPIP.SYS expects that the first NDIS_BUFFER (MDL) contains the entire MAC
and IP (and TCP/UDP) header.

And yes, when I said create new packet that meant a new NDIS_PACKET (from a
packet pool managed by your driver), non-paged pool to actually store the
data, and NDIS_BUFFERs allocated by your driver to describe the NP-pool
buffer and chained to the NDIS_PACKET. Once you are going to copy the data,
trying to optimize away the allocation of an NDIS_PACKET is often ‘small
change’ compared to the overall operation and simply makes your driver much
harder to understand and debug.

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Niranjan
Sent: Wednesday, August 17, 2005 4:07 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Does IM driver copies the data or it just allocate and initialize a new
packet descriptor and then chain the existing buffer descriptor (received
from the upper layer having packet data) to the new packet descriptor. IM
driver doesn’t allocate any buffer (using NdisAllocateBuffer) to copy the
data received from the upper layer or lower layer.

Also, when you say to create a new packet at intermediary layer, it means to
create NDIS buffer first (using NdisAllocateBuffer from the buffer pool) and
then set the private field of NDIS packet with this buffer, not just
allocating and initializing a new packet descriptor.

I am new to NDIS programming, so these question might be very naive. Thanks
for all your help.

Best Regards,
-Niranjan

“David R. Cattley” wrote in message news:xxxxx@ntdev…
> Niranjan,
>
> If you are going to copy the data, you will need to allocate a new
> NDIS_PACKET. In fact, most of the time your IM driver passes a received
> NDIS_PACKT ‘up’ or a sent NDIS_PACKET ‘down’ it will need to allocate a
new
> packet. The exception is if you have used the “Packet Stack” mechanism.
> Even with “packet stacking” you will need to deal with the possibility
that
> the packet stack is exhausted and allocate a new NDIS_PACKET anyway.
>
> I recommend you read the DDK section on packet handling for IM drivers
very
> carefully and take advantage of the resources at www.ndis.com /
> www.pcausa.com which address packet handling.
>
> Now your specific scenarios:
>
> 1. You mean changing the data in the buffer right? You must allocate a
new
> buffer and a new NDIS_PACKET to reference the NDIS_BUFFER chain (MDL
chain).
> Essentially, you must copy the packet. It may be possible to use a packet
> stack approach here and avoid the new NDIS_PACKET but frankly I have not
> ever tried and I don’t recommend it.
>
> 2. Same as above. You changed the data and the ‘description’ of the
> buffers.
>
> 3. If a packet stack is available, you can add the new header although in
> practice the benefit of not performing the copy may be thwarted by other
> rules you must follow. The NDIS_PACKET, most particularly the NDIS_BUFFER
> chain, must be restored to the original state before you can return the
> packet to the owner (the protocol that sent it or the NIC that).
>
>
> Good Luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Development
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Niranjan
> Sent: Wednesday, August 17, 2005 11:45 AM
> To: Windows System Software Devs Interest List
> Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
> There are three cases of updating/chaging a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
packet.
>
> 2. Updating NDIS packet data where the length of the packet changes
>
> 3. Adding a entirely new header at the beginning of the received NDIS
packet
> (no changes in the existing packet data, just addition of few more bytes)
>
> For case 1, I am not sure whether we need to allocate a new NDIS packet or
> we can just change the data in the NDIS buffer as it doesn’t require any
> memory allocation.
>
> For case 2, we need to allocate a new NDIS packet.
>
> But for case 3, do I need to allocate a new packet or just a new NDIS
> buffer. I was thinking of allocating a new NDIS buffer which can hold the
> new intermediary layer header information and then link this buffer to the
> head of the buffer-descriptor chain attached to a packet descriptor (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
original
> packet payload. Please let me know whether this is correct design or not.
>
> Best Regards,
>
> -Niranjan
>
>
>
> “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> Hi Thomas,
>
> I was under the impression that one can change data allocated by
> tcp/ip, e.g. changing the destination IP address, without having to
> allocate your own packet and buffer descriptors. Am I wrong in this ?
> Obviously this involves handling checksum and TCP task offload issues.
>
> On the other hand, adding additional data such as Niranjan described
> requires the copy and allocation of your own packet and buffer
> descriptors ?
>
> Regards
>
> Beyers Cronje
>
> On 8/16/05, Thomas F. Divine wrote:
> >
> >
> >
> > “Niranjan” wrote in message
news:xxxxx@ntdev…
> >
> >
> > Hi,
> >
> >
> >
> > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer
> has
> > its own header and few control packets. I wanted to know whether it is
> > possible to change the packet structure in this intermediate driver
> > (PassThru drivers) or not. Also, whether this is a good design or not.
If
> > not, what are the other ways in which I can implement this middle layer
> > between TCP/IP and MAC layer?
> >
> >
> >
> > NDIS IM drivers seem to be a good choice.
> >
> >
> >
> >
> > I am confused because at many places, it suggests not to modify packet
> data
> > virtual memory or buffer descriptor at the intermediate drivers if you
> > haven’t allocated it.
> >
> >
> >
> > This is not a suggestion. If you didn’t allocate the VM, then you should
> not
> > change or modify it. You must invent a method that inserts your header
> > without modifying other VM or other NDIS buffer descriptors.
> >
> >
> >
> > The simplest approach is to allocate your own VM sufficiently large to
> > accomodate the original packet plus your header. Copy the MAC header,
your
> > header and then the original packet payload into your own VM. Then
> allocate
> > your own NDIS buffer and packet descriptor and send that instead of the
> > original.
> >
> >
> >
> > If you get more experiance with the idea behind NDIS buffer descriptors
> and
> > memory descriptor lists you may see a method that involves less copying.
> >
> >
> >
> > Thomas F. Divine
> >
> >
> >
> >
> >
> > I will greatly appreciate any help in this regard. Thanks in advance.
> >
> >
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag argument:
> ‘’
> > To unsubscribe send a blank email to
> > xxxxx@lists.osr.com
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@msn.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@msn.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You always need to allocate NDIS_PACKET, even for a do-nothing transparent IM.

The only exception if you will use the NDIS 5.1 packet stacking feature, but if fails sometimes, and in these cases you must revert to good old “new NDIS_PACKET” path anyway.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: Niranjan
Newsgroups: ntdev
To: Windows System Software Devs Interest List
Sent: Wednesday, August 17, 2005 7:21 PM
Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Thanks for the reply.

Do I need to allocate a new packet or just a new NDIS buffer. I was thinking of allocating a new NDIS buffer which can hold the new intermediary layer header information and then link this buffer to the head of the buffer-descriptor chain attached to a packet descriptor (using NdisChainBufferAtFront). This will not require unnecessary copy of original packet payload. Please let me know whether this is correct design or not.

Best Regards,
-Niranjan
“Thomas F. Divine” wrote in message news:xxxxx@ntdev…

“Niranjan” wrote in message news:xxxxx@ntdev…
Hi,

I am designing a new layer between TCP/IP and MAC layer using NDIS intermediate drivers (PassThru samples) on Windows CE 5.0. The new layer has its own header and few control packets. I wanted to know whether it is possible to change the packet structure in this intermediate driver (PassThru drivers) or not. Also, whether this is a good design or not. If not, what are the other ways in which I can implement this middle layer between TCP/IP and MAC layer?

NDIS IM drivers seem to be a good choice.

I am confused because at many places, it suggests not to modify packet data virtual memory or buffer descriptor at the intermediate drivers if you haven’t allocated it.

This is not a suggestion. If you didn’t allocate the VM, then you should not change or modify it. You must invent a method that inserts your header without modifying other VM or other NDIS buffer descriptors.

The simplest approach is to allocate your own VM sufficiently large to accomodate the original packet plus your header. Copy the MAC header, your header and then the original packet payload into your own VM. Then allocate your own NDIS buffer and packet descriptor and send that instead of the original.

If you get more experiance with the idea behind NDIS buffer descriptors and memory descriptor lists you may see a method that involves less copying.

Thomas F. Divine

I will greatly appreciate any help in this regard. Thanks in advance.

Best Regards,

-Niranjan


Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

You cannot change the data “in place” in NDIS. You must allocate a new data
space + NDIS_BUFFER for the changed data.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Beyers Cronje”
To: “Windows System Software Devs Interest List”
Sent: Wednesday, August 17, 2005 8:55 PM
Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Hi Niranjan,

First off let me just state I’m also new to NDIS so take my comments
at face value.

For case 1 you can change the data without allocating new packet
(given that the packet has free stack locations) or buffer
descriptors.

For case 3 I can’t imagine why one can’t chain your buffer to the
packet and have your sendcomplete handler recycle and unchain your
buffer. Issues with tcp task offload should be handled.

Beyers

On 8/17/05, Niranjan wrote:
> There are three cases of updating/chaging a NDIS packet:
>
> 1. Updating NDIS packet data without any change in the length of the
> packet.
>
> 2. Updating NDIS packet data where the length of the packet changes
>
> 3. Adding a entirely new header at the beginning of the received NDIS
> packet
> (no changes in the existing packet data, just addition of few more
> bytes)
>
> For case 1, I am not sure whether we need to allocate a new NDIS packet
> or
> we can just change the data in the NDIS buffer as it doesn’t require any
> memory allocation.
>
> For case 2, we need to allocate a new NDIS packet.
>
> But for case 3, do I need to allocate a new packet or just a new NDIS
> buffer. I was thinking of allocating a new NDIS buffer which can hold
> the
> new intermediary layer header information and then link this buffer to
> the
> head of the buffer-descriptor chain attached to a packet descriptor
> (using
> NdisChainBufferAtFront). This will not require unnecessary copy of
> original
> packet payload. Please let me know whether this is correct design or
> not.
>
> Best Regards,
>
> -Niranjan
>
>
>
> “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> Hi Thomas,
>
> I was under the impression that one can change data allocated by
> tcp/ip, e.g. changing the destination IP address, without having to
> allocate your own packet and buffer descriptors. Am I wrong in this ?
> Obviously this involves handling checksum and TCP task offload issues.
>
> On the other hand, adding additional data such as Niranjan described
> requires the copy and allocation of your own packet and buffer
> descriptors ?
>
> Regards
>
> Beyers Cronje
>
> On 8/16/05, Thomas F. Divine wrote:
> >
> >
> >
> > “Niranjan” wrote in message
> news:xxxxx@ntdev…
> >
> >
> > Hi,
> >
> >
> >
> > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> layer
> has
> > its own header and few control packets. I wanted to know whether it is
> > possible to change the packet structure in this intermediate driver
> > (PassThru drivers) or not. Also, whether this is a good design or not.
> If
> > not, what are the other ways in which I can implement this middle
> layer
> > between TCP/IP and MAC layer?
> >
> >
> >
> > NDIS IM drivers seem to be a good choice.
> >
> >
> >
> >
> > I am confused because at many places, it suggests not to modify packet
> data
> > virtual memory or buffer descriptor at the intermediate drivers if you
> > haven’t allocated it.
> >
> >
> >
> > This is not a suggestion. If you didn’t allocate the VM, then you
> should
> not
> > change or modify it. You must invent a method that inserts your header
> > without modifying other VM or other NDIS buffer descriptors.
> >
> >
> >
> > The simplest approach is to allocate your own VM sufficiently large to
> > accomodate the original packet plus your header. Copy the MAC header,
> your
> > header and then the original packet payload into your own VM. Then
> allocate
> > your own NDIS buffer and packet descriptor and send that instead of
> the
> > original.
> >
> >
> >
> > If you get more experiance with the idea behind NDIS buffer
> descriptors
> and
> > memory descriptor lists you may see a method that involves less
> copying.
> >
> >
> >
> > Thomas F. Divine
> >
> >
> >
> >
> >
> > I will greatly appreciate any help in this regard. Thanks in advance.
> >
> >
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag
> argument:
> ‘’
> > To unsubscribe send a blank email to
> > xxxxx@lists.osr.com
>
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

> I have another basic question related to normal packet processing in the

TCP/IP stack. Does each layer (having its own header and control packets) in
the stack creates a new packet (using NdisAllocatePacket and
NdisAllocateBuffer) whenever it receives packet from the upper layer to be

Yes, but not NdisAllocateBuffer. If you do not change the data - you can reuse
the existing buffer chain.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Hi Maxim,

When Ip and Tcp checksum offloading is enabled isn’t the miniport or
intermediate driver doing exactly that, updating the data “in place” ?

I’m busy with an intermediate driver that updates the data including
checksums “in place” and all is working 100%.

Beyers

On 8/23/05, Maxim S. Shatskih wrote:
> You cannot change the data “in place” in NDIS. You must allocate a new data
> space + NDIS_BUFFER for the changed data.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> ----- Original Message -----
> From: “Beyers Cronje”
> To: “Windows System Software Devs Interest List”
> Sent: Wednesday, August 17, 2005 8:55 PM
> Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
>
> Hi Niranjan,
>
> First off let me just state I’m also new to NDIS so take my comments
> at face value.
>
> For case 1 you can change the data without allocating new packet
> (given that the packet has free stack locations) or buffer
> descriptors.
>
> For case 3 I can’t imagine why one can’t chain your buffer to the
> packet and have your sendcomplete handler recycle and unchain your
> buffer. Issues with tcp task offload should be handled.
>
> Beyers
>
> On 8/17/05, Niranjan wrote:
> > There are three cases of updating/chaging a NDIS packet:
> >
> > 1. Updating NDIS packet data without any change in the length of the
> > packet.
> >
> > 2. Updating NDIS packet data where the length of the packet changes
> >
> > 3. Adding a entirely new header at the beginning of the received NDIS
> > packet
> > (no changes in the existing packet data, just addition of few more
> > bytes)
> >
> > For case 1, I am not sure whether we need to allocate a new NDIS packet
> > or
> > we can just change the data in the NDIS buffer as it doesn’t require any
> > memory allocation.
> >
> > For case 2, we need to allocate a new NDIS packet.
> >
> > But for case 3, do I need to allocate a new packet or just a new NDIS
> > buffer. I was thinking of allocating a new NDIS buffer which can hold
> > the
> > new intermediary layer header information and then link this buffer to
> > the
> > head of the buffer-descriptor chain attached to a packet descriptor
> > (using
> > NdisChainBufferAtFront). This will not require unnecessary copy of
> > original
> > packet payload. Please let me know whether this is correct design or
> > not.
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> > “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> > Hi Thomas,
> >
> > I was under the impression that one can change data allocated by
> > tcp/ip, e.g. changing the destination IP address, without having to
> > allocate your own packet and buffer descriptors. Am I wrong in this ?
> > Obviously this involves handling checksum and TCP task offload issues.
> >
> > On the other hand, adding additional data such as Niranjan described
> > requires the copy and allocation of your own packet and buffer
> > descriptors ?
> >
> > Regards
> >
> > Beyers Cronje
> >
> > On 8/16/05, Thomas F. Divine wrote:
> > >
> > >
> > >
> > > “Niranjan” wrote in message
> > news:xxxxx@ntdev…
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> > layer
> > has
> > > its own header and few control packets. I wanted to know whether it is
> > > possible to change the packet structure in this intermediate driver
> > > (PassThru drivers) or not. Also, whether this is a good design or not.
> > If
> > > not, what are the other ways in which I can implement this middle
> > layer
> > > between TCP/IP and MAC layer?
> > >
> > >
> > >
> > > NDIS IM drivers seem to be a good choice.
> > >
> > >
> > >
> > >
> > > I am confused because at many places, it suggests not to modify packet
> > data
> > > virtual memory or buffer descriptor at the intermediate drivers if you
> > > haven’t allocated it.
> > >
> > >
> > >
> > > This is not a suggestion. If you didn’t allocate the VM, then you
> > should
> > not
> > > change or modify it. You must invent a method that inserts your header
> > > without modifying other VM or other NDIS buffer descriptors.
> > >
> > >
> > >
> > > The simplest approach is to allocate your own VM sufficiently large to
> > > accomodate the original packet plus your header. Copy the MAC header,
> > your
> > > header and then the original packet payload into your own VM. Then
> > allocate
> > > your own NDIS buffer and packet descriptor and send that instead of
> > the
> > > original.
> > >
> > >
> > >
> > > If you get more experiance with the idea behind NDIS buffer
> > descriptors
> > and
> > > memory descriptor lists you may see a method that involves less
> > copying.
> > >
> > >
> > >
> > > Thomas F. Divine
> > >
> > >
> > >
> > >
> > >
> > > I will greatly appreciate any help in this regard. Thanks in advance.
> > >
> > >
> > >
> > > Best Regards,
> > >
> > > -Niranjan
> > >
> > >
> > >
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: unknown lmsubst tag
> > argument:
> > ‘’
> > > To unsubscribe send a blank email to
> > > xxxxx@lists.osr.com
> >
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Good that your driver works, but the approach is wrong and can, for
instance, cause interops.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Beyers Cronje”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, August 23, 2005 5:13 PM
Subject: Re: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Hi Maxim,

When Ip and Tcp checksum offloading is enabled isn’t the miniport or
intermediate driver doing exactly that, updating the data “in place” ?

I’m busy with an intermediate driver that updates the data including
checksums “in place” and all is working 100%.

Beyers

On 8/23/05, Maxim S. Shatskih wrote:
> You cannot change the data “in place” in NDIS. You must allocate a new
data
> space + NDIS_BUFFER for the changed data.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> ----- Original Message -----
> From: “Beyers Cronje”
> To: “Windows System Software Devs Interest List”
> Sent: Wednesday, August 17, 2005 8:55 PM
> Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
>
> Hi Niranjan,
>
> First off let me just state I’m also new to NDIS so take my comments
> at face value.
>
> For case 1 you can change the data without allocating new packet
> (given that the packet has free stack locations) or buffer
> descriptors.
>
> For case 3 I can’t imagine why one can’t chain your buffer to the
> packet and have your sendcomplete handler recycle and unchain your
> buffer. Issues with tcp task offload should be handled.
>
> Beyers
>
> On 8/17/05, Niranjan wrote:
> > There are three cases of updating/chaging a NDIS packet:
> >
> > 1. Updating NDIS packet data without any change in the length of the
> > packet.
> >
> > 2. Updating NDIS packet data where the length of the packet changes
> >
> > 3. Adding a entirely new header at the beginning of the received NDIS
> > packet
> > (no changes in the existing packet data, just addition of few more
> > bytes)
> >
> > For case 1, I am not sure whether we need to allocate a new NDIS packet
> > or
> > we can just change the data in the NDIS buffer as it doesn’t require any
> > memory allocation.
> >
> > For case 2, we need to allocate a new NDIS packet.
> >
> > But for case 3, do I need to allocate a new packet or just a new NDIS
> > buffer. I was thinking of allocating a new NDIS buffer which can hold
> > the
> > new intermediary layer header information and then link this buffer to
> > the
> > head of the buffer-descriptor chain attached to a packet descriptor
> > (using
> > NdisChainBufferAtFront). This will not require unnecessary copy of
> > original
> > packet payload. Please let me know whether this is correct design or
> > not.
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> > “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> > Hi Thomas,
> >
> > I was under the impression that one can change data allocated by
> > tcp/ip, e.g. changing the destination IP address, without having to
> > allocate your own packet and buffer descriptors. Am I wrong in this ?
> > Obviously this involves handling checksum and TCP task offload issues.
> >
> > On the other hand, adding additional data such as Niranjan described
> > requires the copy and allocation of your own packet and buffer
> > descriptors ?
> >
> > Regards
> >
> > Beyers Cronje
> >
> > On 8/16/05, Thomas F. Divine wrote:
> > >
> > >
> > >
> > > “Niranjan” wrote in message
> > news:xxxxx@ntdev…
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> > layer
> > has
> > > its own header and few control packets. I wanted to know whether it is
> > > possible to change the packet structure in this intermediate driver
> > > (PassThru drivers) or not. Also, whether this is a good design or not.
> > If
> > > not, what are the other ways in which I can implement this middle
> > layer
> > > between TCP/IP and MAC layer?
> > >
> > >
> > >
> > > NDIS IM drivers seem to be a good choice.
> > >
> > >
> > >
> > >
> > > I am confused because at many places, it suggests not to modify packet
> > data
> > > virtual memory or buffer descriptor at the intermediate drivers if you
> > > haven’t allocated it.
> > >
> > >
> > >
> > > This is not a suggestion. If you didn’t allocate the VM, then you
> > should
> > not
> > > change or modify it. You must invent a method that inserts your header
> > > without modifying other VM or other NDIS buffer descriptors.
> > >
> > >
> > >
> > > The simplest approach is to allocate your own VM sufficiently large to
> > > accomodate the original packet plus your header. Copy the MAC header,
> > your
> > > header and then the original packet payload into your own VM. Then
> > allocate
> > > your own NDIS buffer and packet descriptor and send that instead of
> > the
> > > original.
> > >
> > >
> > >
> > > If you get more experiance with the idea behind NDIS buffer
> > descriptors
> > and
> > > memory descriptor lists you may see a method that involves less
> > copying.
> > >
> > >
> > >
> > > Thomas F. Divine
> > >
> > >
> > >
> > >
> > >
> > > I will greatly appreciate any help in this regard. Thanks in advance.
> > >
> > >
> > >
> > > Best Regards,
> > >
> > > -Niranjan
> > >
> > >
> > >
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: unknown lmsubst tag
> > argument:
> > ‘’
> > > To unsubscribe send a blank email to
> > > xxxxx@lists.osr.com
> >
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Beyers,

The semantics of Offload are quite unique to the send side of the
protocol/miniport binding. In this case it is explicitly understood by the
Miniport that the Protocol has requested that the modification be made (if
necessary) and that the Protocol has made the appropriate assurances that
the modifications can be accomplished without any further synchronization.
This is in general, however, not a guarantee of the NDIS semantics.

Imagine for a moment a mux-like scenario or perhaps a ‘bridging’ scenario
where your IM driver is above the MUX (bridge); On receipt of a packet, the
frame might be indicated to multiple recipient bindings and ‘sent’ out other
NICs. In each case the frame does not need modification and so the same MDL
chain is attached to multiple NDIS_PACKETs as the buffer list. The original
packet is ‘held’ as long as all other uses of the packet remain outstanding.
If your IM driver is above the mux and decides to modify the packet
(buffers), each recipient (indication / send) might see a different result
since there is no synchronization primitive on the packet to ‘lock’ it for
modification. None of the subsequent protocols or miniports that may be
offered the packet for processing (including IM drivers) truly owns the
buffers and can assure a safe update. They are only assured a safe read.

And no, this is not a contrived scenario. There are products on the market
that implement just such bridging-like behaviors. In fact, you might find
that the XP bridge does as well and that other components like the Virtual
Machine ‘bridge’ protocols would likely be rather upset.

Now the reason that your code might just be working OK is that:

  1. It might only make these modifications on the send path.
  2. It might have only been installed as a IM ‘filter’ driver (with other IM
    filter drivers) over a real NIC.
  3. The packet is not going to be looped back by NDIS.

In this case, it may just be that the buffer chain is not ever referenced
multiply and the update you make works.

The rules are in place to help you not have to test every possible
combination of software to ‘prove’ that it works. Just because it works
does not mean it will work under all circumstances. Just because it has not
yet broken does not mean it wont.

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Beyers Cronje
Sent: Tuesday, August 23, 2005 9:13 AM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Hi Maxim,

When Ip and Tcp checksum offloading is enabled isn’t the miniport or
intermediate driver doing exactly that, updating the data “in place” ?

I’m busy with an intermediate driver that updates the data including
checksums “in place” and all is working 100%.

Beyers

On 8/23/05, Maxim S. Shatskih wrote:
> You cannot change the data “in place” in NDIS. You must allocate a new
data
> space + NDIS_BUFFER for the changed data.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> ----- Original Message -----
> From: “Beyers Cronje”
> To: “Windows System Software Devs Interest List”
> Sent: Wednesday, August 17, 2005 8:55 PM
> Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
>
> Hi Niranjan,
>
> First off let me just state I’m also new to NDIS so take my comments
> at face value.
>
> For case 1 you can change the data without allocating new packet
> (given that the packet has free stack locations) or buffer
> descriptors.
>
> For case 3 I can’t imagine why one can’t chain your buffer to the
> packet and have your sendcomplete handler recycle and unchain your
> buffer. Issues with tcp task offload should be handled.
>
> Beyers
>
> On 8/17/05, Niranjan wrote:
> > There are three cases of updating/chaging a NDIS packet:
> >
> > 1. Updating NDIS packet data without any change in the length of the
> > packet.
> >
> > 2. Updating NDIS packet data where the length of the packet changes
> >
> > 3. Adding a entirely new header at the beginning of the received NDIS
> > packet
> > (no changes in the existing packet data, just addition of few more
> > bytes)
> >
> > For case 1, I am not sure whether we need to allocate a new NDIS packet
> > or
> > we can just change the data in the NDIS buffer as it doesn’t require any
> > memory allocation.
> >
> > For case 2, we need to allocate a new NDIS packet.
> >
> > But for case 3, do I need to allocate a new packet or just a new NDIS
> > buffer. I was thinking of allocating a new NDIS buffer which can hold
> > the
> > new intermediary layer header information and then link this buffer to
> > the
> > head of the buffer-descriptor chain attached to a packet descriptor
> > (using
> > NdisChainBufferAtFront). This will not require unnecessary copy of
> > original
> > packet payload. Please let me know whether this is correct design or
> > not.
> >
> > Best Regards,
> >
> > -Niranjan
> >
> >
> >
> > “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> > Hi Thomas,
> >
> > I was under the impression that one can change data allocated by
> > tcp/ip, e.g. changing the destination IP address, without having to
> > allocate your own packet and buffer descriptors. Am I wrong in this ?
> > Obviously this involves handling checksum and TCP task offload issues.
> >
> > On the other hand, adding additional data such as Niranjan described
> > requires the copy and allocation of your own packet and buffer
> > descriptors ?
> >
> > Regards
> >
> > Beyers Cronje
> >
> > On 8/16/05, Thomas F. Divine wrote:
> > >
> > >
> > >
> > > “Niranjan” wrote in message
> > news:xxxxx@ntdev…
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> > layer
> > has
> > > its own header and few control packets. I wanted to know whether it is
> > > possible to change the packet structure in this intermediate driver
> > > (PassThru drivers) or not. Also, whether this is a good design or not.
> > If
> > > not, what are the other ways in which I can implement this middle
> > layer
> > > between TCP/IP and MAC layer?
> > >
> > >
> > >
> > > NDIS IM drivers seem to be a good choice.
> > >
> > >
> > >
> > >
> > > I am confused because at many places, it suggests not to modify packet
> > data
> > > virtual memory or buffer descriptor at the intermediate drivers if you
> > > haven’t allocated it.
> > >
> > >
> > >
> > > This is not a suggestion. If you didn’t allocate the VM, then you
> > should
> > not
> > > change or modify it. You must invent a method that inserts your header
> > > without modifying other VM or other NDIS buffer descriptors.
> > >
> > >
> > >
> > > The simplest approach is to allocate your own VM sufficiently large to
> > > accomodate the original packet plus your header. Copy the MAC header,
> > your
> > > header and then the original packet payload into your own VM. Then
> > allocate
> > > your own NDIS buffer and packet descriptor and send that instead of
> > the
> > > original.
> > >
> > >
> > >
> > > If you get more experiance with the idea behind NDIS buffer
> > descriptors
> > and
> > > memory descriptor lists you may see a method that involves less
> > copying.
> > >
> > >
> > >
> > > Thomas F. Divine
> > >
> > >
> > >
> > >
> > >
> > > I will greatly appreciate any help in this regard. Thanks in advance.
> > >
> > >
> > >
> > > Best Regards,
> > >
> > > -Niranjan
> > >
> > >
> > >
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: unknown lmsubst tag
> > argument:
> > ‘’
> > > To unsubscribe send a blank email to
> > > xxxxx@lists.osr.com
> >
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi David,

Thank you for a clear and detailed explanation. What you say makes
sense. I was thinking inside the box where our driver will only ever
run in a strict determined configuration.

As I mentioned earlier I’m new to Ndis, do you by any chance have any
recommended reading material on ndis semantics apart from the DDK
documentation and ndis.com ?

Thanks

Beyers

On 8/23/05, David R. Cattley wrote:
> Beyers,
>
> The semantics of Offload are quite unique to the send side of the
> protocol/miniport binding. In this case it is explicitly understood by the
> Miniport that the Protocol has requested that the modification be made (if
> necessary) and that the Protocol has made the appropriate assurances that
> the modifications can be accomplished without any further synchronization.
> This is in general, however, not a guarantee of the NDIS semantics.
>
> Imagine for a moment a mux-like scenario or perhaps a ‘bridging’ scenario
> where your IM driver is above the MUX (bridge); On receipt of a packet, the
> frame might be indicated to multiple recipient bindings and ‘sent’ out other
> NICs. In each case the frame does not need modification and so the same MDL
> chain is attached to multiple NDIS_PACKETs as the buffer list. The original
> packet is ‘held’ as long as all other uses of the packet remain outstanding.
> If your IM driver is above the mux and decides to modify the packet
> (buffers), each recipient (indication / send) might see a different result
> since there is no synchronization primitive on the packet to ‘lock’ it for
> modification. None of the subsequent protocols or miniports that may be
> offered the packet for processing (including IM drivers) truly owns the
> buffers and can assure a safe update. They are only assured a safe read.
>
> And no, this is not a contrived scenario. There are products on the market
> that implement just such bridging-like behaviors. In fact, you might find
> that the XP bridge does as well and that other components like the Virtual
> Machine ‘bridge’ protocols would likely be rather upset.
>
> Now the reason that your code might just be working OK is that:
>
> 1. It might only make these modifications on the send path.
> 2. It might have only been installed as a IM ‘filter’ driver (with other IM
> filter drivers) over a real NIC.
> 3. The packet is not going to be looped back by NDIS.
>
> In this case, it may just be that the buffer chain is not ever referenced
> multiply and the update you make works.
>
> The rules are in place to help you not have to test every possible
> combination of software to ‘prove’ that it works. Just because it works
> does not mean it will work under all circumstances. Just because it has not
> yet broken does not mean it wont.
>
> Good Luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Development
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Beyers Cronje
> Sent: Tuesday, August 23, 2005 9:13 AM
> To: Windows System Software Devs Interest List
> Subject: Re: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
> Hi Maxim,
>
> When Ip and Tcp checksum offloading is enabled isn’t the miniport or
> intermediate driver doing exactly that, updating the data “in place” ?
>
> I’m busy with an intermediate driver that updates the data including
> checksums “in place” and all is working 100%.
>
> Beyers
>
> On 8/23/05, Maxim S. Shatskih wrote:
> > You cannot change the data “in place” in NDIS. You must allocate a new
> data
> > space + NDIS_BUFFER for the changed data.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> > ----- Original Message -----
> > From: “Beyers Cronje”
> > To: “Windows System Software Devs Interest List”
> > Sent: Wednesday, August 17, 2005 8:55 PM
> > Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
> >
> >
> > Hi Niranjan,
> >
> > First off let me just state I’m also new to NDIS so take my comments
> > at face value.
> >
> > For case 1 you can change the data without allocating new packet
> > (given that the packet has free stack locations) or buffer
> > descriptors.
> >
> > For case 3 I can’t imagine why one can’t chain your buffer to the
> > packet and have your sendcomplete handler recycle and unchain your
> > buffer. Issues with tcp task offload should be handled.
> >
> > Beyers
> >
> > On 8/17/05, Niranjan wrote:
> > > There are three cases of updating/chaging a NDIS packet:
> > >
> > > 1. Updating NDIS packet data without any change in the length of the
> > > packet.
> > >
> > > 2. Updating NDIS packet data where the length of the packet changes
> > >
> > > 3. Adding a entirely new header at the beginning of the received NDIS
> > > packet
> > > (no changes in the existing packet data, just addition of few more
> > > bytes)
> > >
> > > For case 1, I am not sure whether we need to allocate a new NDIS packet
> > > or
> > > we can just change the data in the NDIS buffer as it doesn’t require any
> > > memory allocation.
> > >
> > > For case 2, we need to allocate a new NDIS packet.
> > >
> > > But for case 3, do I need to allocate a new packet or just a new NDIS
> > > buffer. I was thinking of allocating a new NDIS buffer which can hold
> > > the
> > > new intermediary layer header information and then link this buffer to
> > > the
> > > head of the buffer-descriptor chain attached to a packet descriptor
> > > (using
> > > NdisChainBufferAtFront). This will not require unnecessary copy of
> > > original
> > > packet payload. Please let me know whether this is correct design or
> > > not.
> > >
> > > Best Regards,
> > >
> > > -Niranjan
> > >
> > >
> > >
> > > “Beyers Cronje” wrote in message news:xxxxx@ntdev…
> > > Hi Thomas,
> > >
> > > I was under the impression that one can change data allocated by
> > > tcp/ip, e.g. changing the destination IP address, without having to
> > > allocate your own packet and buffer descriptors. Am I wrong in this ?
> > > Obviously this involves handling checksum and TCP task offload issues.
> > >
> > > On the other hand, adding additional data such as Niranjan described
> > > requires the copy and allocation of your own packet and buffer
> > > descriptors ?
> > >
> > > Regards
> > >
> > > Beyers Cronje
> > >
> > > On 8/16/05, Thomas F. Divine wrote:
> > > >
> > > >
> > > >
> > > > “Niranjan” wrote in message
> > > news:xxxxx@ntdev…
> > > >
> > > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > > > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> > > layer
> > > has
> > > > its own header and few control packets. I wanted to know whether it is
> > > > possible to change the packet structure in this intermediate driver
> > > > (PassThru drivers) or not. Also, whether this is a good design or not.
> > > If
> > > > not, what are the other ways in which I can implement this middle
> > > layer
> > > > between TCP/IP and MAC layer?
> > > >
> > > >
> > > >
> > > > NDIS IM drivers seem to be a good choice.
> > > >
> > > >
> > > >
> > > >
> > > > I am confused because at many places, it suggests not to modify packet
> > > data
> > > > virtual memory or buffer descriptor at the intermediate drivers if you
> > > > haven’t allocated it.
> > > >
> > > >
> > > >
> > > > This is not a suggestion. If you didn’t allocate the VM, then you
> > > should
> > > not
> > > > change or modify it. You must invent a method that inserts your header
> > > > without modifying other VM or other NDIS buffer descriptors.
> > > >
> > > >
> > > >
> > > > The simplest approach is to allocate your own VM sufficiently large to
> > > > accomodate the original packet plus your header. Copy the MAC header,
> > > your
> > > > header and then the original packet payload into your own VM. Then
> > > allocate
> > > > your own NDIS buffer and packet descriptor and send that instead of
> > > the
> > > > original.
> > > >
> > > >
> > > >
> > > > If you get more experiance with the idea behind NDIS buffer
> > > descriptors
> > > and
> > > > memory descriptor lists you may see a method that involves less
> > > copying.
> > > >
> > > >
> > > >
> > > > Thomas F. Divine
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I will greatly appreciate any help in this regard. Thanks in advance.
> > > >
> > > >
> > > >
> > > > Best Regards,
> > > >
> > > > -Niranjan
> > > >
> > > >
> > > >
> > > >
> > > > —
> > > > Questions? First check the Kernel Driver FAQ at
> > > > http://www.osronline.com/article.cfm?id=256
> > > >
> > > > You are currently subscribed to ntdev as: unknown lmsubst tag
> > > argument:
> > > ‘’
> > > > To unsubscribe send a blank email to
> > > > xxxxx@lists.osr.com
> > >
> > >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@gmail.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Beyers,

The NDIS docs in the DDK are very good, of course, but not particularly
tutorial. The excellent NDIS FAQ compiled by Tom Divine & others at
www.ndis.com (www.pcausa.com) should be considered required reading (again
and again!) In particular, the Loop Back discussion is one I recommend.

Ultimately, however, IM drivers are not particularly well covered in the
samples beyond the basic skeleton processing of MUX and PASSTHRU but the
comments in the code are as important as the code itself. There are often
hints as to how to accomplish certain feats of NDIS magic. MUX implements
some creative packet buffer management that I suggest you look at and the
relatively new Virtual Miniport sample in the 2K3 DDK also teaches some good
lessons in how packets and buffers can be handled.

And one thing you might note; Lots of IM drivers disable offload as a
matter of course. It takes a bit of extra work to deal with if the IM
driver is actually processing Layer 3 (I guess we could just say IP now…)
data (like so many of them do).

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Beyers Cronje
Sent: Tuesday, August 23, 2005 5:11 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] NDIS Intermediate Driver - Adding new header

Hi David,

Thank you for a clear and detailed explanation. What you say makes
sense. I was thinking inside the box where our driver will only ever
run in a strict determined configuration.

As I mentioned earlier I’m new to Ndis, do you by any chance have any
recommended reading material on ndis semantics apart from the DDK
documentation and ndis.com ?

Thanks

Beyers

On 8/23/05, David R. Cattley wrote:
> Beyers,
>
> The semantics of Offload are quite unique to the send side of the
> protocol/miniport binding. In this case it is explicitly understood by
the
> Miniport that the Protocol has requested that the modification be made (if
> necessary) and that the Protocol has made the appropriate assurances that
> the modifications can be accomplished without any further synchronization.
> This is in general, however, not a guarantee of the NDIS semantics.
>
> Imagine for a moment a mux-like scenario or perhaps a ‘bridging’ scenario
> where your IM driver is above the MUX (bridge); On receipt of a packet,
the
> frame might be indicated to multiple recipient bindings and ‘sent’ out
other
> NICs. In each case the frame does not need modification and so the same
MDL
> chain is attached to multiple NDIS_PACKETs as the buffer list. The
original
> packet is ‘held’ as long as all other uses of the packet remain
outstanding.
> If your IM driver is above the mux and decides to modify the packet
> (buffers), each recipient (indication / send) might see a different result
> since there is no synchronization primitive on the packet to ‘lock’ it for
> modification. None of the subsequent protocols or miniports that may be
> offered the packet for processing (including IM drivers) truly owns the
> buffers and can assure a safe update. They are only assured a safe read.
>
> And no, this is not a contrived scenario. There are products on the
market
> that implement just such bridging-like behaviors. In fact, you might find
> that the XP bridge does as well and that other components like the Virtual
> Machine ‘bridge’ protocols would likely be rather upset.
>
> Now the reason that your code might just be working OK is that:
>
> 1. It might only make these modifications on the send path.
> 2. It might have only been installed as a IM ‘filter’ driver (with other
IM
> filter drivers) over a real NIC.
> 3. The packet is not going to be looped back by NDIS.
>
> In this case, it may just be that the buffer chain is not ever referenced
> multiply and the update you make works.
>
> The rules are in place to help you not have to test every possible
> combination of software to ‘prove’ that it works. Just because it works
> does not mean it will work under all circumstances. Just because it has
not
> yet broken does not mean it wont.
>
> Good Luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Development
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Beyers Cronje
> Sent: Tuesday, August 23, 2005 9:13 AM
> To: Windows System Software Devs Interest List
> Subject: Re: Re:[ntdev] NDIS Intermediate Driver - Adding new header
>
> Hi Maxim,
>
> When Ip and Tcp checksum offloading is enabled isn’t the miniport or
> intermediate driver doing exactly that, updating the data “in place” ?
>
> I’m busy with an intermediate driver that updates the data including
> checksums “in place” and all is working 100%.
>
> Beyers
>
> On 8/23/05, Maxim S. Shatskih wrote:
> > You cannot change the data “in place” in NDIS. You must allocate a
new
> data
> > space + NDIS_BUFFER for the changed data.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> > ----- Original Message -----
> > From: “Beyers Cronje”
> > To: “Windows System Software Devs Interest List”
> > Sent: Wednesday, August 17, 2005 8:55 PM
> > Subject: Re:[ntdev] NDIS Intermediate Driver - Adding new header
> >
> >
> > Hi Niranjan,
> >
> > First off let me just state I’m also new to NDIS so take my comments
> > at face value.
> >
> > For case 1 you can change the data without allocating new packet
> > (given that the packet has free stack locations) or buffer
> > descriptors.
> >
> > For case 3 I can’t imagine why one can’t chain your buffer to the
> > packet and have your sendcomplete handler recycle and unchain your
> > buffer. Issues with tcp task offload should be handled.
> >
> > Beyers
> >
> > On 8/17/05, Niranjan wrote:
> > > There are three cases of updating/chaging a NDIS packet:
> > >
> > > 1. Updating NDIS packet data without any change in the length of the
> > > packet.
> > >
> > > 2. Updating NDIS packet data where the length of the packet changes
> > >
> > > 3. Adding a entirely new header at the beginning of the received NDIS
> > > packet
> > > (no changes in the existing packet data, just addition of few more
> > > bytes)
> > >
> > > For case 1, I am not sure whether we need to allocate a new NDIS
packet
> > > or
> > > we can just change the data in the NDIS buffer as it doesn’t require
any
> > > memory allocation.
> > >
> > > For case 2, we need to allocate a new NDIS packet.
> > >
> > > But for case 3, do I need to allocate a new packet or just a new NDIS
> > > buffer. I was thinking of allocating a new NDIS buffer which can hold
> > > the
> > > new intermediary layer header information and then link this buffer to
> > > the
> > > head of the buffer-descriptor chain attached to a packet descriptor
> > > (using
> > > NdisChainBufferAtFront). This will not require unnecessary copy of
> > > original
> > > packet payload. Please let me know whether this is correct design or
> > > not.
> > >
> > > Best Regards,
> > >
> > > -Niranjan
> > >
> > >
> > >
> > > “Beyers Cronje” wrote in message
news:xxxxx@ntdev…
> > > Hi Thomas,
> > >
> > > I was under the impression that one can change data allocated by
> > > tcp/ip, e.g. changing the destination IP address, without having to
> > > allocate your own packet and buffer descriptors. Am I wrong in this ?
> > > Obviously this involves handling checksum and TCP task offload issues.
> > >
> > > On the other hand, adding additional data such as Niranjan described
> > > requires the copy and allocation of your own packet and buffer
> > > descriptors ?
> > >
> > > Regards
> > >
> > > Beyers Cronje
> > >
> > > On 8/16/05, Thomas F. Divine wrote:
> > > >
> > > >
> > > >
> > > > “Niranjan” wrote in message
> > > news:xxxxx@ntdev…
> > > >
> > > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > I am designing a new layer between TCP/IP and MAC layer using NDIS
> > > > intermediate drivers (PassThru samples) on Windows CE 5.0. The new
> > > layer
> > > has
> > > > its own header and few control packets. I wanted to know whether it
is
> > > > possible to change the packet structure in this intermediate driver
> > > > (PassThru drivers) or not. Also, whether this is a good design or
not.
> > > If
> > > > not, what are the other ways in which I can implement this middle
> > > layer
> > > > between TCP/IP and MAC layer?
> > > >
> > > >
> > > >
> > > > NDIS IM drivers seem to be a good choice.
> > > >
> > > >
> > > >
> > > >
> > > > I am confused because at many places, it suggests not to modify
packet
> > > data
> > > > virtual memory or buffer descriptor at the intermediate drivers if
you
> > > > haven’t allocated it.
> > > >
> > > >
> > > >
> > > > This is not a suggestion. If you didn’t allocate the VM, then you
> > > should
> > > not
> > > > change or modify it. You must invent a method that inserts your
header
> > > > without modifying other VM or other NDIS buffer descriptors.
> > > >
> > > >
> > > >
> > > > The simplest approach is to allocate your own VM sufficiently large
to
> > > > accomodate the original packet plus your header. Copy the MAC
header,
> > > your
> > > > header and then the original packet payload into your own VM. Then
> > > allocate
> > > > your own NDIS buffer and packet descriptor and send that instead of
> > > the
> > > > original.
> > > >
> > > >
> > > >
> > > > If you get more experiance with the idea behind NDIS buffer
> > > descriptors
> > > and
> > > > memory descriptor lists you may see a method that involves less
> > > copying.
> > > >
> > > >
> > > >
> > > > Thomas F. Divine
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I will greatly appreciate any help in this regard. Thanks in
advance.
> > > >
> > > >
> > > >
> > > > Best Regards,
> > > >
> > > > -Niranjan
> > > >
> > > >
> > > >
> > > >
> > > > —
> > > > Questions? First check the Kernel Driver FAQ at
> > > > http://www.osronline.com/article.cfm?id=256
> > > >
> > > > You are currently subscribed to ntdev as: unknown lmsubst tag
> > > argument:
> > > ‘’
> > > > To unsubscribe send a blank email to
> > > > xxxxx@lists.osr.com
> > >
> > >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@gmail.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com