NDISPROT: Problems with sending packets

Hi All,

My NDIS protocol driver is based on the NDISPROT example.
For better performance I added NdisSendPackets() call with multiple packet
descriptors.

The problem occurs on some machines (Win XP SP1/SP2, Wind 2003 Server SP1) -
it seems that the last packet from a chain is not sent with others and is
being held somewhere in the NDIS driver stack.
My SendComplete callback is called for the packet as for other packets that
have sent OK.
The problem exists only for NICs in 1GbE mode, in 100MbE mode all works
well.

After the timeout occurs my process tries to retry the hanged packet once
more.
After that the other side receives 2 packets - the old one and the new. - It
looks like if the new packet unfreezes something in the NDIS stack…

I’ve analyzed the network traffic with Ethereal and tcpdump from both client
and server sides, and it looks like the client’s NDIS miniport or the NIC is
to blame for the problem.

Maybe somebody of you solved the problem in the past?

My another guess is that in my NDIS protocol I only allocate packets with
NdisAllocatePacket(), but do not set any fields in the packet’s header.
Is this OK or I must set some fields inside it?

Any help or suggestions are appreciated.

Best regards,
Valeriy Glushkov


éÎÔÅÒÅÓÎÙÊ ÐÏÒÔÁÌ - http://www.online.ua

How does the nimber of packets that you send compare to the value returned
by OID_GEN_MAXIMUM_SEND_PACKETS?

Thomas F. Divine, Windows DDK MVP
http://www.pcausa.com

“Valeriy Glushkov” wrote in message news:xxxxx@ntdev…
> Hi All,
>
> My NDIS protocol driver is based on the NDISPROT example.
> For better performance I added NdisSendPackets() call with multiple packet
> descriptors.
>
> The problem occurs on some machines (Win XP SP1/SP2, Wind 2003 Server
> SP1) - it seems that the last packet from a chain is not sent with others
> and is being held somewhere in the NDIS driver stack.
> My SendComplete callback is called for the packet as for other packets
> that have sent OK.
> The problem exists only for NICs in 1GbE mode, in 100MbE mode all works
> well.
>
> After the timeout occurs my process tries to retry the hanged packet once
> more.
> After that the other side receives 2 packets - the old one and the new. -
> It looks like if the new packet unfreezes something in the NDIS stack…
>
> I’ve analyzed the network traffic with Ethereal and tcpdump from both
> client and server sides, and it looks like the client’s NDIS miniport or
> the NIC is to blame for the problem.
>
> Maybe somebody of you solved the problem in the past?
>
> My another guess is that in my NDIS protocol I only allocate packets with
> NdisAllocatePacket(), but do not set any fields in the packet’s header.
> Is this OK or I must set some fields inside it?
>
> Any help or suggestions are appreciated.
>
> Best regards,
> Valeriy Glushkov
>
>
>
>
>
> ___
> éÎÔÅÒÅÓÎÙÊ ÐÏÒÔÁÌ - http://www.online.ua
>
>
>

Valeriy,

Do I understand correctly that your ProtocolSendComplete() gets called for
the (last) packet that you say has not been sent? What is the status on
that callback? Is it NDIS_STATUS_RESOURCES?

If in fact the ProtocolSendComplete() handler is called, the packet is *not*
held in the miniport (NDIS would not hold the packet either except while
trying to loop it back but I believe that all versions of NDIS5x to date
actually force the loopback to be syncronous).

When your driver ‘retries’ the packet, is it simply another new packet with
the same contents or are you actually sending the very same NDIS_PACKET
structure into NdisSend{Packets} again?

AFAIK you do not need to setup any fields in the NDIS_PACKET header or OOB
data as long as the packet was allocated with NdisAllocatePacket(). If the
packet has been returned via ProtocolSendComplete(), the NDIS_PACKET should
be re-initialized before it is used again. Just for the sake of knowing,
however, you might want to check that the packet status is
NDIS_STATUS_SUCCESS before you send it and that the OOB data for ‘time to
send’ (NDIS_{SET|GET}_PACKET_TIME_TO_SEND() ) is not some value indicating
that the packet should be delayed.

Lastly, what is the NIC type that you are experiencing this symptom on?
Have you tried other NICs?

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

Thomas,

I’ve totally missed to examine OID_GEN_MAXIMUM_SEND_PACKETS in the driver.
:frowning:

When I last examined the problem it occured sometimes for 4 packet bursts…

I’ll add the code for checking of OID_GEN_MAXIMUM_SEND_PACKETS and let you
know whether it helps.

Thank you.

Best regards,
Valeriy Glushkov

----- éÓÈÏÄÎÏÅ ÓÏÏÂÝÅÎÉÅ -----
ïÔ: “Thomas F. Divine”
çÒÕÐÐÙ ÎÏ×ÏÓÔÅÊ: ntdev
ëÏÍÕ: “Windows System Software Devs Interest List”
ïÔÐÒÁ×ÌÅÎÏ: 2 Á×ÇÕÓÔÁ 2006 Ç. 18:23
ôÅÍÁ: Re:[ntdev] NDISPROT: Problems with sending packets

> How does the nimber of packets that you send compare to the value returned
> by OID_GEN_MAXIMUM_SEND_PACKETS?
>
> Thomas F. Divine, Windows DDK MVP
> http://www.pcausa.com
>
> “Valeriy Glushkov” wrote in message news:xxxxx@ntdev…
>> Hi All,
>>
>> My NDIS protocol driver is based on the NDISPROT example.
>> For better performance I added NdisSendPackets() call with multiple
>> packet descriptors.
>>
>> The problem occurs on some machines (Win XP SP1/SP2, Wind 2003 Server
>> SP1) - it seems that the last packet from a chain is not sent with others
>> and is being held somewhere in the NDIS driver stack.
>> My SendComplete callback is called for the packet as for other packets
>> that have sent OK.
>> The problem exists only for NICs in 1GbE mode, in 100MbE mode all works
>> well.
>>
>> After the timeout occurs my process tries to retry the hanged packet once
>> more.
>> After that the other side receives 2 packets - the old one and the new. -
>> It looks like if the new packet unfreezes something in the NDIS stack…
>>
>> I’ve analyzed the network traffic with Ethereal and tcpdump from both
>> client and server sides, and it looks like the client’s NDIS miniport or
>> the NIC is to blame for the problem.
>>
>> Maybe somebody of you solved the problem in the past?
>>
>> My another guess is that in my NDIS protocol I only allocate packets with
>> NdisAllocatePacket(), but do not set any fields in the packet’s header.
>> Is this OK or I must set some fields inside it?
>>
>> Any help or suggestions are appreciated.
>>
>> Best regards,
>> Valeriy Glushkov
>>
>>
>>
>>
>>
>> ___
>> éÎÔÅÒÅÓÎÙÊ ÐÏÒÔÁÌ - http://www.online.ua
>>
>>
>>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

David,

??: “David R. Cattley”
???: “Windows System Software Devs Interest List”
???: 2 ??? 2006 ?. 18:57
???: RE: [ntdev] NDISPROT: Problems with sending packets

> Valeriy,
>
> Do I understand correctly that your ProtocolSendComplete() gets called for
> the (last) packet that you say has not been sent? What is the status on
> that callback? Is it NDIS_STATUS_RESOURCES?

The completion handler always returns NDIS_STATUS_SUCCESS ever for these not
yet physically sent packets.

> If in fact the ProtocolSendComplete() handler is called, the packet is
> not
> held in the miniport (NDIS would not hold the packet either except while
> trying to loop it back but I believe that all versions of NDIS5x to date
> actually force the loopback to be syncronous).

I guess the packet’s data has been copied into other buffers somewhere
lower of my driver.
That’s why my original packet is completed with success…

> When your driver ‘retries’ the packet, is it simply another new packet
> with
> the same contents or are you actually sending the very same NDIS_PACKET
> structure into NdisSend{Packets} again?

Of course I allocate another NDIS_PACKET for retries.

> AFAIK you do not need to setup any fields in the NDIS_PACKET header or OOB
> data as long as the packet was allocated with NdisAllocatePacket(). If
> the
> packet has been returned via ProtocolSendComplete(), the NDIS_PACKET
> should
> be re-initialized before it is used again. Just for the sake of knowing,
> however, you might want to check that the packet status is
> NDIS_STATUS_SUCCESS before you send it and that the OOB data for ‘time to
> send’ (NDIS_{SET|GET}_PACKET_TIME_TO_SEND() ) is not some value indicating
> that the packet should be delayed.

I always Allocate and Deallocate packets in the driver.

But I’ll check it out whether the fields are set correctly.

> Lastly, what is the NIC type that you are experiencing this symptom on?
> Have you tried other NICs?

The problem was reproduced on a D-Link DGE-530T Giga NIC and a Broadcom 570x
Gigabit Controller.
Intel, 3COM and NVidia 1GbE NIC seem to work without problems.

> Good Luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Development

Best regards,
Valeriy Glushkov

Sorry, had to ask the packet handling questions to be sure I understood your
symptoms.

Are there any IM drives bound between your protocol and the NIC?
When you say that the packet is sent twice, where are you ‘measuring’ this?
Is it at the receiver or with Netmon/Ethereal on the sender?

Regarding OID_GEN_MAXIMUM_SEND_PACKETS -


OID_GEN_MAXIMUM_SEND_PACKETS
For Serialized Miniport Drivers

The OID_GEN_MAXIMUM_SEND_PACKETS OID specifies the maximum number of send
packet descriptors that a miniport driver’s MiniportSendPackets function can
accept in the array at its PacketArray parameter. NDIS will not call the
miniport driver’s MiniportSendPackets function with more than this number of
packets in the array at PacketArray.

If the underlying driver has only a MiniportSend function, it should return
one for this query. In this case, NDIS will always pass one packet to the
array at PacketArray.

Legacy NIC drivers usually return NDIS_STATUS_NOT_SUPPORTED for this query.

If a protocol, when calling NdisSendPackets, sends more packets in the
packet array than a miniport driver indicates that it can handle, NDIS
breaks the incoming packet array into smaller arrays and then calls the
miniport driver’s MiniportSendPackets function multiple times. It may be
more optimal for the miniport driver to handle this situation?that is, to
break down the packet array into smaller packet arrays, thereby eliminating
multiple calls to its MiniportSendPackets function.

For Deserialized Miniport Drivers

NDIS ignores any value returned by a deserialized driver in response to a
query of OID_GEN_MAXIMUM_SEND_PACKETS. NDIS does not adjust the size of the
array of packet descriptors that it supplies to a deserialized miniport’s
MiniportSendPackets function.

However, it is a bit terse and potentially confusing as it is written from
the perspective of a Miniport author and not the Protocol author. It does
not say that a protocol must not send more than this value. In fact, in a
serialized miniport, NDIS itself spoon-feeds the miniport in chunks thus
allowing a Protocol to send as many packets in the NdisSendPackets() call as
it pleases. This sets up the very likely possiblity that:

A) Protocol authors have no requirement to limit send blocks to <=
OID_GEN_MAXIMUM_SEND_PACKETS
B) Casually written *deserialized* Miniports do not handle (because the do
not expect) this case.

The answer is likely to be something to the effect of “do what TCPIP.SYS
does” (whatever that is).

Perhaps Thomas or one of our SMFT NDIS regulars knows the bottom line answer
on this one (Elias?)

I don’t recall the chipset for the D-Link (RTK8169?) and yours may differ.
But the fact that the Intel works (as expected) makes me think you are not
doing anything terrible. Perhaps the BCM driver owner will comment on the
behavior observed in the 570x.

Good Luck,
Dave Cattley
Constlting Engineer
Systems Software Development