Re: Maximum scatter/gather elements in network driver - - Continued

The number of Scatter/Gather elements has no relation
to pages in this situation.

Here is the situation:

The problem was triggered by a well-known backup program
that performs many tiny sends (between 1 and 20 bytes each)
during initialization.

The first send is sent out, but because of the NAGLE algorithm,
the remaining sends are allowed to accumulate in the Microsoft
TCP stack. When the ACK for the fist send arrives, the
remaining sends (now roughly 40 of them) are sent to
the driver as a single packet.

If NdisMInitializeScatterGatherDma is used, the packet
(with ~40 NDIS_BUFFERS) doesn’t get past NDIS.

I talked to Microsoft about this and they confirmed that
this is a known bug in Windows 2000. It is fixed in
XP. It is not fixed in any of the W2k service packs.

Note that the amount of scatter/gather elements allowed
is related (though not strictly proportional) to the
size of the third parameter (MaximumPhysicalMapping)
to NdisMInitializeScatterGatherDMA. A size of 1514
limits the number of elements to ~16. 64k increases
it to 33. 128k brings it to ~64 elements.

I hope others find this helpful.

Regards,

Pete

----- Original Message -----
From: “Rosner, Gedon”
To: “NT Developers Interest List”
Sent: Thursday, January 24, 2002 7:23 AM
Subject: [ntdev] Re: Maximum scatter/gather elements in network driver - -
Continued

> Under WinXP Pro with “normal” applications, there are between 8-9
fragments
> with TaskOffload under TcpSegmentation, as I mentioned before “Under
extreme
> circumstances … 33 fragments is not an imaginary number…” sometimes a
> “reason to believe” is “real live numbers”.
> I haven’t checked the Win2K hotfix yet.
> Regards,
> Gedon.
>
> -----Original Message-----
> From: xxxxx@hotmail.com [mailto:xxxxx@hotmail.com]
> Sent: Thu, January 24, 2002 4:49 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Maximum scatter/gather elements in network driver-
> - Continued
>
>
> Hmm, even a 64K TCP packet (NDIS_TASK_TCP_LARGE_SEND) would probably
> only span 64k / 4K = 16 pages plus maybe some extra fragments for the
> MAC/IP/TCP headers. That’s not even close to 33.
>
> I haven’t played with TCP large sends yet, though. Some real live
> numbers for the buffer fragment or DMA segment count for large sends
> anyone?
>
> Stephan
> —
> On Thu, 24 Jan 2002 16:18:01 +0200, “Rosner, Gedon”
> wrote:
>
> >
> >Under extreme circumstances - say TCP Segmentation where an ndis_packet
can
> >be up to 64K large- 33 fragments is not an imaginary number… I have
> reason
> >to believe that 33 is not the limit when Offloading is performed, and
since
> >Offloading capabilities is queried after initialization, I don’t believe
> >that NdisMInitializeScatterGatherDma sets such a low limit.
> >
> >-----Original Message-----
> >From: xxxxx@hotmail.com [mailto:xxxxx@hotmail.com]
> >Sent: Thu, January 24, 2002 3:29 PM
> >To: NT Developers Interest List
> >Subject: [ntdev] Re: Maximum scatter/gather elements in network driver -
> >Continued
> >
> >
> >Who is the originator of packets with more than 33 buffer fragments?
> >
> >Although I cannot find any “official” limit for the number of DMA
> >fragments in a SCATTER_GATHER_LIST in the docs, there probably is some
> >limit. However, 33 already sounds like a huge number IMHO. Who ever
> >needs so many packet fragments?
> >
> >Stephan
> >—
> >On Mon, 21 Jan 2002 18:53:12 -0800, “Peter Craft”
> > wrote:
> >
> >>Strange. If I use map registers instead of
> >>NdisMInitializeScatterGatherDma I don’t have this problem.
> >>
> >>Can someone tell me what it is about NdisMInitializeScatterGatherDma
> >>that limits the number of scatter/gather elements in a send to 33?
> >>More importantly, how do I change this limit (short of going back to
> >>using map registers.
> >>
> >>Thanks
> >
> >
> >—
> >You are currently subscribed to ntdev as: xxxxx@intel.com
> >To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> >
> >—
> >You are currently subscribed to ntdev as: xxxxx@intel.com
> >To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntdev as: xxxxx@intel.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntdev as: xxxxx@alacritech.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

NAGLE makes perfect sense. IIRC, NAGLE can be disabled, though. We had
this kind of problem years ago (NT4 or even 3.51). Don’t remember any
details, sorry.

Stephan

On Fri, 25 Jan 2002 11:10:25 -0800, “Peter Craft”
wrote:

>The number of Scatter/Gather elements has no relation
>to pages in this situation.
>
>Here is the situation:
>
>The problem was triggered by a well-known backup program
>that performs many tiny sends (between 1 and 20 bytes each)
>during initialization.
>
>The first send is sent out, but because of the NAGLE algorithm,
>the remaining sends are allowed to accumulate in the Microsoft
>TCP stack. When the ACK for the fist send arrives, the
>remaining sends (now roughly 40 of them) are sent to
>the driver as a single packet.
>
>If NdisMInitializeScatterGatherDma is used, the packet
>(with ~40 NDIS_BUFFERS) doesn’t get past NDIS.
>
>I talked to Microsoft about this and they confirmed that
>this is a known bug in Windows 2000. It is fixed in
>XP. It is not fixed in any of the W2k service packs.
>
>Note that the amount of scatter/gather elements allowed
>is related (though not strictly proportional) to the
>size of the third parameter (MaximumPhysicalMapping)
>to NdisMInitializeScatterGatherDMA. A size of 1514
>limits the number of elements to ~16. 64k increases
>it to 33. 128k brings it to ~64 elements.
>
>I hope others find this helpful.
>
>Regards,
>
>Pete
>
>
>----- Original Message -----
>From: “Rosner, Gedon”
>To: “NT Developers Interest List”
>Sent: Thursday, January 24, 2002 7:23 AM
>Subject: [ntdev] Re: Maximum scatter/gather elements in network driver - -
>Continued
>
>
>> Under WinXP Pro with “normal” applications, there are between 8-9
>fragments
>> with TaskOffload under TcpSegmentation, as I mentioned before “Under
>extreme
>> circumstances … 33 fragments is not an imaginary number…” sometimes a
>> “reason to believe” is “real live numbers”.
>> I haven’t checked the Win2K hotfix yet.
>> Regards,
>> Gedon.
>>
>> -----Original Message-----
>> From: xxxxx@hotmail.com [mailto:xxxxx@hotmail.com]
>> Sent: Thu, January 24, 2002 4:49 PM
>> To: NT Developers Interest List
>> Subject: [ntdev] Re: Maximum scatter/gather elements in network driver-
>> - Continued
>>
>>
>> Hmm, even a 64K TCP packet (NDIS_TASK_TCP_LARGE_SEND) would probably
>> only span 64k / 4K = 16 pages plus maybe some extra fragments for the
>> MAC/IP/TCP headers. That’s not even close to 33.
>>
>> I haven’t played with TCP large sends yet, though. Some real live
>> numbers for the buffer fragment or DMA segment count for large sends
>> anyone?
>>
>> Stephan
>> —
>> On Thu, 24 Jan 2002 16:18:01 +0200, “Rosner, Gedon”
>> wrote:
>>
>> >
>> >Under extreme circumstances - say TCP Segmentation where an ndis_packet
>can
>> >be up to 64K large- 33 fragments is not an imaginary number… I have
>> reason
>> >to believe that 33 is not the limit when Offloading is performed, and
>since
>> >Offloading capabilities is queried after initialization, I don’t believe
>> >that NdisMInitializeScatterGatherDma sets such a low limit.
>> >
>> >-----Original Message-----
>> >From: xxxxx@hotmail.com [mailto:xxxxx@hotmail.com]
>> >Sent: Thu, January 24, 2002 3:29 PM
>> >To: NT Developers Interest List
>> >Subject: [ntdev] Re: Maximum scatter/gather elements in network driver -
>> >Continued
>> >
>> >
>> >Who is the originator of packets with more than 33 buffer fragments?
>> >
>> >Although I cannot find any “official” limit for the number of DMA
>> >fragments in a SCATTER_GATHER_LIST in the docs, there probably is some
>> >limit. However, 33 already sounds like a huge number IMHO. Who ever
>> >needs so many packet fragments?
>> >
>> >Stephan
>> >—
>> >On Mon, 21 Jan 2002 18:53:12 -0800, “Peter Craft”
>> > wrote:
>> >
>> >>Strange. If I use map registers instead of
>> >>NdisMInitializeScatterGatherDma I don’t have this problem.
>> >>
>> >>Can someone tell me what it is about NdisMInitializeScatterGatherDma
>> >>that limits the number of scatter/gather elements in a send to 33?
>> >>More importantly, how do I change this limit (short of going back to
>> >>using map registers.
>> >>
>> >>Thanks


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com