How big is the TDI receiving buffer?

OSR_Community_User · July 8, 2002, 6:39pm

Hi,
I have a driver which sends/recvs data from network using TDI.
I have my own ClientEventReceive handler installed.

The driver works fine when only sending data out. However, it
will crash if i do mixed sending/recving for a while. This seems only happen
when i run on a gigabit ethernet. For 100Mbit network, very rare, although
i did see the problem once.

The bugcheck happened in TcpChecksum() or sometimes TcpFreePartialRB().
I have checked my code many times to see if there is potential memory
violation
in my code but could not find any.

Now I start to concern there could be packets lost if i spend too
much time processing other stuff in my completion routines. Basically, I
guess
the default TDI receiving buffer should be big enough to hold whatever data
has
arrived from the remote peer untill i consume them. But right now, i am not
quite sure if i am fast enough to consume the data. What will happen if
packet get
lost?

Did anyone have similiar experience? By the way, I am using Intel Pro
Gigabit
NIC and W2K systems.

Thanks. Any feedback is welcome.

Wei

Following is a brief description of the problem. Please diccard the symbols
dismatch
message.

****************************************************************************
***
*
*
* Bugcheck Analysis
*
*
*
****************************************************************************
***

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pagable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000c, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: f714a684, address which referenced memory

Debugging Details:

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: D1

LAST_CONTROL_TRANSFER: from 85750d98 to 80466f7c

STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be
wrong.
804717b8 85750d98 85750d9c 00000000 ffdff120 nt!Kei386EoiHelper+0x2ae4
85750d98 d522426a 00000020 00000000 00000000 0x85750d98
e100080f 14000000 02000100 00010100 00000000 0xd522426a
00022000 3f3f3f3f 3f3f3f3f 3f3f3f3f 3f3f3f3f 0x14000000
3f3f3f3f 00000000 00000000 00000000 00000000 0x3f3f3f3f

FOLLOWUP_IP:
nt!Kei386EoiHelper+2ae4
80466f7c f7457000000200 test dword ptr [ebp+0x70],0x20000

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: nt!Kei386EoiHelper+2ae4

MODULE_NAME: nt

IMAGE_NAME: ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 3975dfdf

STACK_COMMAND: kb

BUCKET_ID: WRONG_SYMBOLS

Followup: MachineOwner

kd> kb

ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be
wrong.
804717b8 85750d98 85750d9c 00000000 ffdff120 nt!Kei386EoiHelper+0x2ae4
85750d98 d522426a 00000020 00000000 00000000 0x85750d98
e100080f 14000000 02000100 00010100 00000000 0xd522426a
00022000 3f3f3f3f 3f3f3f3f 3f3f3f3f 3f3f3f3f 0x14000000
3f3f3f3f 00000000 00000000 00000000 00000000 0x3f3f3f3f

kd> ln f714a684
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
tcpip.sys -
(f71452c8) tcpip!tcpxsum+0x53bc | (f715ee00) tcpip!IPInjectPkt

OSR_Community_User-14 · July 8, 2002, 7:28pm

Hi Wei,

The bugcheck happened in TcpChecksum() or sometimes TcpFreePartialRB().
I have checked my code many times to see if there is potential memory
violation
in my code but could not find any.

The address referenced in your blue-screen is bogus. The memory referenced
is
0x0000000c, which is virtually guaranteed to be an invalid system virtual
address.
At first glance, I’d guess that the MDL that you are passing down to TCP is
messed up somehow. I’d also guess that it was one of your send MDLs that
resulted
in the crash due to the fact that it crashed in tcpxsum. TCP has to
calculate checksums
on both inbound and outbound traffic, but inbound data given to TCP would be
located in
a buffer allocated by the Intel driver, which is unlikely to be a bad
address.
So it sounds like something you are passing down on a send is bad.

Now I start to concern there could be packets lost if i spend too
much time processing other stuff in my completion routines. Basically, I
guess
the default TDI receiving buffer should be big enough to hold whatever
data
has
arrived from the remote peer untill i consume them. But right now, i am
not
quite sure if i am fast enough to consume the data. What will happen if
packet get
lost?

If you are using TCP, this isn’t an issue. The TCP window will not get
opened until you consume the data. If you fail to consume the data fast
enough,
the TCP window will close and the remote side will stop sending. If packets
get dropped, TCP will ensure that they get retransmitted and reassembled
properly.

If you are using UDP, then it’s your job to deal with these possible error
cases.

Hope this helps.

Pete

Did anyone have similiar experience? By the way, I am using Intel Pro
Gigabit
NIC and W2K systems.

Thanks. Any feedback is welcome.

Wei

Following is a brief description of the problem. Please diccard the
symbols
dismatch
message.

****************************************************************************

***
*
*
* Bugcheck Analysis
*
*
*

****************************************************************************

***

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pagable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000c, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: f714a684, address which referenced memory

Debugging Details:

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: D1

LAST_CONTROL_TRANSFER: from 85750d98 to 80466f7c

STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be
wrong.
804717b8 85750d98 85750d9c 00000000 ffdff120 nt!Kei386EoiHelper+0x2ae4
85750d98 d522426a 00000020 00000000 00000000 0x85750d98
e100080f 14000000 02000100 00010100 00000000 0xd522426a
00022000 3f3f3f3f 3f3f3f3f 3f3f3f3f 3f3f3f3f 0x14000000
3f3f3f3f 00000000 00000000 00000000 00000000 0x3f3f3f3f

FOLLOWUP_IP:
nt!Kei386EoiHelper+2ae4
80466f7c f7457000000200 test dword ptr [ebp+0x70],0x20000

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: nt!Kei386EoiHelper+2ae4

MODULE_NAME: nt

IMAGE_NAME: ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 3975dfdf

STACK_COMMAND: kb

BUCKET_ID: WRONG_SYMBOLS

Followup: MachineOwner

kd> kb

ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be
wrong.
804717b8 85750d98 85750d9c 00000000 ffdff120 nt!Kei386EoiHelper+0x2ae4
85750d98 d522426a 00000020 00000000 00000000 0x85750d98
e100080f 14000000 02000100 00010100 00000000 0xd522426a
00022000 3f3f3f3f 3f3f3f3f 3f3f3f3f 3f3f3f3f 0x14000000
3f3f3f3f 00000000 00000000 00000000 00000000 0x3f3f3f3f

kd> ln f714a684
*** ERROR: Symbol file could not be found. Defaulted to export symbols
for
tcpip.sys -
(f71452c8) tcpip!tcpxsum+0x53bc | (f715ee00) tcpip!IPInjectPkt

You are currently subscribed to ntdev as: xxxxx@alacritech.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · July 9, 2002, 5:46am

> Hi,

I have a driver which sends/recvs data from network using TDI.
I have my own ClientEventReceive handler installed.

The driver works fine when only sending data out. However, it
will crash if i do mixed sending/recving for a while. This seems only
happen
when i run on a gigabit ethernet. For 100Mbit network, very rare, although
i did see the problem once.

The bugcheck happened in TcpChecksum() or sometimes TcpFreePartialRB().
I have checked my code many times to see if there is potential memory
violation
in my code but could not find any.

I had the same bugcheck but in filter driver when didn’t properly process
STATUS_INSUFFICIENT_RESOURCES in original ClientEventConnect handler.
Problems seem to be different but the result is the same.

It’s thing to think…

OSR_Community_User · July 9, 2002, 4:43pm

> the default TDI receiving buffer should be big enough to hold
whatever data

Correct way of TDI receives:

rely on ClientEventReceive.
if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED,
when the buffer space will open a bit, send the TDI_RECEIVE IRP to
restart the pipeline.
then, if you have the complete packet indicated (BytesIndicated ==
BytesAvailable) - copy it to your buffer.
otherwise, create a TDI_RECEIVE IRP and return it from
ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.

In the latter case, the buffer size attached to the IRP must be at
least BytesAvailable. You can also specify a larger buffer, but it
will have interesting side effects. In this case, the protocol has no
data to fill the whole large buffer. It can fill only the part of it.
So, the protocol can perform optimization of pending the IRP till more
data will arrive. TCP pends such an IRP till a) it will be filled
completely OR b) a packet with PSH bit arrives OR c) some timeout
elapses - around 500ms.
According to RFCs, the sender must set PSH bit in the packet which is
the last packet in a sequence of packets created by a single send()
call (yes, RFCs allow explicit control over PSH from the app, but
usually it is not implemented, and the above heuristic is used). In
this case, PSH means “last packet for now, next packets will arrive
after indefinite time”.
Nevertheless, there are buggy senders which never set PSH. With such a
senders, TCP will possibly wait for 500ms each recv() in this case.

Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
either BytesAvailable or the AFD’s buffer size (larger).

Max

OSR_Community_User · July 9, 2002, 6:25pm

I start to suspect every piece of code in my driver now.

Can anyone tell me the difference between MmBuildMdlForNonPagedPool() and
MmProbeAndLockPages()? I have a receiving buffer which is allocated from
nonpagedpool, should i use MmBuildMdlForNonPagedPool to build the MDL
or just use MmProbeAndLockPages?

Thanks.

Wei

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, July 09, 2002 4:37 PM
To: NT Developers Interest List
Subject: [ntdev] Re: How big is the TDI receiving buffer?

the default TDI receiving buffer should be big enough to hold
whatever data

Correct way of TDI receives:

rely on ClientEventReceive.
if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED,
when the buffer space will open a bit, send the TDI_RECEIVE IRP to
restart the pipeline.
then, if you have the complete packet indicated (BytesIndicated ==
BytesAvailable) - copy it to your buffer.
otherwise, create a TDI_RECEIVE IRP and return it from
ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.

In the latter case, the buffer size attached to the IRP must be at
least BytesAvailable. You can also specify a larger buffer, but it
will have interesting side effects. In this case, the protocol has no
data to fill the whole large buffer. It can fill only the part of it.
So, the protocol can perform optimization of pending the IRP till more
data will arrive. TCP pends such an IRP till a) it will be filled
completely OR b) a packet with PSH bit arrives OR c) some timeout
elapses - around 500ms.
According to RFCs, the sender must set PSH bit in the packet which is
the last packet in a sequence of packets created by a single send()
call (yes, RFCs allow explicit control over PSH from the app, but
usually it is not implemented, and the above heuristic is used). In
this case, PSH means “last packet for now, next packets will arrive
after indefinite time”.
Nevertheless, there are buggy senders which never set PSH. With such a
senders, TCP will possibly wait for 500ms each recv() in this case.

Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
either BytesAvailable or the AFD’s buffer size (larger).

Max

You are currently subscribed to ntdev as: xxxxx@emc.com
To unsubscribe send a blank email to %%email.unsub%%

Dan_Partelly · July 9, 2002, 6:48pm

MmBuildMdlForNonPagedPool.

----- Original Message -----
From: “chen, wei”
To: “NT Developers Interest List”
Sent: Wednesday, July 10, 2002 1:21 AM
Subject: [ntdev] Re: How big is the TDI receiving buffer?

> I start to suspect every piece of code in my driver now.
>
> Can anyone tell me the difference between MmBuildMdlForNonPagedPool() and
> MmProbeAndLockPages()? I have a receiving buffer which is allocated from
> nonpagedpool, should i use MmBuildMdlForNonPagedPool to build the MDL
> or just use MmProbeAndLockPages?
>
> Thanks.
>
> Wei
>
>
>
>
> -----Original Message-----
> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> Sent: Tuesday, July 09, 2002 4:37 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: How big is the TDI receiving buffer?
>
>
> > the default TDI receiving buffer should be big enough to hold
> whatever data
>
> Correct way of TDI receives:
> - rely on ClientEventReceive.
> - if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED,
> when the buffer space will open a bit, send the TDI_RECEIVE IRP to
> restart the pipeline.
> - then, if you have the complete packet indicated (BytesIndicated ==
> BytesAvailable) - copy it to your buffer.
> - otherwise, create a TDI_RECEIVE IRP and return it from
> ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.
>
> In the latter case, the buffer size attached to the IRP must be at
> least BytesAvailable. You can also specify a larger buffer, but it
> will have interesting side effects. In this case, the protocol has no
> data to fill the whole large buffer. It can fill only the part of it.
> So, the protocol can perform optimization of pending the IRP till more
> data will arrive. TCP pends such an IRP till a) it will be filled
> completely OR b) a packet with PSH bit arrives OR c) some timeout
> elapses - around 500ms.
> According to RFCs, the sender must set PSH bit in the packet which is
> the last packet in a sequence of packets created by a single send()
> call (yes, RFCs allow explicit control over PSH from the app, but
> usually it is not implemented, and the above heuristic is used). In
> this case, PSH means “last packet for now, next packets will arrive
> after indefinite time”.
> Nevertheless, there are buggy senders which never set PSH. With such a
> senders, TCP will possibly wait for 500ms each recv() in this case.
>
> Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
> size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
> either BytesAvailable or the AFD’s buffer size (larger).
>
> Max
>
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@emc.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>

OSR_Community_User · July 10, 2002, 7:58am

> nonpagedpool, should i use MmBuildMdlForNonPagedPool to build the
MDL

or just use MmProbeAndLockPages?

Can do both. Note that MmProbeAndLockPages requires MmUnlockPages,
while there is no reciprocal for MmBuildMdlForNonPagedPool.

MmBuildMdlForNonPagedPool just converts the virtual addresses to
physical (for NPP, this is simple) and fills the MDL tail with
physical addresses.
MmProbeAndLockPages also fills the MDL tail with physical addresses,
but it faults in all these pages, which requires IO, waits and adding
the page’s reference counts to avoid them from being outswapped while
they are in the MDL.

Max

OSR_Community_User · July 10, 2002, 11:49am

Hi, Shatskish specially,

In my ClientEventReceive handler, i never consume data even if they
are available. I just post another receiving IRP asking TDI to
fill the MDL for me. In my completion routine associated with this
IRP, i might again post receiving IRP depending on the fact that
i get the packet head or packet body(the actual logic is more complicated,
I have completion routines for packet head and packet body).

My question is:

Is it ok for me just post receiving IRP instead of copying the avalable
data to my receiving buffer directly? I can tolerate some performance
penalty.
In my IRP, I always indicate i want read more bytes than BytesAvailable,
hoping
somehow at the time i post the request, all the data are already
there.
I do take care the situation that less than what i expected bytes
might be returned.
In that case, I use IoBuildPartialMdl against my original MDL and
post another
receiving request and so on so forth.
Is it a right way to get the thing done?

Again, my approach seems working with 100Mbit network. On Gigabit network,
If i don’t receive too much data, meaning most of my operations are sending,

it works. Only when i start to receive large amount of data, the TCPIP.sys
will crash. Coult it be that i posted too many receiving IRPs?

Thanks.

Wei

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, July 09, 2002 4:37 PM
To: NT Developers Interest List
Subject: [ntdev] Re: How big is the TDI receiving buffer?

the default TDI receiving buffer should be big enough to hold
whatever data

Correct way of TDI receives:

rely on ClientEventReceive.
if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED,
when the buffer space will open a bit, send the TDI_RECEIVE IRP to
restart the pipeline.
then, if you have the complete packet indicated (BytesIndicated ==
BytesAvailable) - copy it to your buffer.
otherwise, create a TDI_RECEIVE IRP and return it from
ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.

In the latter case, the buffer size attached to the IRP must be at
least BytesAvailable. You can also specify a larger buffer, but it
will have interesting side effects. In this case, the protocol has no
data to fill the whole large buffer. It can fill only the part of it.
So, the protocol can perform optimization of pending the IRP till more
data will arrive. TCP pends such an IRP till a) it will be filled
completely OR b) a packet with PSH bit arrives OR c) some timeout
elapses - around 500ms.
According to RFCs, the sender must set PSH bit in the packet which is
the last packet in a sequence of packets created by a single send()
call (yes, RFCs allow explicit control over PSH from the app, but
usually it is not implemented, and the above heuristic is used). In
this case, PSH means “last packet for now, next packets will arrive
after indefinite time”.
Nevertheless, there are buggy senders which never set PSH. With such a
senders, TCP will possibly wait for 500ms each recv() in this case.

Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
either BytesAvailable or the AFD’s buffer size (larger).

Max

You are currently subscribed to ntdev as: xxxxx@emc.com
To unsubscribe send a blank email to %%email.unsub%%

Mark_S_Edwards · July 10, 2002, 12:27pm

Wei,

There are known faults in TCPIP.SYS with Gigabit Ethernet and high data
throughput. Have a look for Q318271, “Problems with Very High Data Rates
Over TCP/IP with Windows 2000”.

Having said that, even with this patch TCPIP.SYS can still be flakey if you
start messing with the TCP Window sizes. I’ve crashed it several times
under high and sometimes not so-high stress.

Mark

At 11:44 10/07/2002 -0400, chen, wei wrote:

Hi, Shatskish specially,

In my ClientEventReceive handler, i never consume data even if they
are available. I just post another receiving IRP asking TDI to
fill the MDL for me. In my completion routine associated with this
IRP, i might again post receiving IRP depending on the fact that
i get the packet head or packet body(the actual logic is more complicated,
I have completion routines for packet head and packet body).

My question is:

Is it ok for me just post receiving IRP instead of copying the avalable
data to my receiving buffer directly? I can tolerate some performance
penalty.

In my IRP, I always indicate i want read more bytes than BytesAvailable,
hoping
somehow at the time i post the request, all the data are already
there.
I do take care the situation that less than what i expected bytes
might be returned.
In that case, I use IoBuildPartialMdl against my original MDL and
post another
receiving request and so on so forth.
Is it a right way to get the thing done?

Again, my approach seems working with 100Mbit network. On Gigabit network,
If i don’t receive too much data, meaning most of my operations are sending,

it works. Only when i start to receive large amount of data, the TCPIP.sys
will crash. Coult it be that i posted too many receiving IRPs?

Thanks.

Wei

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, July 09, 2002 4:37 PM
To: NT Developers Interest List
Subject: [ntdev] Re: How big is the TDI receiving buffer?

> the default TDI receiving buffer should be big enough to hold
whatever data

Correct way of TDI receives:

rely on ClientEventReceive.

if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED,
when the buffer space will open a bit, send the TDI_RECEIVE IRP to
restart the pipeline.

then, if you have the complete packet indicated (BytesIndicated ==
BytesAvailable) - copy it to your buffer.

otherwise, create a TDI_RECEIVE IRP and return it from
ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.

In the latter case, the buffer size attached to the IRP must be at
least BytesAvailable. You can also specify a larger buffer, but it
will have interesting side effects. In this case, the protocol has no
data to fill the whole large buffer. It can fill only the part of it.
So, the protocol can perform optimization of pending the IRP till more
data will arrive. TCP pends such an IRP till a) it will be filled
completely OR b) a packet with PSH bit arrives OR c) some timeout
elapses - around 500ms.
According to RFCs, the sender must set PSH bit in the packet which is
the last packet in a sequence of packets created by a single send()
call (yes, RFCs allow explicit control over PSH from the app, but
usually it is not implemented, and the above heuristic is used). In
this case, PSH means “last packet for now, next packets will arrive
after indefinite time”.
Nevertheless, there are buggy senders which never set PSH. With such a
senders, TCP will possibly wait for 500ms each recv() in this case.

Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
either BytesAvailable or the AFD’s buffer size (larger).

Max

You are currently subscribed to ntdev as: xxxxx@emc.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntdev as: xxxxx@muttsnuts.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · July 10, 2002, 12:56pm

> 1. Is it ok for me just post receiving IRP instead of copying the
avalable

data to my receiving buffer directly? I can tolerate some
performance
penalty.

Second post can really be not OK, since it can race with another
ClientEventReceive.

Is it a right way to get the thing done?

A right thing, I described it.

it works. Only when i start to receive large amount of data, the
TCPIP.sys
will crash. Coult it be that i posted too many receiving IRPs?

Posting a second receive IRP seems to be the problematic place.

Max

OSR_Community_User · July 10, 2002, 1:04pm

Mark,

I would be willing to look into this crash in TCPIP.SYS if you are
willing to send a crash dump. It would be good to have it reproed on
Windows XP checked build. However Windows XP free build would work as
well.

If interested zip up the crash dump as ehanson10.zip and place on
ftp://ftppss.microsoft.com/incoming/developer

You may find that ftp does not work due to a firewall issue. In this
case simply cut and past the ftp address into your browser and then use
drag and drop to place the file on our ftp server. - Eric

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Mark S. Edwards [mailto:xxxxx@muttsnuts.com]
Sent: Wednesday, July 10, 2002 9:23 AM
To: NT Developers Interest List
Subject: [ntdev] Re: How big is the TDI receiving buffer?

Wei,

There are known faults in TCPIP.SYS with Gigabit Ethernet and high data
throughput. Have a look for Q318271, “Problems with Very High Data
Rates
Over TCP/IP with Windows 2000”.

Having said that, even with this patch TCPIP.SYS can still be flakey if
you
start messing with the TCP Window sizes. I’ve crashed it several times
under high and sometimes not so-high stress.

Mark

At 11:44 10/07/2002 -0400, chen, wei wrote:

Hi, Shatskish specially,

In my ClientEventReceive handler, i never consume data even if they are

available. I just post another receiving IRP asking TDI to fill the MDL

for me. In my completion routine associated with this IRP, i might
again post receiving IRP depending on the fact that i get the packet
head or packet body(the actual logic is more complicated, I have
completion routines for packet head and packet body).

My question is:

Is it ok for me just post receiving IRP instead of copying the
avalable data to my receiving buffer directly? I can tolerate some
performance penalty.

In my IRP, I always indicate i want read more bytes than
BytesAvailable, hoping
somehow at the time i post the request, all the data are
already there.
I do take care the situation that less than what i expected
bytes might be returned.
In that case, I use IoBuildPartialMdl against my original MDL
and post another
receiving request and so on so forth.
Is it a right way to get the thing done?

Again, my approach seems working with 100Mbit network. On Gigabit
network, If i don’t receive too much data, meaning most of my
operations are sending,

it works. Only when i start to receive large amount of data, the
TCPIP.sys will crash. Coult it be that i posted too many receiving
IRPs?

Thanks.

Wei

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, July 09, 2002 4:37 PM
To: NT Developers Interest List
Subject: [ntdev] Re: How big is the TDI receiving buffer?

> the default TDI receiving buffer should be big enough to hold
whatever data

Correct way of TDI receives:

rely on ClientEventReceive.

if you’re out of buffer space - return STATUS_DATA_NOT_ACCEPTED, when

the buffer space will open a bit, send the TDI_RECEIVE IRP to restart
the pipeline.

then, if you have the complete packet indicated (BytesIndicated ==
BytesAvailable) - copy it to your buffer.

otherwise, create a TDI_RECEIVE IRP and return it from
ClientEventReceive with STATUS_MORE_PROCESSING_REQUIRED.

In the latter case, the buffer size attached to the IRP must be at
least BytesAvailable. You can also specify a larger buffer, but it will

have interesting side effects. In this case, the protocol has no data
to fill the whole large buffer. It can fill only the part of it. So,
the protocol can perform optimization of pending the IRP till more data

will arrive. TCP pends such an IRP till a) it will be filled completely

OR b) a packet with PSH bit arrives OR c) some timeout elapses - around

500ms. According to RFCs, the sender must set PSH bit in the packet
which is the last packet in a sequence of packets created by a single
send() call (yes, RFCs allow explicit control over PSH from the app,
but usually it is not implemented, and the above heuristic is used).
In this case, PSH means “last packet for now, next packets will arrive
after indefinite time”.
Nevertheless, there are buggy senders which never set PSH. With such a
senders, TCP will possibly wait for 500ms each recv() in this case.

Read MSDN for AfdIgnorePushBitOnReceives value, it governs the buffer
size for TDI_RECEIVE IRPs created by AFD’s ClientEventReceive. It is
either BytesAvailable or the AFD’s buffer size (larger).

Max

You are currently subscribed to ntdev as: xxxxx@emc.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntdev as: xxxxx@muttsnuts.com To
unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to %%email.unsub%%