TCP ACK of segments and timing

Hi,
I am trying to monitor how long it takes for a TCP send to
go over the wire. What I have noticed is the TCP driver on the
server machine will sometimes delay sending the ack of the last
segment of the send. What I really want to do is figure out
the time the client would have received the ack of the last
segment if TCP were not delaying acks. This is what RFC 1122
says about TCP delays:

4.2.3.2 When to Send an ACK Segment

A host that is receiving a stream of TCP data segments can
increase efficiency in both the Internet and the hosts by
sending fewer than one ACK (acknowledgment) segment per data
segment received; this is known as a “delayed ACK” [TCP:5].

A TCP SHOULD implement a delayed ACK, but an ACK should not
be excessively delayed; in particular, the delay MUST be
less than 0.5 seconds, and in a stream of full-sized
segments there SHOULD be an ACK for at least every second
segment.

I think I can do this by figuring out the delta between
when I see the previous segment hit the wire, and I see the
segment get acked. I can assume that this delta is the round
trip latency between the two hosts. Then when I see that I am
sending the last segment, I can add the previously computed
round trip latency to the time that the segment goes out and have
a computed end time.

My question is, will this work? If so what type of driver
should I write to do this? If not, what should I do to compute
this time?

Thanks!
-Jeff

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************

Can’t you use the timer built into Network Monitor to calculate these times?

Ed Lau

MidCore Software, Inc.
900 Straits Tpke
Middlebury, CT 06762

www.midcore.com

----- Original Message -----
From: Curless, Jeffrey
To: NT Developers Interest List
Sent: Wednesday, December 04, 2002 5:06 PM
Subject: [ntdev] TCP ACK of segments and timing

Hi,
I am trying to monitor how long it takes for a TCP send to
go over the wire. What I have noticed is the TCP driver on the
server machine will sometimes delay sending the ack of the last
segment of the send. What I really want to do is figure out
the time the client would have received the ack of the last
segment if TCP were not delaying acks. This is what RFC 1122
says about TCP delays:

4.2.3.2 When to Send an ACK Segment

A host that is receiving a stream of TCP data segments can
increase efficiency in both the Internet and the hosts by
sending fewer than one ACK (acknowledgment) segment per data
segment received; this is known as a “delayed ACK” [TCP:5].

A TCP SHOULD implement a delayed ACK, but an ACK should not
be excessively delayed; in particular, the delay MUST be
less than 0.5 seconds, and in a stream of full-sized
segments there SHOULD be an ACK for at least every second
segment.

I think I can do this by figuring out the delta between
when I see the previous segment hit the wire, and I see the
segment get acked. I can assume that this delta is the round
trip latency between the two hosts. Then when I see that I am
sending the last segment, I can add the previously computed
round trip latency to the time that the segment goes out and have
a computed end time.

My question is, will this work? If so what type of driver
should I write to do this? If not, what should I do to compute
this time?

Thanks!
-Jeff

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************


You are currently subscribed to ntdev as: xxxxx@midcore.com
To unsubscribe send a blank email to %%email.unsub%%

No I can’t, because I need this to run on a non-server platform and
I won’t necessarily have SMS installed.

This would be used on client machines to gather network information.

-Jeff

-----Original Message-----
From: Ed Lau [mailto:xxxxx@midcore.com]
Sent: Wednesday, December 04, 2002 5:25 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Can’t you use the timer built into Network Monitor to calculate these times?

Ed Lau

MidCore Software, Inc.
900 Straits Tpke
Middlebury, CT 06762

www.midcore.com

----- Original Message -----
From: Curless, Jeffrey
To: NT Developers Interest List
Sent: Wednesday, December 04, 2002 5:06 PM
Subject: [ntdev] TCP ACK of segments and timing

Hi,
I am trying to monitor how long it takes for a TCP send to
go over the wire. What I have noticed is the TCP driver on the
server machine will sometimes delay sending the ack of the last
segment of the send. What I really want to do is figure out
the time the client would have received the ack of the last
segment if TCP were not delaying acks. This is what RFC 1122
says about TCP delays:

4.2.3.2 When to Send an ACK Segment

A host that is receiving a stream of TCP data segments can
increase efficiency in both the Internet and the hosts by
sending fewer than one ACK (acknowledgment) segment per data
segment received; this is known as a “delayed ACK” [TCP:5].

A TCP SHOULD implement a delayed ACK, but an ACK should not
be excessively delayed; in particular, the delay MUST be
less than 0.5 seconds, and in a stream of full-sized
segments there SHOULD be an ACK for at least every second
segment.

I think I can do this by figuring out the delta between
when I see the previous segment hit the wire, and I see the
segment get acked. I can assume that this delta is the round
trip latency between the two hosts. Then when I see that I am
sending the last segment, I can add the previously computed
round trip latency to the time that the segment goes out and have
a computed end time.

My question is, will this work? If so what type of driver
should I write to do this? If not, what should I do to compute
this time?

Thanks!
-Jeff

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************


You are currently subscribed to ntdev as: xxxxx@midcore.com
To unsubscribe send a blank email to %%email.unsub%% —
You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

IIRC is the PSH bit is set, then delayed ACK mechanism is defunct and
the ACK will be sent immediately.

Max

----- Original Message -----
From: “Curless, Jeffrey”
To: “NT Developers Interest List”
Sent: Thursday, December 05, 2002 1:06 AM
Subject: [ntdev] TCP ACK of segments and timing

> Hi,
> I am trying to monitor how long it takes for a TCP send to
> go over the wire. What I have noticed is the TCP driver on the
> server machine will sometimes delay sending the ack of the last
> segment of the send. What I really want to do is figure out
> the time the client would have received the ack of the last
> segment if TCP were not delaying acks. This is what RFC 1122
> says about TCP delays:
>
> 4.2.3.2 When to Send an ACK Segment
>
> A host that is receiving a stream of TCP data segments
can
> increase efficiency in both the Internet and the hosts
by
> sending fewer than one ACK (acknowledgment) segment per
data
> segment received; this is known as a “delayed ACK”
[TCP:5].
>
> A TCP SHOULD implement a delayed ACK, but an ACK should
not
> be excessively delayed; in particular, the delay MUST be
> less than 0.5 seconds, and in a stream of full-sized
> segments there SHOULD be an ACK for at least every
second
> segment.
>
> I think I can do this by figuring out the delta between
> when I see the previous segment hit the wire, and I see the
> segment get acked. I can assume that this delta is the round
> trip latency between the two hosts. Then when I see that I am
> sending the last segment, I can add the previously computed
> round trip latency to the time that the segment goes out and have
> a computed end time.
>
> My question is, will this work? If so what type of driver
> should I write to do this? If not, what should I do to compute
> this time?
>
> Thanks!
> -Jeff
>
>
>

> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please
notify
> the system manager.
>
> This footnote also confirms that this email message has been swept
by
> the latest virus scan software available for the presence of
computer
> viruses.
>

>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>

Network Monitor can work on clients too, but can only monitor incoming
and outgoing traffic, not the passing-by traffic.

Max

----- Original Message -----
From: “Curless, Jeffrey”
To: “NT Developers Interest List”
Sent: Thursday, December 05, 2002 2:04 AM
Subject: [ntdev] Re: TCP ACK of segments and timing

> No I can’t, because I need this to run on a non-server platform and
> I won’t necessarily have SMS installed.
>
> This would be used on client machines to gather network information.
>
> -Jeff
>
>
> -----Original Message-----
> From: Ed Lau [mailto:xxxxx@midcore.com]
> Sent: Wednesday, December 04, 2002 5:25 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: TCP ACK of segments and timing
>
>
> Can’t you use the timer built into Network Monitor to calculate
these times?
>
> Ed Lau
>
> MidCore Software, Inc.
> 900 Straits Tpke
> Middlebury, CT 06762
>
> www.midcore.com
>
> ----- Original Message -----
> From: Curless, Jeffrey
> To: NT Developers Interest List
> Sent: Wednesday, December 04, 2002 5:06 PM
> Subject: [ntdev] TCP ACK of segments and timing
>
>
> Hi,
> I am trying to monitor how long it takes for a TCP send to
> go over the wire. What I have noticed is the TCP driver on the
> server machine will sometimes delay sending the ack of the last
> segment of the send. What I really want to do is figure out
> the time the client would have received the ack of the last
> segment if TCP were not delaying acks. This is what RFC 1122
> says about TCP delays:
>
> 4.2.3.2 When to Send an ACK Segment
>
> A host that is receiving a stream of TCP data segments
can
> increase efficiency in both the Internet and the hosts
by
> sending fewer than one ACK (acknowledgment) segment per
data
> segment received; this is known as a “delayed ACK”
[TCP:5].
>
> A TCP SHOULD implement a delayed ACK, but an ACK should
not
> be excessively delayed; in particular, the delay MUST be
> less than 0.5 seconds, and in a stream of full-sized
> segments there SHOULD be an ACK for at least every
second
> segment.
>
> I think I can do this by figuring out the delta between
> when I see the previous segment hit the wire, and I see the
> segment get acked. I can assume that this delta is the round
> trip latency between the two hosts. Then when I see that I am
> sending the last segment, I can add the previously computed
> round trip latency to the time that the segment goes out and have
> a computed end time.
>
> My question is, will this work? If so what type of driver
> should I write to do this? If not, what should I do to compute
> this time?
>
> Thanks!
> -Jeff
>
>
>

> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please
notify
> the system manager.
>
> This footnote also confirms that this email message has been swept
by
> the latest virus scan software available for the presence of
computer
> viruses.
>

>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@midcore.com
> To unsubscribe send a blank email to %%email.unsub%% —
> You are currently subscribed to ntdev as: xxxxx@concord.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>

Nai, why you need netmon if you can query those from OS yourself. netmon
does not give you those info programmatically anyway.

Bi

-----Original Message-----
From: Curless, Jeffrey [mailto:xxxxx@concord.com]
Sent: Wednesday, December 04, 2002 3:05 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

No I can’t, because I need this to run on a non-server platform and
I won’t necessarily have SMS installed.

This would be used on client machines to gather network information.

-Jeff

-----Original Message-----
From: Ed Lau [mailto:xxxxx@midcore.com]
Sent: Wednesday, December 04, 2002 5:25 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Can’t you use the timer built into Network Monitor to calculate these times?

Ed Lau

MidCore Software, Inc.
900 Straits Tpke
Middlebury, CT 06762

www.midcore.com

----- Original Message -----
From: Curless, Jeffrey
To: NT Developers Interest List
Sent: Wednesday, December 04, 2002 5:06 PM
Subject: [ntdev] TCP ACK of segments and timing

Hi,
I am trying to monitor how long it takes for a TCP send to
go over the wire. What I have noticed is the TCP driver on the
server machine will sometimes delay sending the ack of the last
segment of the send. What I really want to do is figure out
the time the client would have received the ack of the last
segment if TCP were not delaying acks. This is what RFC 1122
says about TCP delays:

4.2.3.2 When to Send an ACK Segment

A host that is receiving a stream of TCP data segments can
increase efficiency in both the Internet and the hosts by
sending fewer than one ACK (acknowledgment) segment per data
segment received; this is known as a “delayed ACK” [TCP:5].

A TCP SHOULD implement a delayed ACK, but an ACK should not
be excessively delayed; in particular, the delay MUST be
less than 0.5 seconds, and in a stream of full-sized
segments there SHOULD be an ACK for at least every second
segment.

I think I can do this by figuring out the delta between
when I see the previous segment hit the wire, and I see the
segment get acked. I can assume that this delta is the round
trip latency between the two hosts. Then when I see that I am
sending the last segment, I can add the previously computed
round trip latency to the time that the segment goes out and have
a computed end time.

My question is, will this work? If so what type of driver
should I write to do this? If not, what should I do to compute
this time?

Thanks!
-Jeff

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************


You are currently subscribed to ntdev as: xxxxx@midcore.com
To unsubscribe send a blank email to %%email.unsub%% —
You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%

This is from the microsoft Network Monitor SDK:

To call the Network Monitor API sets, you must be running on
Microsoft? Windows NT?
Server version 4.0 or Microsoft? Windows? 2000 Server, or have
Microsoft? Systems
Management Server installed.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/netmon/netm
on/network_monitor.asp

Is this incorrect?

I only want to monitor the incoming and outgoing traffic, I need to
do this so I can pass another process the times that I see so I can
do some reporting on it.

Also, since this is a performance monitoring application I can’t
affect the performance of the machine much, and I need to make
timestamps as close to the actual times as possible.

It doesn’t look to me like the network monitor API gets me what
I need in terms of ability to timestamp in the flow of requests.

-Jeff

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Wednesday, December 04, 2002 6:19 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Network Monitor can work on clients too, but can only monitor incoming
and outgoing traffic, not the passing-by traffic.

Max

----- Original Message -----
From: “Curless, Jeffrey”
To: “NT Developers Interest List”
Sent: Thursday, December 05, 2002 2:04 AM
Subject: [ntdev] Re: TCP ACK of segments and timing

> No I can’t, because I need this to run on a non-server platform and
> I won’t necessarily have SMS installed.
>
> This would be used on client machines to gather network information.
>
> -Jeff
>
>
> -----Original Message-----
> From: Ed Lau [mailto:xxxxx@midcore.com]
> Sent: Wednesday, December 04, 2002 5:25 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: TCP ACK of segments and timing
>
>
> Can’t you use the timer built into Network Monitor to calculate
these times?
>
> Ed Lau
>
> MidCore Software, Inc.
> 900 Straits Tpke
> Middlebury, CT 06762
>
> www.midcore.com
>
> ----- Original Message -----
> From: Curless, Jeffrey
> To: NT Developers Interest List
> Sent: Wednesday, December 04, 2002 5:06 PM
> Subject: [ntdev] TCP ACK of segments and timing
>
>
> Hi,
> I am trying to monitor how long it takes for a TCP send to
> go over the wire. What I have noticed is the TCP driver on the
> server machine will sometimes delay sending the ack of the last
> segment of the send. What I really want to do is figure out
> the time the client would have received the ack of the last
> segment if TCP were not delaying acks. This is what RFC 1122
> says about TCP delays:
>
> 4.2.3.2 When to Send an ACK Segment
>
> A host that is receiving a stream of TCP data segments
can
> increase efficiency in both the Internet and the hosts
by
> sending fewer than one ACK (acknowledgment) segment per
data
> segment received; this is known as a “delayed ACK”
[TCP:5].
>
> A TCP SHOULD implement a delayed ACK, but an ACK should
not
> be excessively delayed; in particular, the delay MUST be
> less than 0.5 seconds, and in a stream of full-sized
> segments there SHOULD be an ACK for at least every
second
> segment.
>
> I think I can do this by figuring out the delta between
> when I see the previous segment hit the wire, and I see the
> segment get acked. I can assume that this delta is the round
> trip latency between the two hosts. Then when I see that I am
> sending the last segment, I can add the previously computed
> round trip latency to the time that the segment goes out and have
> a computed end time.
>
> My question is, will this work? If so what type of driver
> should I write to do this? If not, what should I do to compute
> this time?
>
> Thanks!
> -Jeff
>
>
>

> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please
notify
> the system manager.
>
> This footnote also confirms that this email message has been swept
by
> the latest virus scan software available for the presence of
computer
> viruses.
>

>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@midcore.com
> To unsubscribe send a blank email to %%email.unsub%% —
> You are currently subscribed to ntdev as: xxxxx@concord.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>


You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

As far as I can tell (from RFC 1122) the PSH bit only disables
the Nagle algorithm, and tells the receiver to wake up the
application without waiting.

It doesn’t mention anything about the delayed ACK mechanism

-Jeff

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Wednesday, December 04, 2002 6:14 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

IIRC is the PSH bit is set, then delayed ACK mechanism is defunct and
the ACK will be sent immediately.

Max

----- Original Message -----
From: “Curless, Jeffrey”
To: “NT Developers Interest List”
Sent: Thursday, December 05, 2002 1:06 AM
Subject: [ntdev] TCP ACK of segments and timing

> Hi,
> I am trying to monitor how long it takes for a TCP send to
> go over the wire. What I have noticed is the TCP driver on the
> server machine will sometimes delay sending the ack of the last
> segment of the send. What I really want to do is figure out
> the time the client would have received the ack of the last
> segment if TCP were not delaying acks. This is what RFC 1122
> says about TCP delays:
>
> 4.2.3.2 When to Send an ACK Segment
>
> A host that is receiving a stream of TCP data segments
can
> increase efficiency in both the Internet and the hosts
by
> sending fewer than one ACK (acknowledgment) segment per
data
> segment received; this is known as a “delayed ACK”
[TCP:5].
>
> A TCP SHOULD implement a delayed ACK, but an ACK should
not
> be excessively delayed; in particular, the delay MUST be
> less than 0.5 seconds, and in a stream of full-sized
> segments there SHOULD be an ACK for at least every
second
> segment.
>
> I think I can do this by figuring out the delta between
> when I see the previous segment hit the wire, and I see the
> segment get acked. I can assume that this delta is the round
> trip latency between the two hosts. Then when I see that I am
> sending the last segment, I can add the previously computed
> round trip latency to the time that the segment goes out and have
> a computed end time.
>
> My question is, will this work? If so what type of driver
> should I write to do this? If not, what should I do to compute
> this time?
>
> Thanks!
> -Jeff
>
>
>

> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please
notify
> the system manager.
>
> This footnote also confirms that this email message has been swept
by
> the latest virus scan software available for the presence of
computer
> viruses.
>

>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>


You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

> As far as I can tell (from RFC 1122) the PSH bit only disables

the Nagle algorithm

It has nothing to do with Nagle.

, and tells the receiver to wake up the
application without waiting.

Yes, it completes recv() immediately even if not all specified buffer
was filled. Otherwise, it will be completed either on timeout or when
the whole buffer will be filled.

Max

Max,

setting “IgnorePushBitOnReceives” has no effect under Windows XP (applied
both to \Tcp\Parameters & Afd\Parameters). The same 50-200ms delay in
“send-send-recv” logic. And to set push bit manually raw sockets required.
Or did i miss something and there is a way to set raise PSH on the packet
explicitly?

Anton Kolomyeytsev

IIRC is the PSH bit is set, then delayed ACK mechanism is defunct and
the ACK will be sent immediately.

Max

Jeff,

is there any way to disable delayed ACK? At least under Windows NT? To
have workaround on “send-send-recv” logic that for some reasons must be
preserved…

Anton Kolomyeytsev

As far as I can tell (from RFC 1122) the PSH bit only disables
the Nagle algorithm, and tells the receiver to wake up the
application without waiting.

It doesn’t mention anything about the delayed ACK mechanism

-Jeff

Yes there is, the TcpDelAckTicks value in the registry
can be set to 0 to disable the delay ack. My problem
is that my code will only run on clients, not on
servers since I will be measuring network connectivity
from the client to any server on the internet they go to.
So changing the local ack delay doesn’t do anything for me.

-Jeff

-----Original Message-----
From: Anton Kolomyeytsev [mailto:xxxxx@cooldev.com]
Sent: Thursday, December 05, 2002 6:47 AM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Jeff,

is there any way to disable delayed ACK? At least under Windows NT? To
have workaround on “send-send-recv” logic that for some reasons must be
preserved…

Anton Kolomyeytsev

As far as I can tell (from RFC 1122) the PSH bit only disables
the Nagle algorithm, and tells the receiver to wake up the
application without waiting.

It doesn’t mention anything about the delayed ACK mechanism

-Jeff


You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************

Anton

Are you using Winsock in user mode or TDI client in kernel mode?

In user mode, if you zero the send buffer on individual socket, it will
disable Nagle algorithem. In kerenl TDI client, Max said that there are
undocumented or special licensed way doing that.

Bi

-----Original Message-----
From: Anton Kolomyeytsev [mailto:xxxxx@cooldev.com]
Sent: Thursday, December 05, 2002 3:47 AM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Jeff,

is there any way to disable delayed ACK? At least under Windows NT? To
have workaround on “send-send-recv” logic that for some reasons must be
preserved…

Anton Kolomyeytsev

As far as I can tell (from RFC 1122) the PSH bit only disables
the Nagle algorithm, and tells the receiver to wake up the
application without waiting.

It doesn’t mention anything about the delayed ACK mechanism

-Jeff


You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%

Bi,

we have intermediate level that wraps own calls to OS-specific primitives.
In NT user mode it maps internal API to sockets, in NT kernel mode it maps
them to TDI client calls (to make the code portable). I think this is the
way everybody writes portable software…

User mode works fine w/o any efforts. Kernel has “send-send-recv” problem.
We’ve found a workaround but it requires double buffering we’d like to
avoid.
Looks like AFD does double buffering as well, but it works more effective
then we.

We’ve tried to disable Nagle but it does not help. It seems to me that
delayed ACK is the source of our problems and not Nagle. What do you do to
disable Nagle? Can you send me (maybe in private) source code sample?
Maybe we’re doing something wrong here? ((

Thanks!

Anton Kolomyeytsev

Anton

Are you using Winsock in user mode or TDI client in kernel mode?

In user mode, if you zero the send buffer on individual socket, it will
disable Nagle algorithem. In kerenl TDI client, Max said that there are
undocumented or special licensed way doing that.

Bi

Delayed ack does sound like your problem. But remember that TCP was
designed that way on purpose, why do you want to work around it?
The reason for the delayed ack is to help lower network congestion.

-Jeff

-----Original Message-----
From: Anton Kolomyeytsev [mailto:xxxxx@cooldev.com]
Sent: Thursday, December 05, 2002 4:26 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Bi,

we have intermediate level that wraps own calls to OS-specific primitives.
In NT user mode it maps internal API to sockets, in NT kernel mode it maps
them to TDI client calls (to make the code portable). I think this is the
way everybody writes portable software…

User mode works fine w/o any efforts. Kernel has “send-send-recv” problem.
We’ve found a workaround but it requires double buffering we’d like to
avoid.
Looks like AFD does double buffering as well, but it works more effective
then we.

We’ve tried to disable Nagle but it does not help. It seems to me that
delayed ACK is the source of our problems and not Nagle. What do you do to
disable Nagle? Can you send me (maybe in private) source code sample?
Maybe we’re doing something wrong here? ((

Thanks!

Anton Kolomyeytsev

Anton

Are you using Winsock in user mode or TDI client in kernel mode?

In user mode, if you zero the send buffer on individual socket, it will
disable Nagle algorithem. In kerenl TDI client, Max said that there are
undocumented or special licensed way doing that.

Bi


You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************

Anton:

I agree with Jeff. If you have control over server and client software,
maybe UPD is your option since it is now up to you to decide what kind of
network io optimization. If you cannot use UDP, maybe disabling Nagle
algorithm on both client and server side and change send-send-receive to
send-receive model to force acknowledge immediately. This may increase
network congestion dramatically if you have to send many small packets.

Currently, I have similar problem because the server implementation. I’ll
get my client to work first (combo of WinSocket and TDI client) and then
I’ll modify both client and server to make it work better without forcefully
turn off TCP traffic optimization.

Bi

-----Original Message-----
From: Curless, Jeffrey [mailto:xxxxx@concord.com]
Sent: Thursday, December 05, 2002 1:58 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Delayed ack does sound like your problem. But remember that TCP was
designed that way on purpose, why do you want to work around it?
The reason for the delayed ack is to help lower network congestion.

-Jeff

-----Original Message-----
From: Anton Kolomyeytsev [mailto:xxxxx@cooldev.com]
Sent: Thursday, December 05, 2002 4:26 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Bi,

we have intermediate level that wraps own calls to OS-specific primitives.
In NT user mode it maps internal API to sockets, in NT kernel mode it maps
them to TDI client calls (to make the code portable). I think this is the
way everybody writes portable software…

User mode works fine w/o any efforts. Kernel has “send-send-recv” problem.
We’ve found a workaround but it requires double buffering we’d like to
avoid.
Looks like AFD does double buffering as well, but it works more effective
then we.

We’ve tried to disable Nagle but it does not help. It seems to me that
delayed ACK is the source of our problems and not Nagle. What do you do to
disable Nagle? Can you send me (maybe in private) source code sample?
Maybe we’re doing something wrong here? ((

Thanks!

Anton Kolomyeytsev

Anton

Are you using Winsock in user mode or TDI client in kernel mode?

In user mode, if you zero the send buffer on individual socket, it will
disable Nagle algorithem. In kerenl TDI client, Max said that there are
undocumented or special licensed way doing that.

Bi


You are currently subscribed to ntdev as: xxxxx@concord.com
To unsubscribe send a blank email to %%email.unsub%%

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************


You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%

Hello Bi,

I’m strugging with Nagle too and have written the following
piece of code. It uses undocumented IOCTL to tcpip.sys to set extended
connection parameters. Some of the code was stolen from winsock helper
sample. The code returns STATUS_SUCCESS, but I don’t see any difference
in TCP timings.

Any suggestions why it doesn’t work?

Thank you.

IOCTL is sent to \Device\Tcp

{
Connect();

int OptionValue;
//
// Set TCP_NODELAY option
//
OptionValue = TRUE;

Status = SetTdiInformation(
TransportDeviceObject,
ConnectionFileObject,
CO_TL_ENTITY,
INFO_CLASS_PROTOCOL,
INFO_TYPE_CONNECTION,
TCP_SOCKET_NODELAY,
&OptionValue,
sizeof(OptionValue)
);
}

BOOLEAN TdiDeviceIoControl(
IN PDEVICE_OBJECT DeviceObject,
IN PFILE_OBJECT FileObject,
IN ULONG IoControlCode,
IN PVOID InputBuffer,
IN ULONG InputBufferLength,
OUT PVOID OutputBuffer,
IN ULONG OutputBufferLength,
OUT PULONG OutputReturned
)
{
PIRP Irp;
NTSTATUS Status;
KEVENT Event;
IO_STATUS_BLOCK IoStatusBlock;
PIO_STACK_LOCATION StackLocation;

RtlZeroMemory(&IoStatusBlock, sizeof(IoStatusBlock));

KeInitializeEvent(&Event, NotificationEvent, FALSE);

Irp = IoBuildDeviceIoControlRequest(
IoControlCode,
DeviceObject,
InputBuffer,
InputBufferLength,
OutputBuffer,
OutputBufferLength,
FALSE,
&Event,
&IoStatusBlock
);

if(Irp == NULL)
{
DbgPrint(“TdiDeviceIoControl: Unable to allocate IRP\n”);
return FALSE;
}

StackLocation = IoGetNextIrpStackLocation( Irp );
StackLocation->FileObject = FileObject;

//
// Send Irp to the Device Object
//
Status = IoCallDriver(DeviceObject, Irp);
if(Status == STATUS_PENDING)
{
KeWaitForSingleObject(&Event, Executive, KernelMode, FALSE, NULL);
}
else if(Status != STATUS_SUCCESS)
{
DbgPrint(“TdiDeviceIoControl: Status = %x\n”, Status);
return FALSE;
}

//
// Get Irp Final status
//
if( IoStatusBlock.Status != STATUS_SUCCESS)
{
DbgPrint(“TdiDeviceIoControl: Status = %x, Irp Status = 0x%08x\n”, Status, IoStatusBlock.Status);
return FALSE;
}

if(OutputReturned != NULL)
*OutputReturned = IoStatusBlock.Information;

return TRUE;
}

NTSTATUS
SetTdiInformation (
IN PDEVICE_OBJECT TdiDeviceObject,
IN PFILE_OBJECT TdiFileObject,
IN ULONG Entity,
IN ULONG Class,
IN ULONG Type,
IN ULONG Id,
IN PVOID Value,
IN ULONG ValueLength
)

/*++

Routine Description:

Performs a TDI action to the TCP/IP driver. A TDI action translates
into a streams T_OPTMGMT_REQ.

Arguments:

TdiConnectionObjectHandle - a TDI connection object on which to perform
the TDI action.

Entity - value to put in the tei_entity field of the TDIObjectID
structure.

Class - value to put in the toi_class field of the TDIObjectID
structure.

Type - value to put in the toi_type field of the TDIObjectID
structure.

Id - value to put in the toi_id field of the TDIObjectID structure.

Value - a pointer to a buffer to set as the information.

ValueLength - the length of the buffer.

Return Value:

–*/

{
BOOLEAN CallResult;
PTCP_REQUEST_SET_INFORMATION_EX SetInfoEx;
ULONG BufferSize;

BufferSize = sizeof(TCP_REQUEST_SET_INFORMATION_EX) +
ValueLength - 1;

SetInfoEx = ExAllocatePool(NonPagedPool, BufferSize);

if(SetInfoEx == NULL)
{
DbgPrint(“SetTdiInformation is unable to allocate memory %d bytes\n”, BufferSize);
return STATUS_NO_MEMORY;
}

SetInfoEx->ID.toi_entity.tei_entity = Entity;
SetInfoEx->ID.toi_entity.tei_instance = TL_INSTANCE;
SetInfoEx->ID.toi_class = Class;
SetInfoEx->ID.toi_type = Type;
SetInfoEx->ID.toi_id = Id;

RtlCopyMemory( SetInfoEx->Buffer, Value, ValueLength );
SetInfoEx->BufferSize = ValueLength;

CallResult = TdiDeviceIoControl(
TdiDeviceObject,
TdiFileObject,
IOCTL_TCP_SET_INFORMATION_EX,
SetInfoEx,
BufferSize,
NULL,
0,
NULL
);

ExFreePool(SetInfoEx);
if( !CallResult)
{
DbgPrint(“SetTdiInformation: DeviceIoControl failed\n”);
return STATUS_UNSUCCESSFUL;
}

return STATUS_SUCCESS;
}

BC> Are you using Winsock in user mode or TDI client in kernel mode?
BC> In user mode, if you zero the send buffer on individual socket, it will
BC> disable Nagle algorithem. In kerenl TDI client, Max said that there are
BC> undocumented or special licensed way doing that.

Best regards,
Dmytro Pohromskyy mailto:xxxxx@synclive.com


Softick Card Export - Palm’s card as disk. Simple and powerful.
http://www.softick.com

Thanks Dmytro.

I’ll try this after my vacation. I think I’ll trace down setsockopt with
TCP_NODELAY to see what is the proper paramsters. By looking at your code, I
wonder if change entity to CO_TL_TCP will help.

Bi

-----Original Message-----
From: Dmytro Pohromskyy [mailto:xxxxx@synclive.com]
Sent: Thursday, December 05, 2002 3:45 PM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Hello Bi,

I’m strugging with Nagle too and have written the following
piece of code. It uses undocumented IOCTL to tcpip.sys to set extended
connection parameters. Some of the code was stolen from winsock helper
sample. The code returns STATUS_SUCCESS, but I don’t see any difference
in TCP timings.

Any suggestions why it doesn’t work?

Thank you.

IOCTL is sent to \Device\Tcp

{
Connect();

int OptionValue;
//
// Set TCP_NODELAY option
//
OptionValue = TRUE;

Status = SetTdiInformation(
TransportDeviceObject,
ConnectionFileObject,
CO_TL_ENTITY,
INFO_CLASS_PROTOCOL,
INFO_TYPE_CONNECTION,
TCP_SOCKET_NODELAY,
&OptionValue,
sizeof(OptionValue)
);
}

BOOLEAN TdiDeviceIoControl(
IN PDEVICE_OBJECT DeviceObject,
IN PFILE_OBJECT FileObject,
IN ULONG IoControlCode,
IN PVOID InputBuffer,
IN ULONG InputBufferLength,
OUT PVOID OutputBuffer,
IN ULONG OutputBufferLength,
OUT PULONG OutputReturned
)
{
PIRP Irp;
NTSTATUS Status;
KEVENT Event;
IO_STATUS_BLOCK IoStatusBlock;
PIO_STACK_LOCATION StackLocation;

RtlZeroMemory(&IoStatusBlock, sizeof(IoStatusBlock));

KeInitializeEvent(&Event, NotificationEvent, FALSE);

Irp = IoBuildDeviceIoControlRequest(
IoControlCode,
DeviceObject,
InputBuffer,
InputBufferLength,
OutputBuffer,
OutputBufferLength,
FALSE,
&Event,
&IoStatusBlock
);

if(Irp == NULL)
{
DbgPrint(“TdiDeviceIoControl: Unable to allocate IRP\n”);
return FALSE;
}

StackLocation = IoGetNextIrpStackLocation( Irp );
StackLocation->FileObject = FileObject;

//
// Send Irp to the Device Object
//
Status = IoCallDriver(DeviceObject, Irp);
if(Status == STATUS_PENDING)
{
KeWaitForSingleObject(&Event, Executive, KernelMode, FALSE, NULL);
}
else if(Status != STATUS_SUCCESS)
{
DbgPrint(“TdiDeviceIoControl: Status = %x\n”, Status);
return FALSE;
}

//
// Get Irp Final status
//
if( IoStatusBlock.Status != STATUS_SUCCESS)
{
DbgPrint(“TdiDeviceIoControl: Status = %x, Irp Status = 0x%08x\n”,
Status, IoStatusBlock.Status);
return FALSE;
}

if(OutputReturned != NULL)
*OutputReturned = IoStatusBlock.Information;

return TRUE;
}

NTSTATUS
SetTdiInformation (
IN PDEVICE_OBJECT TdiDeviceObject,
IN PFILE_OBJECT TdiFileObject,
IN ULONG Entity,
IN ULONG Class,
IN ULONG Type,
IN ULONG Id,
IN PVOID Value,
IN ULONG ValueLength
)

/*++

Routine Description:

Performs a TDI action to the TCP/IP driver. A TDI action translates
into a streams T_OPTMGMT_REQ.

Arguments:

TdiConnectionObjectHandle - a TDI connection object on which to perform
the TDI action.

Entity - value to put in the tei_entity field of the TDIObjectID
structure.

Class - value to put in the toi_class field of the TDIObjectID
structure.

Type - value to put in the toi_type field of the TDIObjectID
structure.

Id - value to put in the toi_id field of the TDIObjectID structure.

Value - a pointer to a buffer to set as the information.

ValueLength - the length of the buffer.

Return Value:

–*/

{
BOOLEAN CallResult;
PTCP_REQUEST_SET_INFORMATION_EX SetInfoEx;
ULONG BufferSize;

BufferSize = sizeof(TCP_REQUEST_SET_INFORMATION_EX) +
ValueLength - 1;

SetInfoEx = ExAllocatePool(NonPagedPool, BufferSize);

if(SetInfoEx == NULL)
{
DbgPrint(“SetTdiInformation is unable to allocate memory %d
bytes\n”, BufferSize);
return STATUS_NO_MEMORY;
}

SetInfoEx->ID.toi_entity.tei_entity = Entity;
SetInfoEx->ID.toi_entity.tei_instance = TL_INSTANCE;
SetInfoEx->ID.toi_class = Class;
SetInfoEx->ID.toi_type = Type;
SetInfoEx->ID.toi_id = Id;

RtlCopyMemory( SetInfoEx->Buffer, Value, ValueLength );
SetInfoEx->BufferSize = ValueLength;

CallResult = TdiDeviceIoControl(
TdiDeviceObject,
TdiFileObject,
IOCTL_TCP_SET_INFORMATION_EX,
SetInfoEx,
BufferSize,
NULL,
0,
NULL
);

ExFreePool(SetInfoEx);
if( !CallResult)
{
DbgPrint(“SetTdiInformation: DeviceIoControl failed\n”);
return STATUS_UNSUCCESSFUL;
}

return STATUS_SUCCESS;
}

BC> Are you using Winsock in user mode or TDI client in kernel mode?
BC> In user mode, if you zero the send buffer on individual socket, it will
BC> disable Nagle algorithem. In kerenl TDI client, Max said that there are
BC> undocumented or special licensed way doing that.

Best regards,
Dmytro Pohromskyy mailto:xxxxx@synclive.com


Softick Card Export - Palm’s card as disk. Simple and powerful.
http://www.softick.com


You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%

I do agree with both of you. But we are writing only netowork layer code.
We do not have control over the client and server source code. And we need
to make the existing source code use TCP, use “send-send-recv” logic and
work fast. At least not slower then the same code running in user mode.
That’s all…

Anton Kolomyeytsev

Anton:

I agree with Jeff. If you have control over server and client software,
maybe UPD is your option since it is now up to you to decide what kind of
network io optimization. If you cannot use UDP, maybe disabling Nagle
algorithm on both client and server side and change send-send-receive to
send-receive model to force acknowledge immediately. This may increase
network congestion dramatically if you have to send many small packets.

Currently, I have similar problem because the server implementation. I’ll
get my client to work first (combo of WinSocket and TDI client) and then
I’ll modify both client and server to make it work better without forcefully
turn off TCP traffic optimization.

Bi

RE: [ntdev] Re: TCP ACK of segments and timingNo. Do not mix the 2 different things.

SO_SNDBUF = 0 disables the AFD’s buffering, and turns each send() to TDI_SEND. Advantages - no memcpy() in send path. Drawbacks - send() will wait for all ACKs to arrive, which is suboptimal operation mode for a connection, especially with small sends. In fact, this can only be advised together with overlapped WriteFile or async WSASend, not with send(), and surely with large enough requests.

Nagle algorithm is absolutely different beast, it is in TCPIP and not in AFD.

Max

----- Original Message -----
From: Bi Chen
To: NT Developers Interest List
Sent: Thursday, December 05, 2002 9:08 PM
Subject: [ntdev] Re: TCP ACK of segments and timing

Anton

Are you using Winsock in user mode or TDI client in kernel mode?

In user mode, if you zero the send buffer on individual socket, it will disable Nagle algorithem. In kerenl TDI client, Max said that there are undocumented or special licensed way doing that.

Bi

-----Original Message-----
From: Anton Kolomyeytsev [mailto:xxxxx@cooldev.com]
Sent: Thursday, December 05, 2002 3:47 AM
To: NT Developers Interest List
Subject: [ntdev] Re: TCP ACK of segments and timing

Jeff,

is there any way to disable delayed ACK? At least under Windows NT? To
have workaround on “send-send-recv” logic that for some reasons must be
preserved…

Anton Kolomyeytsev

As far as I can tell (from RFC 1122) the PSH bit only disables
> the Nagle algorithm, and tells the receiver to wake up the
> application without waiting.
>
> It doesn’t mention anything about the delayed ACK mechanism
>
> -Jeff


You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to %%email.unsub%%