Losing Received Data?

OSR_Community_User · October 11, 2004, 9:31am

Hi,

The story so far:
I have succesfully installed a serial upper filter driver and can see the
various IOCTLs from the application. I am passing all IOCTLs down to the
lower device and HyperTerminal confirms read and write data is passing
through the driver to the attached modem correctly.

The problem:
When I use the serial port to set up a Windows “Dial Up Networking” session
via the modem, I do NOT see all of the received data in my completion
routine (I now have the correct data count). Furthermore, the connection
fails, which suggests that the modem may not be seeing all of the data
either. Clearly, somewhere I have broken something since the modem connects
perfectly when my driver is not installed.

My only clue so far is when I monitor the serial port using Mark
Russinovich’s PortMonitor utility I can see that the received data does not
always arrive in one “lump” (for want of a better term), but 2 or 3 read
requests are required to obtain the whole incoming TCP/IP packet. I do NOT
see this behaviour in my driver, I only see one received “lump” which always
consists of the start of a TCP/IP packet. I do not see the middle or the end
of the packet.

As usual, I’m in the dark, and any suggestions are gratefully received.

Thanks

Alasdair

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

OSR_Community_User · October 11, 2004, 9:52am

xxxxx@lists.osr.com wrote on 10/11/2004 02:29:11 PM:

Hi,

The story so far:
I have succesfully installed a serial upper filter driver and can
see the various IOCTLs from the application. I am passing all IOCTLs
down to the lower device and HyperTerminal confirms read and write
data is passing through the driver to the attached modem correctly.

The problem:
When I use the serial port to set up a Windows “Dial Up Networking”
session via the modem, I do NOT see all of the received data in my
completion routine (I now have the correct data count). Furthermore,
the connection fails, which suggests that the modem may not be
seeing all of the data either. Clearly, somewhere I have broken
something since the modem connects perfectly when my driver is not
installed.

My only clue so far is when I monitor the serial port using Mark
Russinovich’s PortMonitor utility I can see that the received data
does not always arrive in one “lump” (for want of a better term),
but 2 or 3 read requests are required to obtain the whole incoming
TCP/IP packet. I do NOT see this behaviour in my driver, I only see
one received “lump” which always consists of the start of a TCP/IP
packet. I do not see the middle or the end of the packet.

Some “stupid” questions (I don’t know much about serial filter drivers, but
I may be able to ask the “right” question so that we together find a
solution):

Do you know (from experience, previous packets, or something) the
supposed length of the expected packet when this happens?
Do you loose packets all the time, or just under some circumstances?
How do you know that you only see the beginning of a packet, rather than
seeing for instance the MIDDLE or END of a packet?
Do you know for sure that the reason that you’re loosing packets isn’t
the amount of time spent in your filter driver (i.e. do you do something in
your filter driver that may cause the maximum baudrate allowed to drop
dramatically, for instance long delays, debug printouts, or some such)?
Have you tried running a slower baud-rate? Also what baud-rate ARE you
running?

If you’re doing printouts from your filter driver, consider buffering them
into a “block” of some sort, and then use the debugger to fetch it later.
I’ve certainly seen things go wrong in our display driver when writing
excessive amounts of data from functions. Note: Display drivers are
generally not time-critical, whilst serial ports definitely ARE
time-critical, so you’ll have to be careful with the amount of time you use
in the filter function.

–
Mats

As usual, I’m in the dark, and any suggestions are gratefully received.

Thanks

Alasdair

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

ForwardSourceID:NT000051C2

OSR_Community_User · October 11, 2004, 10:12am

Hi Mats,

Thanks for the reply, you have asked some interesting questions!

I will ansewr in-line.

Do you know (from experience, previous packets, or
something) the supposed length of the expected packet when
this happens?

The received packet length is variable, these are PPP packets, but during
the set-up phase of the connection are not more than a few dozen bytes in
length, typically 30 to 50.

Do you loose packets all the time, or just
under some circumstances?

I do not lose any information during the modem-initialisation part of the
connection, this is the part that consists of ASCII “AT” commands sent to
and received from the modem. When the connection is established with the ISP
and the PPP set-up begins, that is when data starts to get lost, and it
appears to be with all packets received from that point on. Obviously, there
aren’t that many since the PPP negotiation fails and the connection is
abandoned.

How do you know that you only
see the beginning of a packet, rather than seeing for
instance the MIDDLE or END of a packet?

Because the PPP packets are framed with the flag 0x7E which should appear at
the start and end of each packet. I am seeing data which is recognizably the
start of a packet since it has the start flag which is followed by the
encapsulated protocol etc. I do not see the end of the packet.

Do you know for
sure that the reason that you’re loosing packets isn’t the
amount of time spent in your filter driver (i.e. do you do
something in your filter driver that may cause the maximum
baudrate allowed to drop dramatically, for instance long
delays, debug printouts, or some such)?

Ah - now this I don’t know! I do have a quantity of debug statements in the
driver to help me locate the problem and these are all passing across the
serial cable to the debugger but my observation has been that even if I put
in a huge amount of trace statements and the target machine runs slowly,
things do still work as expected. The quantity of received data is quite
small and is handled in a serial fashion in that the TCP/IP packets are not
pouring in but each received packet is in response to a transmitted packet.
If it takes a bit longer for a packet to arrive (as may well be the case
with a bad TCP/IP connection) there still should not be a problem with loss
of data. I can certainly remove all trace statements to prove the point.

Have you tried
running a slower baud-rate? Also what baud-rate ARE you running?

The connection is established at 57600 baud, I haven’t tried changing this,
I am not sure that the modem (actually a GPRS mobile handset) would like it.

If you’re doing printouts from your filter driver, consider
buffering them into a “block” of some sort, and then use the
debugger to fetch it later. I’ve certainly seen things go
wrong in our display driver when writing excessive amounts of
data from functions. Note: Display drivers are generally not
time-critical, whilst serial ports definitely ARE
time-critical, so you’ll have to be careful with the amount
of time you use in the filter function.

I don’t know how to buffer stuff and fetch it later.

One more point - it is quite normal for a TCP/IP packet to arrive in several
fragments so that fact that I am only seeing the start of the packet is not
a problem in itself. The problem is that I am not seeing any fragments other
than start fragments.

Thanks

Alasdair

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

OSR_Community_User · October 11, 2004, 10:41am

xxxxx@lists.osr.com wrote on 10/11/2004 03:10:32 PM:

Hi Mats,
Thanks for the reply, you have asked some interesting questions!
I will ansewr in-line.
> 1. Do you know (from experience, previous packets, or
> something) the supposed length of the expected packet when
> this happens?
The received packet length is variable, these are PPP packets, but
during the set-up phase of the connection are not more than a few
dozen bytes in length, typically 30 to 50.

Ok, that makes sense.

> 2. Do you loose packets all the time, or just
> under some circumstances?
I do not lose any information during the modem-initialisation part
of the connection, this is the part that consists of ASCII “AT”
commands sent to and received from the modem. When the connection is
established with the ISP and the PPP set-up begins, that is when
data starts to get lost, and it appears to be with all packets
received from that point on. Obviously, there aren’t that many since
the PPP negotiation fails and the connection is abandoned.

OK. So when it fails, how many bytes are in the packets.

> 3. How do you know that you only
> see the beginning of a packet, rather than seeing for
> instance the MIDDLE or END of a packet?
Because the PPP packets are framed with the flag 0x7E which should
appear at the start and end of each packet. I am seeing data which
is recognizably the start of a packet since it has the start flag
which is followed by the encapsulated protocol etc. I do not see the
end of the packet.

Ok, gotcha. So you’re seeing ONLY the start of a packet, and no middle or
end, ever? Or just sometimes not seeing all of the packets?

> 4. Do you know for
> sure that the reason that you’re loosing packets isn’t the
> amount of time spent in your filter driver (i.e. do you do
> something in your filter driver that may cause the maximum
> baudrate allowed to drop dramatically, for instance long
> delays, debug printouts, or some such)?
Ah - now this I don’t know! I do have a quantity of debug statements
in the driver to help me locate the problem and these are all
passing across the serial cable to the debugger but my observation
has been that even if I put in a huge amount of trace statements and
the target machine runs slowly, things do still work as expected.
The quantity of received data is quite small and is handled in a
serial fashion in that the TCP/IP packets are not pouring in but
each received packet is in response to a transmitted packet. If it
takes a bit longer for a packet to arrive (as may well be the case
with a bad TCP/IP connection) there still should not be a problem
with loss of data. I can certainly remove all trace statements to
prove the point.
> 5. Have you tried
> running a slower baud-rate? Also what baud-rate ARE you running?
The connection is established at 57600 baud, I haven’t tried
changing this, I am not sure that the modem (actually a GPRS mobile
handset) would like it.
> If you’re doing printouts from your filter driver, consider
> buffering them into a “block” of some sort, and then use the
> debugger to fetch it later. I’ve certainly seen things go
> wrong in our display driver when writing excessive amounts of
> data from functions. Note: Display drivers are generally not
> time-critical, whilst serial ports definitely ARE
> time-critical, so you’ll have to be careful with the amount
> of time you use in the filter function.
I don’t know how to buffer stuff and fetch it later.
One more point - it is quite normal for a TCP/IP packet to arrive in
several fragments so that fact that I am only seeing the start of
the packet is not a problem in itself. The problem is that I am not
seeing any fragments other than start fragments.

The simple way to do this is something like this:

static char bigBuffer[20000000];
static char smallBuffer[1000]; // Assuming a single string isn’t
more than 1000 char’s.
static char *bbPtr = bigBuffer;
static ULONG bbCount = 0;
static LONG CurrentDebugLevel = 0;

void DebugPrintf( LONG DebugPrintLevel, PCHAR DebugMessage, … )
{
va_list ap;
ULONG cnt;

if (CurrentDebugLevel < DebugPrintLevel)
return;

va_start(ap, DebugMessage);

cnt = _vsnprintf(smallBuffer, sizeof(logAssertStr), DebugMessage,
ap);
ASSERT(cnt < sizeof(smallBuffer));
ASSERT(cnt + bbCount < sizeof(bigBuffer));
strcat(bbPtr, cnt);
bbPtr += cnt;
bbCount += cnt;

va_end(ap);
}

Then just stop in the debugger, and do something like:
db bigBuffer

A more advanced method would be to use the kernel debug print to later on
print the strings, when a special IOCTL (IOCLT_MY_FILTER_PRINTDEBUGINFO) is
called, with suitable splitting of debug strings, etc, etc. But for this
purpose, it’s a bit overkill to do that advanced a design…

–
Mats

Thanks
Alasdair

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
ForwardSourceID:NT000051D6

OSR_Community_User · October 11, 2004, 11:18am

Hi Mats,

OK. So when it fails, how many bytes are in the packets.

24, and always 24!

I have just stripped out about half of the trace statements and run another
test, this is how the write and read packet sizes look after the modem
“CONNECT” string arrives.
I have shown “Read” when the read request goes down the stack with the
requested buffer size, “Write” when the write does down the stack and “Comp”
when the read completion routine gets called with the actual byte count
read.
The first packet should be a read packet:

Read 6
Comp 0 (status CANCELLED)
Read 3114
Comp 0
Write 46
Read 3114
Comp 24
Write 46
Read 3090
Comp 24
Read 3090
Comp 24
Write 46
Read 3090
Comp 24
Write 47
Read 3090
Comp 24

And on until the connection is abandoned.

It would appear that the original read request for 3114 bytes is partially
satisfied by the first “Comp” of 24 and thereafter further read requests are
for 3114-24 = 3090 bytes. I am guessing that the PPP software has recognised
that the incoming packet is incomplete and is asking for the rest of it. I
don’t know why we don’t see it arrive.

Ok, gotcha. So you’re seeing ONLY the start of a packet, and
no middle or end, ever? Or just sometimes not seeing all of
the packets?

Yes.

The simple way to do this is something like this:

static char bigBuffer[20000000];
static char smallBuffer[1000]; // Assuming a
single string isn’t
more than 1000 char’s.
static char *bbPtr = bigBuffer;
static ULONG bbCount = 0;
static LONG CurrentDebugLevel = 0;

void DebugPrintf( LONG DebugPrintLevel, PCHAR DebugMessage, … ) {
va_list ap;
ULONG cnt;

if (CurrentDebugLevel < DebugPrintLevel)
return;

va_start(ap, DebugMessage);

cnt = _vsnprintf(smallBuffer, sizeof(logAssertStr),
DebugMessage, ap);
ASSERT(cnt < sizeof(smallBuffer));
ASSERT(cnt + bbCount < sizeof(bigBuffer));
strcat(bbPtr, cnt);
bbPtr += cnt;
bbCount += cnt;

va_end(ap);
}

Then just stop in the debugger, and do something like:
db bigBuffer

A more advanced method would be to use the kernel debug print
to later on print the strings, when a special IOCTL
(IOCLT_MY_FILTER_PRINTDEBUGINFO) is called, with suitable
splitting of debug strings, etc, etc. But for this purpose,
it’s a bit overkill to do that advanced a design…

Thanks for this Mats - I shall look into this.

I am new to driver-writing so I don’t understand how all of these things
work but should ALL the read packets be coming back up the stack and into my
completion routine? I only set up the completion routine when I see the read
request going down the stack, is there any “other” way read data can arrive.
I don’t think there is but I thought I would ask the question. Also, could
overlapped or non-overlapped IO have any bearing on this?

Thanks

Alasdair

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

OSR_Community_User · October 11, 2004, 11:39am

xxxxx@lists.osr.com wrote on 10/11/2004 04:16:23 PM:

Hi Mats,
> OK. So when it fails, how many bytes are in the packets.
24, and always 24!

Curious. I wonder if the buffer in your COM-port is exactly 24 bytes,
perchance…

The other question is, if you stop in your Completion routine, (say at the
point where you seee the Read 3114 at the beginning of PPP packets) [You
could build a breakpoint trap for this, for instance using { __asm int 3; }
to stop the debugger at the read phase, and then set a break on the
completion routine.

So, when you get the break in the completion routine, do a “ib port” where
port is your com-port that you’ve connected to…

Get your PC hardware book out and check the different status values, such
as Overflow. Also, if the data is actually ending with 7E (remember, there
is a fifo in there, so it will hold a number of bytes before you get to the
LAST received one)…

I have just stripped out about half of the trace statements and run
another test, this is how the write and read packet sizes look after
the modem “CONNECT” string arrives.
I have shown “Read” when the read request goes down the stack with
the requested buffer size, “Write” when the write does down the
stack and “Comp” when the read completion routine gets called with
the actual byte count read.
The first packet should be a read packet:
Read 6
Comp 0 (status CANCELLED)
Read 3114
Comp 0
Write 46
Read 3114
Comp 24
Write 46
Read 3090
Comp 24
Read 3090
Comp 24
Write 46
Read 3090
Comp 24
Write 47
Read 3090
Comp 24
And on until the connection is abandoned.
It would appear that the original read request for 3114 bytes is
partially satisfied by the first “Comp” of 24 and thereafter further
read requests are for 3114-24 = 3090 bytes. I am guessing that the
PPP software has recognised that the incoming packet is incomplete
and is asking for the rest of it. I don’t know why we don’t see it
arrive.
> Ok, gotcha. So you’re seeing ONLY the start of a packet, and
> no middle or end, ever? Or just sometimes not seeing all of
> the packets?
Yes.

> The simple way to do this is something like this:
>
> static char bigBuffer[20000000];
> static char smallBuffer[1000]; // Assuming a
> single string isn’t
> more than 1000 char’s.
> static char *bbPtr = bigBuffer;
> static ULONG bbCount = 0;
> static LONG CurrentDebugLevel = 0;
>
> void DebugPrintf( LONG DebugPrintLevel, PCHAR DebugMessage, … ) {
> va_list ap;
> ULONG cnt;
>
> if (CurrentDebugLevel < DebugPrintLevel)
> return;
>
> va_start(ap, DebugMessage);
>
> cnt = _vsnprintf(smallBuffer, sizeof(logAssertStr),
> DebugMessage, ap);
> ASSERT(cnt < sizeof(smallBuffer));
> ASSERT(cnt + bbCount < sizeof(bigBuffer));
> strcat(bbPtr, cnt);
> bbPtr += cnt;
> bbCount += cnt;
>
> va_end(ap);
> }
>
> Then just stop in the debugger, and do something like:
> db bigBuffer
>
> A more advanced method would be to use the kernel debug print
> to later on print the strings, when a special IOCTL
> (IOCLT_MY_FILTER_PRINTDEBUGINFO) is called, with suitable
> splitting of debug strings, etc, etc. But for this purpose,
> it’s a bit overkill to do that advanced a design…
Thanks for this Mats - I shall look into this.
I am new to driver-writing so I don’t understand how all of these
things work but should ALL the read packets be coming back up the
stack and into my completion routine? I only set up the completion
routine when I see the read request going down the stack, is there
any “other” way read data can arrive. I don’t think there is but I
thought I would ask the question. Also, could overlapped or non-
overlapped IO have any bearing on this?

I expect that your completion routine should be called from all packets
that are completed. But I’m not at all an expert here, so do listen to
other people…

–
Mats

Thanks
Alasdair

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
ForwardSourceID:NT000051F6

OSR_Community_User · October 11, 2004, 11:40am

Alasdair Tompson (external) wrote:

Hi Mats,

Thanks for the reply, you have asked some interesting questions!

I will ansewr in-line.

> 1. Do you know (from experience, previous packets, or
> something) the supposed length of the expected packet when
> this happens?

The received packet length is variable, these are PPP packets, but
during the set-up phase of the connection are not more than a few dozen
bytes in length, typically 30 to 50.

> 2. Do you loose packets all the time, or just
> under some circumstances?

I do not lose any information during the modem-initialisation part of
the connection, this is the part that consists of ASCII “AT” commands
sent to and received from the modem. When the connection is established
with the ISP and the PPP set-up begins, that is when data starts to get
lost, and it appears to be with all packets received from that point on.
Obviously, there aren’t that many since the PPP negotiation fails and
the connection is abandoned.

> 3. How do you know that you only
> see the beginning of a packet, rather than seeing for
> instance the MIDDLE or END of a packet?

Because the PPP packets are framed with the flag 0x7E which should
appear at the start and end of each packet. I am seeing data which is
recognizably the start of a packet since it has the start flag which is
followed by the encapsulated protocol etc. I do not see the end of the
packet.

> 4. Do you know for
> sure that the reason that you’re loosing packets isn’t the
> amount of time spent in your filter driver (i.e. do you do
> something in your filter driver that may cause the maximum
> baudrate allowed to drop dramatically, for instance long
> delays, debug printouts, or some such)?

Ah - now this I don’t know! I do have a quantity of debug statements in
the driver to help me locate the problem and these are all passing
across the serial cable to the debugger but my observation has been that
even if I put in a huge amount of trace statements and the target
machine runs slowly, things do still work as expected. The quantity of
received data is quite small and is handled in a serial fashion in that
the TCP/IP packets are not pouring in but each received packet is in
response to a transmitted packet. If it takes a bit longer for a packet
to arrive (as may well be the case with a bad TCP/IP connection) there
still should not be a problem with loss of data. I can certainly remove
all trace statements to prove the point.

As a disclaimer, I’ve not done an kernel mode serial programming,
but I’ve done a bit of usermode programming…

How big is the UART (or equivilent device) hardware buffer?
I think these buffers are typically pretty small… the web
document I just pulled up says the original 16450A can only
buffer 16 bytes in each direction… and that newer models
are under development which will be able to buffer 32 bytes
each direction… I also see mention of a 64 byte buffer…
that should give you an idea of order of magnitude.

If you are sending loads of data to the debugger across
a serial wire, you may be preventing the serial driver
from servicing interupts from the device in a timely
fashion, causing this 16-32 byte buffer to overflow.

On the other hand, if you are blocking or slowing down
the I/O too much be the processing you are doing on
it before completing it to the application (in the
read case) you may be slowing the application down to
the point where it isn’t clearing the serial driver’s
buffer’s fast enough.

if you are sending loads of data to the debugger across
a serial wire (or if you are blocking I/O completion for
too long), you may be preventing the serial driver from
servicing the device fast enough, or preventing the
application from clearing the serial driver buffers fast
enough.

If this is the case, the serial driver probably knows it
and has a bit set someplace because the UART flagged an
overflow. You can probably send a IOCTL of some sort
to ask the serial driver about it… from user mode,
the function you want is ClearCommError(), and the
bit you are interested in is probably either the
CE_OVERRUN bit or the CE_RXOVER bit.

Joseph

PS. You can probably write a simple usermode program
to run on the other side and generate this problem…
just open the comm port and write a pretty good burst
of data.

You can probably also do it using hyperterms file
transfer facility (ZModem/Xmodem I think it supports
at least one of these.)

Typing in hyperterm isn’t a good test since even with
all the debugging, the computer is still probably faster
than you are.

OSR_Community_User · October 11, 2004, 11:56am

> PS. You can probably write a simple usermode program

to run on the other side and generate this problem…
just open the comm port and write a pretty good burst
of data.

This is a good idea. Instead of using the phone (which of course is the
ultimate target), use a PC with either standard of custom software on it.

You can probably also do it using hyperterms file
transfer facility (ZModem/Xmodem I think it supports
at least one of these.)

Typing in hyperterm isn’t a good test since even with
all the debugging, the computer is still probably faster
than you are.

I don’t know if Hyperterm supports it, but definitely some other terminal
programs (TeraTerm comes to mind, but I’m unsure if this feature is
included) support logging and transmitting of text-files (ASCII transfer
mode). So buiild a text-file with a heap of text that you can identify if
it’s been transmitted correctly at the other end, with varying lenghts and
“no protocol” (because using a protocol just confuses the issue) and send
some data across.

If you write a little test-app, then one test method should be to transmit
a file and compare at the other end (so you start the compare first, then
start the transmit, and the test-app shows which lines come across badly).
That way, you can test different patterns, sizes, etc, etc, and change the
test-setup without having to do too much programming.

Writing custom test-tools is a big part of device drivers, because you need
to be able to do “directed testing”, e.g. testing special cases without
hitting some other stuff that is currently not working, etc, etc.

–
Mats

OSR_Community_User · October 11, 2004, 11:59am

Thanks Mats and Joseph,

Good idea, I am going home now so I’ll look into that tomorrow.

Alasdair

-----Original Message-----
From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
Sent: 11 October 2004 16:55
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Losing Received Data?

> PS. You can probably write a simple usermode program
> to run on the other side and generate this problem…
> just open the comm port and write a pretty good burst
> of data.

This is a good idea. Instead of using the phone (which of
course is the ultimate target), use a PC with either standard
of custom software on it.
>
> You can probably also do it using hyperterms file
> transfer facility (ZModem/Xmodem I think it supports
> at least one of these.)
>
> Typing in hyperterm isn’t a good test since even with
> all the debugging, the computer is still probably faster
> than you are.

I don’t know if Hyperterm supports it, but definitely some
other terminal programs (TeraTerm comes to mind, but I’m
unsure if this feature is
included) support logging and transmitting of text-files
(ASCII transfer mode). So buiild a text-file with a heap of
text that you can identify if it’s been transmitted correctly
at the other end, with varying lenghts and “no protocol”
(because using a protocol just confuses the issue) and send
some data across.

If you write a little test-app, then one test method should
be to transmit a file and compare at the other end (so you
start the compare first, then start the transmit, and the
test-app shows which lines come across badly). That way, you
can test different patterns, sizes, etc, etc, and change the
test-setup without having to do too much programming.

Writing custom test-tools is a big part of device drivers,
because you need to be able to do “directed testing”, e.g.
testing special cases without hitting some other stuff that
is currently not working, etc, etc.

–
Mats

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as:
xxxxx@t-mobile.co.uk To unsubscribe send a blank
email to xxxxx@lists.osr.com

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

OSR_Community_User · October 12, 2004, 3:57am

Hi again,

I noticed that when the driver was running under the debugger the system
load was very high, the red part of the graph indicating kernel load was
hovering around 60% when the driver was loaded and with no serial comms
taking place, I have no idea why. So I tried the same (debug) version of the
driver in the non-debug version of W2K to remove any issues caused by the
debugger and it’s serial comms, system load was the same but the dial-up
connection succeeded! so I think it is a performance issue rather than a
logic issue.

I don’t know why the load should be so high, there are no threads in the
driver and as far as I am aware it should just be sitting there doing
nothing, you probably know better.

My next test will be to see what the load is with the free version of the
driver.

Thanks for the suggestions regarding using a test app to thrash the comms, I
may still need to resort to that but for time being I think I will need to
track down this load problem.

As usual, any suggestions greatly appreciated.

Thanks

Alasdair

-----Original Message-----
From: Alasdair Tompson (external) [mailto:xxxxx@t-mobile.co.uk]
Sent: 11 October 2004 16:58
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Losing Received Data?

Thanks Mats and Joseph,

Good idea, I am going home now so I’ll look into that tomorrow.

Alasdair

-----Original Message-----
From: Mats PETERSSON [mailto:xxxxx@3dlabs.com
mailto:xxxxx ]
> Sent: 11 October 2004 16:55
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Losing Received Data?
>
>
>
>
>
>
>
> > PS. You can probably write a simple usermode program
> > to run on the other side and generate this problem…
> > just open the comm port and write a pretty good burst
> > of data.
>
> This is a good idea. Instead of using the phone (which of
> course is the ultimate target), use a PC with either standard
> of custom software on it.
> >
> > You can probably also do it using hyperterms file
> > transfer facility (ZModem/Xmodem I think it supports
> > at least one of these.)
> >
> > Typing in hyperterm isn’t a good test since even with
> > all the debugging, the computer is still probably faster
> > than you are.
>
> I don’t know if Hyperterm supports it, but definitely some
> other terminal programs (TeraTerm comes to mind, but I’m
> unsure if this feature is
> included) support logging and transmitting of text-files
> (ASCII transfer mode). So buiild a text-file with a heap of
> text that you can identify if it’s been transmitted correctly
> at the other end, with varying lenghts and “no protocol”
> (because using a protocol just confuses the issue) and send
> some data across.
>
> If you write a little test-app, then one test method should
> be to transmit a file and compare at the other end (so you
> start the compare first, then start the transmit, and the
> test-app shows which lines come across badly). That way, you
> can test different patterns, sizes, etc, etc, and change the
> test-setup without having to do too much programming.
>
> Writing custom test-tools is a big part of device drivers,
> because you need to be able to do “directed testing”, e.g.
> testing special cases without hitting some other stuff that
> is currently not working, etc, etc.
>
>
> –
> Mats
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
http:
>
> You are currently subscribed to ntdev as:
> xxxxx@t-mobile.co.uk To unsubscribe send a blank
> email to xxxxx@lists.osr.com
>

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@t-mobile.co.uk
To unsubscribe send a blank email to xxxxx@lists.osr.com

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.</http:></mailto:xxxxx>

OSR_Community_User · October 12, 2004, 4:38am

You could always use VTune (or some other performance profiling tool) to
figure out what takes time in your driver (or someone elses driver).

Debug output that doesn’t actually print anything (if you use levels for
instance) will still take a fair amount of time, simply because of argument
passing and calling the debut output function.

But to achieve something like 60% CPU performance on a reasonably modern
processor, you need to actually do quite a lot. Why not break into the
system a few times and see where it’s at. (Press CTRL-Break and check the
stack to see what’s going on). You should have a hit in whatever it is that
takes a lot of time every other time. This is called “poor mans profiling”,
and it actually works much better than you’d think.

–
Mats

xxxxx@lists.osr.com wrote on 10/12/2004 08:55:25 AM:

Hi again,

I noticed that when the driver was running under the debugger the
system load was very high, the red part of the graph indicating
kernel load was hovering around 60% when the driver was loaded and
with no serial comms taking place, I have no idea why. So I tried
the same (debug) version of the driver in the non-debug version of
W2K to remove any issues caused by the debugger and it’s serial
comms, system load was the same but the dial-up connection
succeeded! so I think it is a performance issue rather than a logic
issue.

I don’t know why the load should be so high, there are no threads in
the driver and as far as I am aware it should just be sitting there
doing nothing, you probably know better.

My next test will be to see what the load is with the free version
of the driver.

Thanks for the suggestions regarding using a test app to thrash the
comms, I may still need to resort to that but for time being I think
I will need to track down this load problem.

As usual, any suggestions greatly appreciated.

Thanks

Alasdair

-----Original Message-----
From: Alasdair Tompson (external)
[mailto:xxxxx@t-mobile.co.uk]
Sent: 11 October 2004 16:58
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Losing Received Data?

Thanks Mats and Joseph,
Good idea, I am going home now so I’ll look into that tomorrow.
Alasdair

> -----Original Message-----
> From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
> Sent: 11 October 2004 16:55
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Losing Received Data?
>
>
>
>
>
>
>
> > PS. You can probably write a simple usermode program
> > to run on the other side and generate this problem…
> > just open the comm port and write a pretty good burst
> > of data.
>
> This is a good idea. Instead of using the phone (which of
> course is the ultimate target), use a PC with either standard
> of custom software on it.
> >
> > You can probably also do it using hyperterms file
> > transfer facility (ZModem/Xmodem I think it supports
> > at least one of these.)
> >
> > Typing in hyperterm isn’t a good test since even with
> > all the debugging, the computer is still probably faster
> > than you are.
>
> I don’t know if Hyperterm supports it, but definitely some
> other terminal programs (TeraTerm comes to mind, but I’m
> unsure if this feature is
> included) support logging and transmitting of text-files
> (ASCII transfer mode). So buiild a text-file with a heap of
> text that you can identify if it’s been transmitted correctly
> at the other end, with varying lenghts and “no protocol”
> (because using a protocol just confuses the issue) and send
> some data across.
>
> If you write a little test-app, then one test method should
> be to transmit a file and compare at the other end (so you
> start the compare first, then start the transmit, and the
> test-app shows which lines come across badly). That way, you
> can test different patterns, sizes, etc, etc, and change the
> test-setup without having to do too much programming.
>
> Writing custom test-tools is a big part of device drivers,
> because you need to be able to do “directed testing”, e.g.
> testing special cases without hitting some other stuff that
> is currently not working, etc, etc.
>
>
> –
> Mats
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@t-mobile.co.uk To unsubscribe send a blank
> email to xxxxx@lists.osr.com
>

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@t-mobile.co.uk

To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

ForwardSourceID:NT00005356

OSR_Community_User · October 12, 2004, 11:03am

I would probably do the same here; try breaking in a
couple times.

I might also try the !process 0 0x17 to dump all the
processes and tread stacks on the system. This may take
a while.

Then look for one of the threads hanging out in your filter.

Is it a single CPU or an SMP machine? If you are 60% on a
dual processor SMP machine, my experience is that you’ve
might have managed to go into a tight loop on one processor.
If this is the case, it should be pretty easy to spot.

There are probably some other WinDBG commands I’m
not familiar with that would help you find a completion
routine running in an DPC context (which probably
won’t show up in !process??? I’m not sure on that one.)

Also, it seems like I recall windows being pretty
chatty with the serial ports, especially during
startup (looking for modems, mice, etc.) So even
though you think the serial port is idle, it may
not really be (completely) idle.

Joseph

Mats PETERSSON wrote:

You could always use VTune (or some other performance profiling tool) to
figure out what takes time in your driver (or someone elses driver).

Debug output that doesn’t actually print anything (if you use levels for
instance) will still take a fair amount of time, simply because of argument
passing and calling the debut output function.

But to achieve something like 60% CPU performance on a reasonably modern
processor, you need to actually do quite a lot. Why not break into the
system a few times and see where it’s at. (Press CTRL-Break and check the
stack to see what’s going on). You should have a hit in whatever it is that
takes a lot of time every other time. This is called “poor mans profiling”,
and it actually works much better than you’d think.

–
Mats

xxxxx@lists.osr.com wrote on 10/12/2004 08:55:25 AM:

>Hi again,
>
>I noticed that when the driver was running under the debugger the
>system load was very high, the red part of the graph indicating
>kernel load was hovering around 60% when the driver was loaded and
>with no serial comms taking place, I have no idea why. So I tried
>the same (debug) version of the driver in the non-debug version of
>W2K to remove any issues caused by the debugger and it’s serial
>comms, system load was the same but the dial-up connection
>succeeded! so I think it is a performance issue rather than a logic

issue.

>I don’t know why the load should be so high, there are no threads in
>the driver and as far as I am aware it should just be sitting there
>doing nothing, you probably know better.
>
>My next test will be to see what the load is with the free version
>of the driver.
>
>Thanks for the suggestions regarding using a test app to thrash the
>comms, I may still need to resort to that but for time being I think
>I will need to track down this load problem.
>
>As usual, any suggestions greatly appreciated.
>
>Thanks
>
>Alasdair
>
>
>-----Original Message-----
>From: Alasdair Tompson (external)

[mailto:xxxxx@t-mobile.co.uk]

>Sent: 11 October 2004 16:58
>To: Windows System Software Devs Interest List
>Subject: RE: [ntdev] Losing Received Data?

>Thanks Mats and Joseph,
>Good idea, I am going home now so I’ll look into that tomorrow.
>Alasdair
>
>
>>-----Original Message-----
>>From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
>>Sent: 11 October 2004 16:55
>>To: Windows System Software Devs Interest List
>>Subject: Re: [ntdev] Losing Received Data?
>>
>>
>>
>>
>>
>>
>>
>>
>>>PS. You can probably write a simple usermode program
>>>to run on the other side and generate this problem…
>>>just open the comm port and write a pretty good burst
>>>of data.
>>
>>This is a good idea. Instead of using the phone (which of
>>course is the ultimate target), use a PC with either standard
>>of custom software on it.
>>
>>>You can probably also do it using hyperterms file
>>>transfer facility (ZModem/Xmodem I think it supports
>>>at least one of these.)
>>>
>>>Typing in hyperterm isn’t a good test since even with
>>>all the debugging, the computer is still probably faster
>>>than you are.
>>
>>I don’t know if Hyperterm supports it, but definitely some
>>other terminal programs (TeraTerm comes to mind, but I’m
>>unsure if this feature is
>>included) support logging and transmitting of text-files
>>(ASCII transfer mode). So buiild a text-file with a heap of
>>text that you can identify if it’s been transmitted correctly
>>at the other end, with varying lenghts and “no protocol”
>>(because using a protocol just confuses the issue) and send
>>some data across.
>>
>>If you write a little test-app, then one test method should
>>be to transmit a file and compare at the other end (so you
>>start the compare first, then start the transmit, and the
>>test-app shows which lines come across badly). That way, you
>>can test different patterns, sizes, etc, etc, and change the
>>test-setup without having to do too much programming.
>>
>>Writing custom test-tools is a big part of device drivers,
>>because you need to be able to do “directed testing”, e.g.
>>testing special cases without hitting some other stuff that
>>is currently not working, etc, etc.
>>
>>
>>–
>>Mats
>>
>>
>>
>>—
>>Questions? First check the Kernel Driver FAQ at
>>http://www.osronline.com/article.cfm?id=256
>>
>>You are currently subscribed to ntdev as:
>>xxxxx@t-mobile.co.uk To unsubscribe send a blank
>>email to xxxxx@lists.osr.com
>>
>

>NOTICE AND DISCLAIMER:
>This email (including attachments) is confidential. If you have
>received this email in error please notify the sender immediately
>and delete this email from your system without copying or
>disseminating it or placing any reliance upon its contents. We
>cannot accept liability for any breaches of confidence arising
>through use of email. Any opinions expressed in this email
>(including attachments) are those of the author and do not
>necessarily reflect our opinions. We will not accept responsibility
>for any commitments made by our employees outside the scope of our
>business. We do not warrant the accuracy or completeness of such

information.

>—
>Questions? First check the Kernel Driver FAQ at http://www.
>osronline.com/article.cfm?id=256
>
>You are currently subscribed to ntdev as: xxxxx@t-mobile.co.uk

>To unsubscribe send a blank email to xxxxx@lists.osr.com
>—
>Questions? First check the Kernel Driver FAQ at http://www.
>osronline.com/article.cfm?id=256
>
>You are currently subscribed to ntdev as: xxxxx@3dlabs.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>

>NOTICE AND DISCLAIMER:
>This email (including attachments) is confidential. If you have
>received this email in error please notify the sender immediately
>and delete this email from your system without copying or
>disseminating it or placing any reliance upon its contents. We
>cannot accept liability for any breaches of confidence arising
>through use of email. Any opinions expressed in this email
>(including attachments) are those of the author and do not
>necessarily reflect our opinions. We will not accept responsibility
>for any commitments made by our employees outside the scope of our
>business. We do not warrant the accuracy or completeness of such

information.

>ForwardSourceID:NT00005356

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@vandyke.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · October 12, 2004, 11:10am

Hi Mats and Joseph,

Thanks for your suggestions.

I’m still looking at this. The serial port in question is actually on a
PCMCIA 4-port card and it looks as though the driver for THAT is causing all
the load. However, when changing to a different make of card whose driver
does NOT result in so much load I still have the same problem! I am now
trying to get a free version of the driver to run (I can’t load it at the
moment) so I’ll get back to you when I’ve sorted that out.

Thanks again

Alasdair

-----Original Message-----
From: Joseph Galbraith [mailto:xxxxx@vandyke.com]
Sent: 12 October 2004 16:02
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Losing Received Data?

I would probably do the same here; try breaking in a
couple times.

I might also try the !process 0 0x17 to dump all the
processes and tread stacks on the system. This may take
a while.

Then look for one of the threads hanging out in your filter.

Is it a single CPU or an SMP machine? If you are 60% on a
dual processor SMP machine, my experience is that you’ve
might have managed to go into a tight loop on one processor.
If this is the case, it should be pretty easy to spot.

There are probably some other WinDBG commands I’m
not familiar with that would help you find a completion
routine running in an DPC context (which probably
won’t show up in !process??? I’m not sure on that one.)

Also, it seems like I recall windows being pretty
chatty with the serial ports, especially during
startup (looking for modems, mice, etc.) So even
though you think the serial port is idle, it may
not really be (completely) idle.

Joseph

Mats PETERSSON wrote:
> You could always use VTune (or some other performance
profiling tool)
> to figure out what takes time in your driver (or someone elses
> driver).
>
> Debug output that doesn’t actually print anything (if you
use levels
> for
> instance) will still take a fair amount of time, simply
because of argument
> passing and calling the debut output function.
>
> But to achieve something like 60% CPU performance on a reasonably
> modern processor, you need to actually do quite a lot. Why
not break
> into the system a few times and see where it’s at. (Press
CTRL-Break
> and check the stack to see what’s going on). You should
have a hit in
> whatever it is that takes a lot of time every other time. This is
> called “poor mans profiling”, and it actually works much
better than
> you’d think.
>
> –
> Mats
>
> xxxxx@lists.osr.com wrote on 10/12/2004 08:55:25 AM:
>
>
>>Hi again,
>>
>>I noticed that when the driver was running under the debugger the
>>system load was very high, the red part of the graph
indicating kernel
>>load was hovering around 60% when the driver was loaded and with no
>>serial comms taking place, I have no idea why. So I tried the same
>>(debug) version of the driver in the non-debug version of W2K to
>>remove any issues caused by the debugger and it’s serial
comms, system
>>load was the same but the dial-up connection succeeded! so
I think it
>>is a performance issue rather than a logic
>
> issue.
>
>>I don’t know why the load should be so high, there are no
threads in
>>the driver and as far as I am aware it should just be sitting there
>>doing nothing, you probably know better.
>>
>>My next test will be to see what the load is with the free
version of
>>the driver.
>>
>>Thanks for the suggestions regarding using a test app to thrash the
>>comms, I may still need to resort to that but for time
being I think I
>>will need to track down this load problem.
>>
>>As usual, any suggestions greatly appreciated.
>>
>>Thanks
>>
>>Alasdair
>>
>>
>>-----Original Message-----
>>From: Alasdair Tompson (external)
>
> [mailto:xxxxx@t-mobile.co.uk]
>
>>Sent: 11 October 2004 16:58
>>To: Windows System Software Devs Interest List
>>Subject: RE: [ntdev] Losing Received Data?
>
>
>>Thanks Mats and Joseph,
>>Good idea, I am going home now so I’ll look into that tomorrow.
>>Alasdair
>>
>>
>>>-----Original Message-----
>>>From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
>>>Sent: 11 October 2004 16:55
>>>To: Windows System Software Devs Interest List
>>>Subject: Re: [ntdev] Losing Received Data?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>PS. You can probably write a simple usermode program
>>>>to run on the other side and generate this problem…
>>>>just open the comm port and write a pretty good burst
>>>>of data.
>>>
>>>This is a good idea. Instead of using the phone (which of
course is
>>>the ultimate target), use a PC with either standard of custom
>>>software on it.
>>>
>>>>You can probably also do it using hyperterms file
>>>>transfer facility (ZModem/Xmodem I think it supports
>>>>at least one of these.)
>>>>
>>>>Typing in hyperterm isn’t a good test since even with
>>>>all the debugging, the computer is still probably faster than you
>>>>are.
>>>
>>>I don’t know if Hyperterm supports it, but definitely some other
>>>terminal programs (TeraTerm comes to mind, but I’m unsure if this
>>>feature is
>>>included) support logging and transmitting of text-files (ASCII
>>>transfer mode). So buiild a text-file with a heap of text that you
>>>can identify if it’s been transmitted correctly at the other end,
>>>with varying lenghts and “no protocol” (because using a
protocol just
>>>confuses the issue) and send some data across.
>>>
>>>If you write a little test-app, then one test method should be to
>>>transmit a file and compare at the other end (so you start the
>>>compare first, then start the transmit, and the test-app
shows which
>>>lines come across badly). That way, you can test different
patterns,
>>>sizes, etc, etc, and change the test-setup without having
to do too
>>>much programming.
>>>
>>>Writing custom test-tools is a big part of device drivers, because
>>>you need to be able to do “directed testing”, e.g. testing special
>>>cases without hitting some other stuff that is currently
not working,
>>>etc, etc.
>>>
>>>
>>>–
>>>Mats
>>>
>>>
>>>
>>>—
>>>Questions? First check the Kernel Driver FAQ at
>>>http://www.osronline.com/article.cfm?id=256
>>>
>>>You are currently subscribed to ntdev as:
>>>xxxxx@t-mobile.co.uk To unsubscribe send a
blank email to
>>>xxxxx@lists.osr.com
>>>
>>
>
>>NOTICE AND DISCLAIMER:
>>This email (including attachments) is confidential. If you have
>>received this email in error please notify the sender
immediately and
>>delete this email from your system without copying or
disseminating it
>>or placing any reliance upon its contents. We cannot
accept liability
>>for any breaches of confidence arising through use of email. Any
>>opinions expressed in this email (including attachments)
are those of
>>the author and do not necessarily reflect our opinions. We
will not
>>accept responsibility for any commitments made by our employees
>>outside the scope of our business. We do not warrant the
accuracy or
>>completeness of such
>
> information.
>
>>—
>>Questions? First check the Kernel Driver FAQ at http://www.
>>osronline.com/article.cfm?id=256
>>
>>You are currently subscribed to ntdev as:
>>xxxxx@t-mobile.co.uk
>
>
>>To unsubscribe send a blank email to
xxxxx@lists.osr.com
>>—
>>Questions? First check the Kernel Driver FAQ at http://www.
>>osronline.com/article.cfm?id=256
>>
>>You are currently subscribed to ntdev as:
xxxxx@3dlabs.com To
>>unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>
>
>>NOTICE AND DISCLAIMER:
>>This email (including attachments) is confidential. If you have
>>received this email in error please notify the sender
immediately and
>>delete this email from your system without copying or
disseminating it
>>or placing any reliance upon its contents. We cannot accept
liability
>>for any breaches of confidence arising through use of email. Any
>>opinions expressed in this email (including attachments)
are those of
>>the author and do not necessarily reflect our opinions. We will not
>>accept responsibility for any commitments made by our employees
>>outside the scope of our business. We do not warrant the
accuracy or
>>completeness of such
>
> information.
>
>>ForwardSourceID:NT00005356
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@vandyke.com
> To unsubscribe send a blank email to
xxxxx@lists.osr.com
>

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as:
xxxxx@t-mobile.co.uk To unsubscribe send a blank
email to xxxxx@lists.osr.com

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

Prokash_Sinha-1 · October 12, 2004, 11:39am

RE: [ntdev] Losing Received Data?Really sorry to step into this thread, as
Mats and Joseph pointed out lots of ways to tackle this …

As they pointed out, one easy way is to profile the drivers, using VTune,
numega code coverage (True Coverage ) etc …

There are two situation ( either your driver is on the path of execution for
some scenairo or not ). Determining that is important !. If, by any chance,
you can not
comeup with some scenarios where your driver is totally inactive in the
execution path(s) ( often happens for some filter drivers - as an example a
file system filter driver for file activities ) then very likely you will
find that your driver might have quite a bit of kernel mode waits for
various reasons …

Recently I was trying to find the performance effects of various virus
scanners ( often implemented as file system filters ). Found that lot of the
time CPU is carrying 50 to 60 % load, but some out liers (ie one in a while
spike for near 100% ). When trapped thru getting usr time and kernel time,
we see there are lot of kernel mode waits …

Just in case if it gives you any more pointers !!!

-pro
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Alasdair Tompson
(external)
Sent: Tuesday, October 12, 2004 8:08 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Losing Received Data?

Hi Mats and Joseph,

Thanks for your suggestions.

I’m still looking at this. The serial port in question is actually on a
PCMCIA 4-port card and it looks as though the driver for THAT is causing all
the load. However, when changing to a different make of card whose driver
does NOT result in so much load I still have the same problem! I am now
trying to get a free version of the driver to run (I can’t load it at the
moment) so I’ll get back to you when I’ve sorted that out.

Thanks again

Alasdair

-----Original Message-----
> From: Joseph Galbraith [mailto:xxxxx@vandyke.com]
> Sent: 12 October 2004 16:02
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Losing Received Data?
>
>
> I would probably do the same here; try breaking in a
> couple times.
>
> I might also try the !process 0 0x17 to dump all the
> processes and tread stacks on the system. This may take
> a while.
>
> Then look for one of the threads hanging out in your filter.
>
> Is it a single CPU or an SMP machine? If you are 60% on a
> dual processor SMP machine, my experience is that you’ve
> might have managed to go into a tight loop on one processor.
> If this is the case, it should be pretty easy to spot.
>
> There are probably some other WinDBG commands I’m
> not familiar with that would help you find a completion
> routine running in an DPC context (which probably
> won’t show up in !process??? I’m not sure on that one.)
>
> Also, it seems like I recall windows being pretty
> chatty with the serial ports, especially during
> startup (looking for modems, mice, etc.) So even
> though you think the serial port is idle, it may
> not really be (completely) idle.
>
> - Joseph
>
> Mats PETERSSON wrote:
> > You could always use VTune (or some other performance
> profiling tool)
> > to figure out what takes time in your driver (or someone elses
> > driver).
> >
> > Debug output that doesn’t actually print anything (if you
> use levels
> > for
> > instance) will still take a fair amount of time, simply
> because of argument
> > passing and calling the debut output function.
> >
> > But to achieve something like 60% CPU performance on a reasonably
> > modern processor, you need to actually do quite a lot. Why
> not break
> > into the system a few times and see where it’s at. (Press
> CTRL-Break
> > and check the stack to see what’s going on). You should
> have a hit in
> > whatever it is that takes a lot of time every other time. This is
> > called “poor mans profiling”, and it actually works much
> better than
> > you’d think.
> >
> > –
> > Mats
> >
> > xxxxx@lists.osr.com wrote on 10/12/2004 08:55:25 AM:
> >
> >
> >>Hi again,
> >>
> >>I noticed that when the driver was running under the debugger the
> >>system load was very high, the red part of the graph
> indicating kernel
> >>load was hovering around 60% when the driver was loaded and with no
> >>serial comms taking place, I have no idea why. So I tried the same
> >>(debug) version of the driver in the non-debug version of W2K to
> >>remove any issues caused by the debugger and it’s serial
> comms, system
> >>load was the same but the dial-up connection succeeded! so
> I think it
> >>is a performance issue rather than a logic
> >
> > issue.
> >
> >>I don’t know why the load should be so high, there are no
> threads in
> >>the driver and as far as I am aware it should just be sitting there
> >>doing nothing, you probably know better.
> >>
> >>My next test will be to see what the load is with the free
> version of
> >>the driver.
> >>
> >>Thanks for the suggestions regarding using a test app to thrash the
> >>comms, I may still need to resort to that but for time
> being I think I
> >>will need to track down this load problem.
> >>
> >>As usual, any suggestions greatly appreciated.
> >>
> >>Thanks
> >>
> >>Alasdair
> >>
> >>
> >>-----Original Message-----
> >>From: Alasdair Tompson (external)
> >
> > [mailto:xxxxx@t-mobile.co.uk]
> >
> >>Sent: 11 October 2004 16:58
> >>To: Windows System Software Devs Interest List
> >>Subject: RE: [ntdev] Losing Received Data?
> >
> >
> >>Thanks Mats and Joseph,
> >>Good idea, I am going home now so I’ll look into that tomorrow.
> >>Alasdair
> >>
> >>
> >>>-----Original Message-----
> >>>From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
> >>>Sent: 11 October 2004 16:55
> >>>To: Windows System Software Devs Interest List
> >>>Subject: Re: [ntdev] Losing Received Data?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>PS. You can probably write a simple usermode program
> >>>>to run on the other side and generate this problem…
> >>>>just open the comm port and write a pretty good burst
> >>>>of data.
> >>>
> >>>This is a good idea. Instead of using the phone (which of
> course is
> >>>the ultimate target), use a PC with either standard of custom
> >>>software on it.
> >>>
> >>>>You can probably also do it using hyperterms file
> >>>>transfer facility (ZModem/Xmodem I think it supports
> >>>>at least one of these.)
> >>>>
> >>>>Typing in hyperterm isn’t a good test since even with
> >>>>all the debugging, the computer is still probably faster than you
> >>>>are.
> >>>
> >>>I don’t know if Hyperterm supports it, but definitely some other
> >>>terminal programs (TeraTerm comes to mind, but I’m unsure if this
> >>>feature is
> >>>included) support logging and transmitting of text-files (ASCII
> >>>transfer mode). So buiild a text-file with a heap of text that you
> >>>can identify if it’s been transmitted correctly at the other end,
> >>>with varying lenghts and “no protocol” (because using a
> protocol just
> >>>confuses the issue) and send some data across.
> >>>
> >>>If you write a little test-app, then one test method should be to
> >>>transmit a file and compare at the other end (so you start the
> >>>compare first, then start the transmit, and the test-app
> shows which
> >>>lines come across badly). That way, you can test different
> patterns,
> >>>sizes, etc, etc, and change the test-setup without having
> to do too
> >>>much programming.
> >>>
> >>>Writing custom test-tools is a big part of device drivers, because
> >>>you need to be able to do “directed testing”, e.g. testing special
> >>>cases without hitting some other stuff that is currently
> not working,
> >>>etc, etc.
> >>>
> >>>
> >>>–
> >>>Mats
> >>>
> >>>
> >>>
> >>>—
> >>>Questions? First check the Kernel Driver FAQ at
> >>>http://www.osronline.com/article.cfm?id=256
> >>>
> >>>You are currently subscribed to ntdev as:
> >>>xxxxx@t-mobile.co.uk To unsubscribe send a
> blank email to
> >>>xxxxx@lists.osr.com
> >>>
> >>
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot
> accept liability
> >>for any breaches of confidence arising through use of email. Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We
> will not
> >>accept responsibility for any commitments made by our employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>—
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> >>xxxxx@t-mobile.co.uk
> >
> >
> >>To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >>—
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> xxxxx@3dlabs.com To
> >>unsubscribe send a blank email to %%email.unsub%%
> >>
> >>
> >
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot accept
> liability
> >>for any breaches of confidence arising through use of email. Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We will not
> >>accept responsibility for any commitments made by our employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>ForwardSourceID:NT00005356
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@vandyke.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@t-mobile.co.uk To unsubscribe send a blank
> email to xxxxx@lists.osr.com
>

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents. We cannot accept liability for any breaches of
confidence arising through use of email. Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions. We will not accept responsibility for any commitments
made by our employees outside the scope of our business. We do not warrant
the accuracy or completeness of such information.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · October 12, 2004, 11:52am

Aside from earlier suggestions:

Try with a standard desktop PC that has built-in serial ports. That
eliminates any (possibly disruptive) PCMCIA drivers that get in the way.
Use Hyperterminal or similar to send data from one standard PC to
another. Vary the bit-rate to see if it’s making any difference. If it does
make a difference, then it’s most likely a performance issue. If it doesn’t
make any difference, stop worrying about performance, and start looking for
logic bugs. Go as low as the hardware allows you, if you can’t get it to
work before then… (You may want to do this in a “binary search” way, so
you don’t have to step through every speed from 115200 down to 300 bps).
Do the same without your filter driver as a sanity check.

Best of luck.

–
Mats

xxxxx@lists.osr.com wrote on 10/12/2004 04:08:06 PM:

Hi Mats and Joseph,
Thanks for your suggestions.
I’m still looking at this. The serial port in question is actually
on a PCMCIA 4-port card and it looks as though the driver for THAT
is causing all the load. However, when changing to a different make
of card whose driver does NOT result in so much load I still have
the same problem! I am now trying to get a free version of the
driver to run (I can’t load it at the moment) so I’ll get back to
you when I’ve sorted that out.
Thanks again
Alasdair
> -----Original Message-----
> From: Joseph Galbraith [mailto:xxxxx@vandyke.com]
> Sent: 12 October 2004 16:02
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Losing Received Data?
>
>
> I would probably do the same here; try breaking in a
> couple times.
>
> I might also try the !process 0 0x17 to dump all the
> processes and tread stacks on the system. This may take
> a while.
>
> Then look for one of the threads hanging out in your filter.
>
> Is it a single CPU or an SMP machine? If you are 60% on a
> dual processor SMP machine, my experience is that you’ve
> might have managed to go into a tight loop on one processor.
> If this is the case, it should be pretty easy to spot.
>
> There are probably some other WinDBG commands I’m
> not familiar with that would help you find a completion
> routine running in an DPC context (which probably
> won’t show up in !process??? I’m not sure on that one.)
>
> Also, it seems like I recall windows being pretty
> chatty with the serial ports, especially during
> startup (looking for modems, mice, etc.) So even
> though you think the serial port is idle, it may
> not really be (completely) idle.
>
> - Joseph
>
> Mats PETERSSON wrote:
> > You could always use VTune (or some other performance
> profiling tool)
> > to figure out what takes time in your driver (or someone elses
> > driver).
> >
> > Debug output that doesn’t actually print anything (if you
> use levels
> > for
> > instance) will still take a fair amount of time, simply
> because of argument
> > passing and calling the debut output function.
> >
> > But to achieve something like 60% CPU performance on a reasonably
> > modern processor, you need to actually do quite a lot. Why
> not break
> > into the system a few times and see where it’s at. (Press
> CTRL-Break
> > and check the stack to see what’s going on). You should
> have a hit in
> > whatever it is that takes a lot of time every other time. This is
> > called “poor mans profiling”, and it actually works much
> better than
> > you’d think.
> >
> > –
> > Mats
> >
> > xxxxx@lists.osr.com wrote on 10/12/2004 08:55:25 AM:
> >
> >
> >>Hi again,
> >>
> >>I noticed that when the driver was running under the debugger the
> >>system load was very high, the red part of the graph
> indicating kernel
> >>load was hovering around 60% when the driver was loaded and with no
> >>serial comms taking place, I have no idea why. So I tried the same
> >>(debug) version of the driver in the non-debug version of W2K to
> >>remove any issues caused by the debugger and it’s serial
> comms, system
> >>load was the same but the dial-up connection succeeded! so
> I think it
> >>is a performance issue rather than a logic
> >
> > issue.
> >
> >>I don’t know why the load should be so high, there are no
> threads in
> >>the driver and as far as I am aware it should just be sitting there
> >>doing nothing, you probably know better.
> >>
> >>My next test will be to see what the load is with the free
> version of
> >>the driver.
> >>
> >>Thanks for the suggestions regarding using a test app to thrash the
> >>comms, I may still need to resort to that but for time
> being I think I
> >>will need to track down this load problem.
> >>
> >>As usual, any suggestions greatly appreciated.
> >>
> >>Thanks
> >>
> >>Alasdair
> >>
> >>
> >>-----Original Message-----
> >>From: Alasdair Tompson (external)
> >
> > [mailto:xxxxx@t-mobile.co.uk]
> >
> >>Sent: 11 October 2004 16:58
> >>To: Windows System Software Devs Interest List
> >>Subject: RE: [ntdev] Losing Received Data?
> >
> >
> >>Thanks Mats and Joseph,
> >>Good idea, I am going home now so I’ll look into that tomorrow.
> >>Alasdair
> >>
> >>
> >>>-----Original Message-----
> >>>From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
> >>>Sent: 11 October 2004 16:55
> >>>To: Windows System Software Devs Interest List
> >>>Subject: Re: [ntdev] Losing Received Data?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>PS. You can probably write a simple usermode program
> >>>>to run on the other side and generate this problem…
> >>>>just open the comm port and write a pretty good burst
> >>>>of data.
> >>>
> >>>This is a good idea. Instead of using the phone (which of
> course is
> >>>the ultimate target), use a PC with either standard of custom
> >>>software on it.
> >>>
> >>>>You can probably also do it using hyperterms file
> >>>>transfer facility (ZModem/Xmodem I think it supports
> >>>>at least one of these.)
> >>>>
> >>>>Typing in hyperterm isn’t a good test since even with
> >>>>all the debugging, the computer is still probably faster than you
> >>>>are.
> >>>
> >>>I don’t know if Hyperterm supports it, but definitely some other
> >>>terminal programs (TeraTerm comes to mind, but I’m unsure if this
> >>>feature is
> >>>included) support logging and transmitting of text-files (ASCII
> >>>transfer mode). So buiild a text-file with a heap of text that you
> >>>can identify if it’s been transmitted correctly at the other end,
> >>>with varying lenghts and “no protocol” (because using a
> protocol just
> >>>confuses the issue) and send some data across.
> >>>
> >>>If you write a little test-app, then one test method should be to
> >>>transmit a file and compare at the other end (so you start the
> >>>compare first, then start the transmit, and the test-app
> shows which
> >>>lines come across badly). That way, you can test different
> patterns,
> >>>sizes, etc, etc, and change the test-setup without having
> to do too
> >>>much programming.
> >>>
> >>>Writing custom test-tools is a big part of device drivers, because
> >>>you need to be able to do “directed testing”, e.g. testing special
> >>>cases without hitting some other stuff that is currently
> not working,
> >>>etc, etc.
> >>>
> >>>
> >>>–
> >>>Mats
> >>>
> >>>
> >>>
> >>>—
> >>>Questions? First check the Kernel Driver FAQ at
> >>>http://www.osronline.com/article.cfm?id=256
> >>>
> >>>You are currently subscribed to ntdev as:
> >>>xxxxx@t-mobile.co.uk To unsubscribe send a
> blank email to
> >>>xxxxx@lists.osr.com
> >>>
> >>
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot
> accept liability
> >>for any breaches of confidence arising through use of email. Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We
> will not
> >>accept responsibility for any commitments made by our employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>—
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> >>xxxxx@t-mobile.co.uk
> >
> >
> >>To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >>—
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> xxxxx@3dlabs.com To
> >>unsubscribe send a blank email to %%email.unsub%%
> >>
> >>
> >
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot accept
> liability
> >>for any breaches of confidence arising through use of email. Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We will not
> >>accept responsibility for any commitments made by our employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>ForwardSourceID:NT00005356
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@vandyke.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@t-mobile.co.uk To unsubscribe send a blank
> email to xxxxx@lists.osr.com
>

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately
and delete this email from your system without copying or
disseminating it or placing any reliance upon its contents. We
cannot accept liability for any breaches of confidence arising
through use of email. Any opinions expressed in this email
(including attachments) are those of the author and do not
necessarily reflect our opinions. We will not accept responsibility
for any commitments made by our employees outside the scope of our
business. We do not warrant the accuracy or completeness of such
information.

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
ForwardSourceID:NT000053CE

OSR_Community_User · October 12, 2004, 7:26pm

Kernrate is another profiler that can be downloaded from microsoft.com.
Just google for it. The URL is too ugly to include here.

/George V. Reilly xxxxx@microsoft.com

This posting is provided "AS IS" with no warranties, and confers no
rights.

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Tuesday, October 12, 2004 8:37 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Losing Received Data?

Really sorry to step into this thread, as Mats and Joseph pointed out
lots of ways to tackle this ...

As they pointed out, one easy way is to profile the drivers, using
VTune, numega code coverage (True Coverage ) etc ...

There are two situation ( either your driver is on the path of execution
for some scenairo or not ). Determining that is important !. If, by any
chance, you can not
comeup with some scenarios where your driver is totally inactive in the
execution path(s) ( often happens for some filter drivers - as an
example a file system filter driver for file activities ) then very
likely you will find that your driver might have quite a bit of kernel
mode waits for various reasons ...

Recently I was trying to find the performance effects of various virus
scanners ( often implemented as file system filters ). Found that lot of
the time CPU is carrying 50 to 60 % load, but some out liers (ie one in
a while spike for near 100% ). When trapped thru getting usr time and
kernel time, we see there are lot of kernel mode waits ...

Just in case if it gives you any more pointers !!!

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Alasdair Tompson
(external)
Sent: Tuesday, October 12, 2004 8:08 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Losing Received Data?

Hi Mats and Joseph,

Thanks for your suggestions.

I'm still looking at this. The serial port in question is
actually on a PCMCIA 4-port card and it looks as though the driver for
THAT is causing all the load. However, when changing to a different make
of card whose driver does NOT result in so much load I still have the
same problem! I am now trying to get a free version of the driver to run
(I can't load it at the moment) so I'll get back to you when I've sorted
that out.

Thanks again

Alasdair

-----Original Message-----
> From: Joseph Galbraith [mailto:xxxxx@vandyke.com]
> Sent: 12 October 2004 16:02
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Losing Received Data?
>
>
> I would probably do the same here; try breaking in a
> couple times.
>
> I might also try the !process 0 0x17 to dump all the
> processes and tread stacks on the system. This may take
> a while.
>
> Then look for one of the threads hanging out in your filter.
>
> Is it a single CPU or an SMP machine? If you are 60% on a
> dual processor SMP machine, my experience is that you've
> might have managed to go into a tight loop on one processor.
> If this is the case, it should be pretty easy to spot.
>
> There are probably some other WinDBG commands I'm
> not familiar with that would help you find a completion
> routine running in an DPC context (which probably
> won't show up in !process??? I'm not sure on that one.)
>
> Also, it seems like I recall windows being pretty
> chatty with the serial ports, especially during
> startup (looking for modems, mice, etc.) So even
> though you think the serial port is idle, it may
> not really be (completely) idle.
>
> - Joseph
>
> Mats PETERSSON wrote:
> > You could always use VTune (or some other performance
> profiling tool)
> > to figure out what takes time in your driver (or someone
elses
> > driver).
> >
> > Debug output that doesn't actually print anything (if you
> use levels
> > for
> > instance) will still take a fair amount of time, simply
> because of argument
> > passing and calling the debut output function.
> >
> > But to achieve something like 60% CPU performance on a
reasonably
> > modern processor, you need to actually do quite a lot. Why
> not break
> > into the system a few times and see where it's at. (Press
> CTRL-Break
> > and check the stack to see what's going on). You should
> have a hit in
> > whatever it is that takes a lot of time every other time.
This is
> > called "poor mans profiling", and it actually works much
> better than
> > you'd think.
> >
> > --
> > Mats
> >
> > xxxxx@lists.osr.com wrote on 10/12/2004
08:55:25 AM:
> >
> >
> >>Hi again,
> >>
> >>I noticed that when the driver was running under the
debugger the
> >>system load was very high, the red part of the graph
> indicating kernel
> >>load was hovering around 60% when the driver was loaded and
with no
> >>serial comms taking place, I have no idea why. So I tried
the same
> >>(debug) version of the driver in the non-debug version of
W2K to
> >>remove any issues caused by the debugger and it's serial
> comms, system
> >>load was the same but the dial-up connection succeeded! so
> I think it
> >>is a performance issue rather than a logic
> >
> > issue.
> >
> >>I don't know why the load should be so high, there are no
> threads in
> >>the driver and as far as I am aware it should just be
sitting there
> >>doing nothing, you probably know better.
> >>
> >>My next test will be to see what the load is with the free
> version of
> >>the driver.
> >>
> >>Thanks for the suggestions regarding using a test app to
thrash the
> >>comms, I may still need to resort to that but for time
> being I think I
> >>will need to track down this load problem.
> >>
> >>As usual, any suggestions greatly appreciated.
> >>
> >>Thanks
> >>
> >>Alasdair
> >>
> >>
> >>-----Original Message-----
> >>From: Alasdair Tompson (external)
> >
> > [mailto:xxxxx@t-mobile.co.uk]
> >
> >>Sent: 11 October 2004 16:58
> >>To: Windows System Software Devs Interest List
> >>Subject: RE: [ntdev] Losing Received Data?
> >
> >
> >>Thanks Mats and Joseph,
> >>Good idea, I am going home now so I'll look into that
tomorrow.
> >>Alasdair
> >>
> >>
> >>>-----Original Message-----
> >>>From: Mats PETERSSON [mailto:xxxxx@3dlabs.com]
> >>>Sent: 11 October 2004 16:55
> >>>To: Windows System Software Devs Interest List
> >>>Subject: Re: [ntdev] Losing Received Data?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>PS. You can probably write a simple usermode program
> >>>>to run on the other side and generate this problem...
> >>>>just open the comm port and write a pretty good burst
> >>>>of data.
> >>>
> >>>This is a good idea. Instead of using the phone (which of
> course is
> >>>the ultimate target), use a PC with either standard of
custom
> >>>software on it.
> >>>
> >>>>You can probably also do it using hyperterms file
> >>>>transfer facility (ZModem/Xmodem I think it supports
> >>>>at least one of these.)
> >>>>
> >>>>Typing in hyperterm isn't a good test since even with
> >>>>all the debugging, the computer is still probably faster
than you
> >>>>are.
> >>>
> >>>I don't know if Hyperterm supports it, but definitely some
other
> >>>terminal programs (TeraTerm comes to mind, but I'm unsure
if this
> >>>feature is
> >>>included) support logging and transmitting of text-files
(ASCII
> >>>transfer mode). So buiild a text-file with a heap of text
that you
> >>>can identify if it's been transmitted correctly at the
other end,
> >>>with varying lenghts and "no protocol" (because using a
> protocol just
> >>>confuses the issue) and send some data across.
> >>>
> >>>If you write a little test-app, then one test method should
be to
> >>>transmit a file and compare at the other end (so you start
the
> >>>compare first, then start the transmit, and the test-app
> shows which
> >>>lines come across badly). That way, you can test different
> patterns,
> >>>sizes, etc, etc, and change the test-setup without having
> to do too
> >>>much programming.
> >>>
> >>>Writing custom test-tools is a big part of device drivers,
because
> >>>you need to be able to do "directed testing", e.g. testing
special
> >>>cases without hitting some other stuff that is currently
> not working,
> >>>etc, etc.
> >>>
> >>>
> >>>--
> >>>Mats
> >>>
> >>>
> >>>
> >>>---
> >>>Questions? First check the Kernel Driver FAQ at
> >>>http://www.osronline.com/article.cfm?id=256
> >>>
> >>>You are currently subscribed to ntdev as:
> >>>xxxxx@t-mobile.co.uk To unsubscribe send a
> blank email to
> >>>xxxxx@lists.osr.com
> >>>
> >>
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you
have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot
> accept liability
> >>for any breaches of confidence arising through use of email.
Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We
> will not
> >>accept responsibility for any commitments made by our
employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>---
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> >>xxxxx@t-mobile.co.uk
> >
> >
> >>To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >>---
> >>Questions? First check the Kernel Driver FAQ at http://www.
> >>osronline.com/article.cfm?id=256
> >>
> >>You are currently subscribed to ntdev as:
> xxxxx@3dlabs.com To
> >>unsubscribe send a blank email to %%email.unsub%%
> >>
> >>
> >
> >
> >>NOTICE AND DISCLAIMER:
> >>This email (including attachments) is confidential. If you
have
> >>received this email in error please notify the sender
> immediately and
> >>delete this email from your system without copying or
> disseminating it
> >>or placing any reliance upon its contents. We cannot accept
> liability
> >>for any breaches of confidence arising through use of email.
Any
> >>opinions expressed in this email (including attachments)
> are those of
> >>the author and do not necessarily reflect our opinions. We
will not
> >>accept responsibility for any commitments made by our
employees
> >>outside the scope of our business. We do not warrant the
> accuracy or
> >>completeness of such
> >
> > information.
> >
> >>ForwardSourceID:NT00005356
> >
> >
> >
> >
> > ---
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@vandyke.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >
>
>
>
> ---
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@t-mobile.co.uk To unsubscribe send a blank
> email to xxxxx@lists.osr.com
>

NOTICE AND DISCLAIMER:
This email (including attachments) is confidential. If you have
received this email in error please notify the sender immediately and
delete this email from your system without copying or disseminating it
or placing any reliance upon its contents. We cannot accept liability
for any breaches of confidence arising through use of email. Any
opinions expressed in this email (including attachments) are those of
the author and do not necessarily reflect our opinions. We will not
accept responsibility for any commitments made by our employees outside
the scope of our business. We do not warrant the accuracy or
completeness of such information.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ''
To unsubscribe send a blank email to
xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
''
To unsubscribe send a blank email to xxxxx@lists.osr.com

Losing Received Data?

To unsubscribe send a blank email to xxxxx@lists.osr.com

Kernrate is another profiler that can be downloaded from microsoft.com. Just google for it. The URL is too ugly to include here.

Kernrate is another profiler that can be downloaded from microsoft.com.
Just google for it. The URL is too ugly to include here.