A few points:
- Your code looks fine without queuing.
- The queuing issue I mentioned in my pseudo code was something I have seen
that caused a race condition leading to leaking of NDIS packet resources,
which I thought may be causing your problem. The queuing was needed for a
number of reasons in that particular application, but may not apply to your
case. For example, queuing can be employed to detect out-of-order packet
completion, for sanity checking, or is needed when this is the protocol half
of an intermediate driver that needs to perform packet processing.
- Are you binding to a stock hardware NIC driver or is it your NIC driver
as well? What I'm getting at is, do you see a matching number of
SendCompleteHandler() calls as the number of NdisSend() calls you make?
- The code as it is shown should not have any problems being called
multiple times by multiple threads, as there are no shared data shown.
However, as this is probably just a skeleton of your actual driver you may
want to double check if there are any shared data structure being accessed
between the SendFunction() and the SendComplete() function that may
contribute to a race condition.
- The fact that you are seeing this allocate failure easily with SoftICE
breakpointing is something to ponder with respect to the underlying NIC
driver completing the packets. If the underlying NIC driver is a stock
hardware driver, you might consider changing the network card (thus a
different NIC driver) to see if the behavior changes, and hopefully get more
clues.
Hope it helps.
Alan Kan
xxxxx@flyingeaglesoftware.com
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mayank Kumar
Sent: Monday, May 10, 2004 10:38 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
Hi Alan
First of all i would like to thank u for taking this pain
to explain me all this in such a great detail. But i still dont
see any point in queuing the packets. Suppose i dont
queue the packets to any internal queue and i am handling
the return value of ndissend then why should the problem
occur.
My piece of code which is causing the problem looks like this
:-
YourSendFunction()
{
loop:
packet = NdisAllocatePacket(mypool);
NdisAllocateBuffer()
NdisChainBufferAtFront()
status = NdisSend(packet);
if (status != NDIS_STATUS_PENDING)
{
SendCompleteHandler(packet)
}
go to loop 2 times
}
SendCompleteHandler(pkt)
{
NdisGetFirstBufferFromPacket(
pkt,&pNdisBuffer,&pVirtualAdrs,&BufLen,&TotalLen);
NdisFreeMemory(pVirtualAdrs,BufLen,0);
NdisFreeBuffer(pNdisBuffer);
NdisReinitializePacket(pkt);
NdisFreePacket(pkt);
}
my NdisSend is being called 2 Or more times in a row before the ndissend
function exits
Can this cause a problem. Is queuing still required here . One other issue
is that the sendfunction()
is called 8 or more times in a ms as per the number of active flows and
within a flow it is called on a
per ms basis.
Thanks and regds
Mayank
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Alan Kan
Sent: Friday, May 07, 2004 8:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
The best way to understand the behavior is to set breakpoints in the send
handler and the statement immediately following the NdisSend(Packet)
function, and you may see the send handler breakpoints before the next
statement following the call to NdisSend(Packet) breakpoints. This behavior
is dependent on the underlying NIC driver. If this is the case, and if you
don't handle the freeing of the packet correctly based on the return value
of the NdisSend(Packet) function, you may not free some of the packets.
Let's use the following pseudo code:
YourSendFunction()
{
packet = NdisAllocatePacket(mypool);
NdisSend(packet);
<<< your SendCompleteHandler() may be called right here
QueuePacket(packet);
}
SendCompleteHandler(pkt)
{
if (PacketIsFoundInQueue(pkt))
NdisFreePacket(pkt);
}
Under this scenario, when the SendCompleteHandler() is called the pkt cannot
be found in the queue as it has not been enqueued in the YourSendFunction()
yet, thus this pkt will never be freed back to the pool.
As to how SoftICE may make this happen faster, I'm guessing that
YourSendFunction() is being called from the PacketReceiveHandler() directly
or indirectly. When SoftICE breaks in the local machine the remote machine
continues to send the packets to the local machine every ms, and depending
on the hardware NIC, the NIC can continue to receive these packets in its
internal hardware and buffer them, until you let SoftICE run again, and then
you get a flurry of PacketReceiveHandler() calls which results in a flurry
of the above scenario happening, which then triggers the leaking of the
packet descriptors in a hurry. If the NIC driver buffers 500 packets (in
500 ms of time) then you only need to stay in SoftICE breakpoint for 0.5
second to trigger the out-of-packet symptom.
The above behavior is a classic race condition because a function callback
is being involved, you are actually dealing with 2 entry points into your
driver thus you need to protect the shared resource (the packet) between the
2 entry points. What NDIS does in this case is like this:
YourSendFunction() calls
NdisSend(Packet)() function which is inside NDIS
NDIS calls the underlying NIC driver's NicSend() function based on the
binding
The underlying NicSend() function is called and process the packet.
If the underlying NicSend() can complete the packet without the need to set
the status to NDIS_STATUS_PENDING, it will return back to NDIS with the
packet being completed
NDIS notes that the complete is complete
NDIS calls the protocol driver (your driver)'s
SendCompleteHandler()
Your driver's SendCompleteHandler() is called
Your driver's SendCompleteHandler() returns back to NDIS
NDIS resumes running in the NdisSend(Packet)() function
NdisSend(Packet)() function returns back to YourSendFunction()
YourSendFunction() resumes running
Hope this helps.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mayank Kumar
Sent: Friday, May 07, 2004 1:24 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
Hi Alan
Thanks for ur reply
this seems interesting but i really did not understand it all.
What do u mean by "do your freeing only after the send function is complete"
The status flags is being checked and only if the flag in not PENDING then
the sendcomplete handler is being called.
Do u mean that after NdisSend returns and before the function that called it
terminates, the SendComplete handler could be called for that packet. In
this case
does the SendComplete handler preempt this function or waits or this
invocation
of it is lost.
Thanks and Regards
Mayank
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Alan Kan
Sent: Friday, May 07, 2004 10:15 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
Thought of something more right after I've posted my initial reply.
Sometimes the SendComplete comes back before your original function that
calls NdisSend is allowed to run to completion, so if you do your freeing
only after the send function is complete then you may lose the send complete
handlers being called. My way of solving it (and there could be other
solutions) is to chain the outgoing packet to my internal queue waiting for
the status to be pending, then call NdisSend(Packet)(), and then in the
SendCompleteHandler() unchain the packet and free it back to the pool. This
scheme works whether the send complete comes back before the sending
function is allowed to run to completion. This kind of callback by NDIS of
other completion functions before the original requesting function is
allowed to finish happens very often.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alan Kan
Sent: Thursday, May 06, 2004 9:38 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
Did you check for the return status of NdisSend()? If the return value is
anything but NDIS_STATUS_PENDING you have to free the packet back to the
pool or reuse the packet. Only when the return value is NDIS_STATUS_PENDING
when you need to keep the packet around waiting for the
SendCompleteHandler() to be called, and then free it back to the pool.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mayank Kumar
Sent: Thursday, May 06, 2004 9:22 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
HI Jef
i did put a break point and made sure that the sendComplete handler was
being called.
Infact i even counted the number of alloc's and number of frees and found
out that
when ever this problem occurs the number of allocs are greater then number
of frees
by about 500 or even more.
can anybody hint as to what could be causing this??
regds
Mayank
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Curless, Jeffrey
Sent: Thursday, May 06, 2004 8:23 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Ndisallocatepacket failing
I would ask the simpler question first, have you proven that the packets are
getting freed
in your SendComplete handler? Or are you just assuming they are based on
looking at the code.
I'd put a breakpoint in the SendComplete handler and maker sure they really
are being freed.
-Jeff
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Thursday, May 06, 2004 10:46 AM
To: Windows System Software Devs Interest List
Cc: Johnson, Doug
Subject: RE: [ntdev] Ndisallocatepacket failing
Hi, Mayank,
I passed the item to Doug Johnson, our SoftICE support guy, but meanwhile, a
lot of questions immediately pop up. Which version of SoftICE, and which OS
? Also, how big's your nonpaged pool, and how much memory's left after the
issue ? Does it happen when you pop SoftICE remote from another machine ?
Does it happen if you use Visual SoftICE instead of SoftICE ? If you turn on
BoundsChecker to track NdisAllocatePacket calls, and/or the functions in
your driver that call it, do you get anything funny in the event list around
the time of the issue ? Does it ever pop up SoftICE on one of your calls to
NdisAllocatePacket ? Do you get any funny message if you turn DriverMonitor
on ? Is this an SMP machine ? Will BoundsChecker Monitor ever trigger after
your Control-D ? If you use one of the DriverNetworks sample that calls
NdisAllocatePacket, do you get the same problem ?
Inquiring minds want to know ! 
Alberto.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Mayank Kumar
Sent: Thursday, May 06, 2004 3:35 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Ndisallocatepacket failing
Hi All
i have yet another query from u all.
I am encountering a problem with ndis.
When my packet driver is running which
is just responsible for sending and
receving packets, and i do a ctrl D
for opening my Soft Ice Window then
after closing it again, what i observe
is that my calls to NdisallocatePacket starts
failing and sometimes the problem continues
for very long without even receovering.
Does some body have any idea why would this
be happening and why is it not able to recover
from the problem.
The SendComplete handler is specified and
it frees packet descriptors.
Pls suggest something
Regds
Mayank
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@compuware.com To
unsubscribe send a blank email to xxxxx@lists.osr.com
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@concord.com To
unsubscribe send a blank email to xxxxx@lists.osr.com
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.
This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@intersolutions.stpn.soft.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@flyingeaglesoftware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com ---
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@flyingeaglesoftware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com ---
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@intersolutions.stpn.soft.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@flyingeaglesoftware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com ---
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@intersolutions.stpn.soft.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@flyingeaglesoftware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
