System Queueing Question

Hi Everybody,

In “Windows NT Device Driver Development” by Viscarola and Mason, there
are a couple of statements that say slightly different things to me (as I
read it).

In Chapter 14 with regard to System Queueing it says “Drivers that use
System Queueing can have only one operation per device in progress at a
time.” Right after that there is a note which says “Using system Queueing
implies that only one IRP can be in progress per device at a time.” The
word “implies” says to me “but not neccesarily”. Thus my confusion.

So my question is, if I use System Queueing why can’t I take the IRP
that is being processed by my StartIo routine, stash the pointer away,
remove it from the queue, and then call IoStartNextPacket, and complete the
original IRP sometime later when I see fit? What problem’s might I
eventually run into.

My device is a PCI device and the IRP in question specifically does not
touch the hardware (but rather is simply waiting for a periodic event
message to come back from the device). The OS is NT4.0 SP6a.

Thanks,
Joe

There’s nothing wrong with what you want to do but technically, your StartIo
is still only processing one IRP at a time. You are still basically
serializing the processing of your IRP’s…

  • Dennis

Dennis Merrill
Embedded Systems Engineer
Thermo Electron Corporation
Spectroscopy Division

-----Original Message-----
From: lists.osr.com [mailto:xxxxx@voicenet.com]
Sent: Tuesday, June 18, 2002 9:16 AM
To: NT Developers Interest List
Subject: [ntdev] System Queueing Question

Hi Everybody,

In “Windows NT Device Driver Development” by Viscarola and Mason, there
are a couple of statements that say slightly different things to me (as I
read it).

In Chapter 14 with regard to System Queueing it says “Drivers that use
System Queueing can have only one operation per device in progress at a
time.” Right after that there is a note which says “Using system Queueing
implies that only one IRP can be in progress per device at a time.” The
word “implies” says to me “but not neccesarily”. Thus my confusion.

So my question is, if I use System Queueing why can’t I take the IRP
that is being processed by my StartIo routine, stash the pointer away,
remove it from the queue, and then call IoStartNextPacket, and complete the
original IRP sometime later when I see fit? What problem’s might I
eventually run into.

My device is a PCI device and the IRP in question specifically does not
touch the hardware (but rather is simply waiting for a periodic event
message to come back from the device). The OS is NT4.0 SP6a.

Thanks,
Joe


You are currently subscribed to ntdev as: xxxxx@thermonicolet.com
To unsubscribe send a blank email to %%email.unsub%%

the system queueing is there to allow your driver to process one request
at a time. If processing means building some HW context structure,
saving an IRP pointer in it, passing the request off to hardware then
asking for the next request this is entirely fine. Just remember that
you may need to synchronize your ISR/DPC/completion code with itself and
your StartIo routine.

scsiport uses startio quite heavily but it can have more than one
outstanding request at a time. It only attempts to dispatch one at a
time to the hardware (well the miniport) and system supplied queuing is
part of this.

-p

-----Original Message-----
From: lists.osr.com [mailto:xxxxx@voicenet.com]
Sent: Tuesday, June 18, 2002 7:16 AM
To: NT Developers Interest List
Subject: [ntdev] System Queueing Question

Hi Everybody,

In “Windows NT Device Driver Development” by Viscarola and Mason,
there are a couple of statements that say slightly different things to
me (as I read it).

In Chapter 14 with regard to System Queueing it says “Drivers that
use System Queueing can have only one operation per device in progress
at a time.” Right after that there is a note which says “Using system
Queueing implies that only one IRP can be in progress per device at a
time.” The word “implies” says to me “but not neccesarily”. Thus my
confusion.

So my question is, if I use System Queueing why can’t I take the IRP
that is being processed by my StartIo routine, stash the pointer away,
remove it from the queue, and then call IoStartNextPacket, and complete
the original IRP sometime later when I see fit? What problem’s might I
eventually run into.

My device is a PCI device and the IRP in question specifically does
not touch the hardware (but rather is simply waiting for a periodic
event message to come back from the device). The OS is NT4.0 SP6a.

Thanks,
Joe


You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to %%email.unsub%%

> So my question is, if I use System Queueing why can’t I take the
IRP

that is being processed by my StartIo routine, stash the pointer
away,
remove it from the queue, and then call IoStartNextPacket, and
complete the
original IRP sometime later when I see fit? What problem’s might I
eventually run into.

No problems.
Only 1 IRP will be delivered to StartIo queue till IoStartNextPacket
will be called.

On top of StartIo queue, you can build a logic which will deliver not
more then 1, not more then 2, not more then 10 etc. IRPs to the
driver’s core logic.

Max

It would be perfectly reasonable for you to do this, further, you don’t ever
have to put this IRP in the queue to begin with if it doesn’t need to be
queued. It is quite common for a driver to queue a specific type of IRP
say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology, but if
your IRPs don’t need to be serialized, don’t use a queue for them.

Further, I would NEVER use system queuing anymore. The system queue uses
and abuses the system wide cancel spinlock. This spinlock is a single
global resource used by every driver in the system. Over use of this lock
stomps system performance. So, you would be better off using a driver queue
and minimizing the use of this resource. There is a new library available
for 2000 and later, (and maybe earlier OSes, I just don’t know), that is
supposed to help you create a cancel-safe driver IRP queue. It comes with
the latest DDKs. I have never used it, because while I was at BlueWater and
BSquare I used the WinDK toolkit which provided its own cancel-safe IRP
queues. In fact looking at the new library in the DDK I see that the WinDK
functions and the new DDK library functions are AMAZINGLY similar. There is
a sample in the newer DDKs under \src\general\cancel that shows how to use
the new DDK library.

Anyway, I probably told you more than you needed to know here, but I hope I
answered your question.


Bill McKenzie

lists.osr.com” wrote in message news:xxxxx@ntdev…
>
> Hi Everybody,
>
> In “Windows NT Device Driver Development” by Viscarola and Mason,
there
> are a couple of statements that say slightly different things to me (as I
> read it).
>
> In Chapter 14 with regard to System Queueing it says “Drivers that use
> System Queueing can have only one operation per device in progress at a
> time.” Right after that there is a note which says “Using system Queueing
> implies that only one IRP can be in progress per device at a time.” The
> word “implies” says to me “but not neccesarily”. Thus my confusion.
>
> So my question is, if I use System Queueing why can’t I take the IRP
> that is being processed by my StartIo routine, stash the pointer away,
> remove it from the queue, and then call IoStartNextPacket, and complete
the
> original IRP sometime later when I see fit? What problem’s might I
> eventually run into.
>
> My device is a PCI device and the IRP in question specifically does
not
> touch the hardware (but rather is simply waiting for a periodic event
> message to come back from the device). The OS is NT4.0 SP6a.
>
> Thanks,
> Joe
>
>
>
>
>

BTW, just checked out the new cancel-safe queue library documentation and it
is usable on 98/Me, but I have no idea why they incorporated the code into
the OS on XP instead of leaving it a library. As it is, it is impossible to
use this library in one cross platform binary, which is most unfortunate.
Further, that cancel sample in the DDK doesn’t show much. It really doesn’t
show any of the hard stuff related to queuing IRPs, like in-progress
cancellable IRPs, dealing with ISR/ DPC/StartIo synchronization problems,
lots of hard cases. Kind of disappointing really.

Further, I really wish they would have included the source for the library,
as I am curious how they solved some of the delicate synchronization
problems inherent with these queues. Not saying I don’t trust them, but it
would be good to be able to check what is going on here. Having gone
through writing a cancel-safe queue with a team of some of the best
developers in the business, I know how easy it is to get this code wrong.


Bill McKenzie

“Bill McKenzie” wrote in message
news:xxxxx@ntdev…
>
> It would be perfectly reasonable for you to do this, further, you don’t
ever
> have to put this IRP in the queue to begin with if it doesn’t need to be
> queued. It is quite common for a driver to queue a specific type of IRP
> say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology, but
if
> your IRPs don’t need to be serialized, don’t use a queue for them.
>
> Further, I would NEVER use system queuing anymore. The system queue uses
> and abuses the system wide cancel spinlock. This spinlock is a single
> global resource used by every driver in the system. Over use of this lock
> stomps system performance. So, you would be better off using a driver
queue
> and minimizing the use of this resource. There is a new library available
> for 2000 and later, (and maybe earlier OSes, I just don’t know), that is
> supposed to help you create a cancel-safe driver IRP queue. It comes with
> the latest DDKs. I have never used it, because while I was at BlueWater
and
> BSquare I used the WinDK toolkit which provided its own cancel-safe IRP
> queues. In fact looking at the new library in the DDK I see that the
WinDK
> functions and the new DDK library functions are AMAZINGLY similar. There
is
> a sample in the newer DDKs under \src\general\cancel that shows how to use
> the new DDK library.
>
> Anyway, I probably told you more than you needed to know here, but I hope
I
> answered your question.
>
> –
> Bill McKenzie
>
>
>
> “lists.osr.com” wrote in message news:xxxxx@ntdev…
> >
> > Hi Everybody,
> >
> > In “Windows NT Device Driver Development” by Viscarola and Mason,
> there
> > are a couple of statements that say slightly different things to me (as
I
> > read it).
> >
> > In Chapter 14 with regard to System Queueing it says “Drivers that
use
> > System Queueing can have only one operation per device in progress at a
> > time.” Right after that there is a note which says “Using system
Queueing
> > implies that only one IRP can be in progress per device at a time.” The
> > word “implies” says to me “but not neccesarily”. Thus my confusion.
> >
> > So my question is, if I use System Queueing why can’t I take the IRP
> > that is being processed by my StartIo routine, stash the pointer away,
> > remove it from the queue, and then call IoStartNextPacket, and complete
> the
> > original IRP sometime later when I see fit? What problem’s might I
> > eventually run into.
> >
> > My device is a PCI device and the IRP in question specifically does
> not
> > touch the hardware (but rather is simply waiting for a periodic event
> > message to come back from the device). The OS is NT4.0 SP6a.
> >
> > Thanks,
> > Joe
> >
> >
> >
> >
> >
>
>
>
>

Thanks for the info everyone. That is what I was hoping for.

Bill,
No amount of usefull information is too much. Thanks for the insight.
We ARE using the WinDk toolkit. I will investigate useing the IRP queues
provided. I don’t think I’ve really needed to handle the cases of cancel
routines, but I will check it out when I get a free moment. :wink:

The WinDK toolkit was well worth the price in my opinion. And the
support I got from Blue Water Systems was great. I havn’t needed any
support from BSquare, but hopefully they would be just as responsive.

Also, my apologies to the list for the mis-configuration I had of my
news reader. I didn’t mean to have my name sent out as “list.osr.com”. My
bad.

Thanks Again,
Joe

If you have specific guidelines or examples on what the sample should do,
please send feedback to this newgroup or to Microsoft directly and we will
try to improve the sample.


This posting is provided “AS IS” with no warranties, and confers no rights.
“Bill McKenzie” wrote in message
news:xxxxx@ntdev…
>
> BTW, just checked out the new cancel-safe queue library documentation and
it
> is usable on 98/Me, but I have no idea why they incorporated the code into
> the OS on XP instead of leaving it a library. As it is, it is impossible
to
> use this library in one cross platform binary, which is most unfortunate.
> Further, that cancel sample in the DDK doesn’t show much. It really
doesn’t
> show any of the hard stuff related to queuing IRPs, like in-progress
> cancellable IRPs, dealing with ISR/ DPC/StartIo synchronization problems,
> lots of hard cases. Kind of disappointing really.
>
> Further, I really wish they would have included the source for the
library,
> as I am curious how they solved some of the delicate synchronization
> problems inherent with these queues. Not saying I don’t trust them, but
it
> would be good to be able to check what is going on here. Having gone
> through writing a cancel-safe queue with a team of some of the best
> developers in the business, I know how easy it is to get this code wrong.
>
> –
> Bill McKenzie
>
>
>
> “Bill McKenzie” wrote in message
> news:xxxxx@ntdev…
> >
> > It would be perfectly reasonable for you to do this, further, you don’t
> ever
> > have to put this IRP in the queue to begin with if it doesn’t need to be
> > queued. It is quite common for a driver to queue a specific type of IRP
> > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology, but
> if
> > your IRPs don’t need to be serialized, don’t use a queue for them.
> >
> > Further, I would NEVER use system queuing anymore. The system queue
uses
> > and abuses the system wide cancel spinlock. This spinlock is a single
> > global resource used by every driver in the system. Over use of this
lock
> > stomps system performance. So, you would be better off using a driver
> queue
> > and minimizing the use of this resource. There is a new library
available
> > for 2000 and later, (and maybe earlier OSes, I just don’t know), that is
> > supposed to help you create a cancel-safe driver IRP queue. It comes
with
> > the latest DDKs. I have never used it, because while I was at BlueWater
> and
> > BSquare I used the WinDK toolkit which provided its own cancel-safe IRP
> > queues. In fact looking at the new library in the DDK I see that the
> WinDK
> > functions and the new DDK library functions are AMAZINGLY similar.
There
> is
> > a sample in the newer DDKs under \src\general\cancel that shows how to
use
> > the new DDK library.
> >
> > Anyway, I probably told you more than you needed to know here, but I
hope
> I
> > answered your question.
> >
> > –
> > Bill McKenzie
> >
> >
> >
> > “lists.osr.com” wrote in message news:xxxxx@ntdev…
> > >
> > > Hi Everybody,
> > >
> > > In “Windows NT Device Driver Development” by Viscarola and Mason,
> > there
> > > are a couple of statements that say slightly different things to me
(as
> I
> > > read it).
> > >
> > > In Chapter 14 with regard to System Queueing it says “Drivers that
> use
> > > System Queueing can have only one operation per device in progress at
a
> > > time.” Right after that there is a note which says “Using system
> Queueing
> > > implies that only one IRP can be in progress per device at a time.”
The
> > > word “implies” says to me “but not neccesarily”. Thus my confusion.
> > >
> > > So my question is, if I use System Queueing why can’t I take the
IRP
> > > that is being processed by my StartIo routine, stash the pointer
away,
> > > remove it from the queue, and then call IoStartNextPacket, and
complete
> > the
> > > original IRP sometime later when I see fit? What problem’s might I
> > > eventually run into.
> > >
> > > My device is a PCI device and the IRP in question specifically
does
> > not
> > > touch the hardware (but rather is simply waiting for a periodic event
> > > message to come back from the device). The OS is NT4.0 SP6a.
> > >
> > > Thanks,
> > > Joe
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>

> > Further, I really wish they would have included the source for the

library, as I am curious how they solved some of the delicate
synchronization
> problems inherent with these queues. Not saying I don’t trust them, but

The code of cancelapi.obj from csq.lib is really extremly simple, and you
can revert it yourslef to C source with minimum effort,
providing you understand x86 assembely.

>
> –
> Bill McKenzie
>
>
>
> “Bill McKenzie” wrote in message
> > news:xxxxx@ntdev…
> > >
> > > It would be perfectly reasonable for you to do this, further, you
don’t
> > ever
> > > have to put this IRP in the queue to begin with if it doesn’t need to
be
> > > queued. It is quite common for a driver to queue a specific type of
IRP
> > > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology,
but
> > if
> > > your IRPs don’t need to be serialized, don’t use a queue for them.
> > >
> > > Further, I would NEVER use system queuing anymore. The system queue
> uses
> > > and abuses the system wide cancel spinlock. This spinlock is a single
> > > global resource used by every driver in the system. Over use of this
> lock
> > > stomps system performance. So, you would be better off using a driver
> > queue
> > > and minimizing the use of this resource. There is a new library
> available
> > > for 2000 and later, (and maybe earlier OSes, I just don’t know), that
is
> > > supposed to help you create a cancel-safe driver IRP queue. It comes
> with
> > > the latest DDKs. I have never used it, because while I was at
BlueWater
> > and
> > > BSquare I used the WinDK toolkit which provided its own cancel-safe
IRP
> > > queues. In fact looking at the new library in the DDK I see that the
> > WinDK
> > > functions and the new DDK library functions are AMAZINGLY similar.
> There
> > is
> > > a sample in the newer DDKs under \src\general\cancel that shows how to
> use
> > > the new DDK library.
> > >
> > > Anyway, I probably told you more than you needed to know here, but I
> hope
> > I
> > > answered your question.
> > >
> > > –
> > > Bill McKenzie
> > >
> > >
> > >
> > > “lists.osr.com” wrote in message
news:xxxxx@ntdev…
> > > >
> > > > Hi Everybody,
> > > >
> > > > In “Windows NT Device Driver Development” by Viscarola and
Mason,
> > > there
> > > > are a couple of statements that say slightly different things to me
> (as
> > I
> > > > read it).
> > > >
> > > > In Chapter 14 with regard to System Queueing it says “Drivers
that
> > use
> > > > System Queueing can have only one operation per device in progress
at
> a
> > > > time.” Right after that there is a note which says “Using system
> > Queueing
> > > > implies that only one IRP can be in progress per device at a time.”
> The
> > > > word “implies” says to me “but not neccesarily”. Thus my confusion.
> > > >
> > > > So my question is, if I use System Queueing why can’t I take the
> IRP
> > > > that is being processed by my StartIo routine, stash the pointer
> away,
> > > > remove it from the queue, and then call IoStartNextPacket, and
> complete
> > > the
> > > > original IRP sometime later when I see fit? What problem’s might I
> > > > eventually run into.
> > > >
> > > > My device is a PCI device and the IRP in question specifically
> does
> > > not
> > > > touch the hardware (but rather is simply waiting for a periodic
event
> > > > message to come back from the device). The OS is NT4.0 SP6a.
> > > >
> > > > Thanks,
> > > > Joe
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>

Well, I named some specifics, but okay, how about this:

show a DMA device with interrupts, cancellable in-progress IRPs and a DMA
start timeout timer. This hits a lot of the ugly synchronization cases I
can think of. Particularily, you would have an ISR, DPC, timer callback,
and cancel routine all competing to complete the same IRP. And, handling
the case of a piece of DMA hardware that clears the interrupt register on
check, (that is hardware you can only check once to see if it interrupted),
would be good. Not having a hardware check in the DpcForIsr makes checking
for the correct IRP there a virtual nightmare.

A driver with multiple queues, say separate queues for reads and writes,
would be really handy just to show people its possible and how.

Then, something like a serial driver using the new queue library would round
out everything I could imagine anyone would need. The serial driver handles
I/O in a much different manner than a typical DMA driver, multiple
interrupts per IRP, lots of ISR synchronization and such.

So, why didn’t you guys release the source to the library? I can’t think
there would really be any critical IP in there would there? As in, I
wouldn’t expect you are using any non-standard DDK calls. Just curious, it
really surprised me that source for this wasn’t included in the DDK.


Bill McKenzie

“Nar Ganapathy [MS]” wrote in message
news:xxxxx@ntdev…
>
> If you have specific guidelines or examples on what the sample should do,
> please send feedback to this newgroup or to Microsoft directly and we will
> try to improve the sample.
>
> –
> This posting is provided “AS IS” with no warranties, and confers no
rights.
> “Bill McKenzie” wrote in message
> news:xxxxx@ntdev…
> >
> > BTW, just checked out the new cancel-safe queue library documentation
and
> it
> > is usable on 98/Me, but I have no idea why they incorporated the code
into
> > the OS on XP instead of leaving it a library. As it is, it is
impossible
> to
> > use this library in one cross platform binary, which is most
unfortunate.
> > Further, that cancel sample in the DDK doesn’t show much. It really
> doesn’t
> > show any of the hard stuff related to queuing IRPs, like in-progress
> > cancellable IRPs, dealing with ISR/ DPC/StartIo synchronization
problems,
> > lots of hard cases. Kind of disappointing really.
> >
> > Further, I really wish they would have included the source for the
> library,
> > as I am curious how they solved some of the delicate synchronization
> > problems inherent with these queues. Not saying I don’t trust them, but
> it
> > would be good to be able to check what is going on here. Having gone
> > through writing a cancel-safe queue with a team of some of the best
> > developers in the business, I know how easy it is to get this code
wrong.
> >
> > –
> > Bill McKenzie
> >
> >
> >
> > “Bill McKenzie” wrote in message
> > news:xxxxx@ntdev…
> > >
> > > It would be perfectly reasonable for you to do this, further, you
don’t
> > ever
> > > have to put this IRP in the queue to begin with if it doesn’t need to
be
> > > queued. It is quite common for a driver to queue a specific type of
IRP
> > > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology,
but
> > if
> > > your IRPs don’t need to be serialized, don’t use a queue for them.
> > >
> > > Further, I would NEVER use system queuing anymore. The system queue
> uses
> > > and abuses the system wide cancel spinlock. This spinlock is a single
> > > global resource used by every driver in the system. Over use of this
> lock
> > > stomps system performance. So, you would be better off using a driver
> > queue
> > > and minimizing the use of this resource. There is a new library
> available
> > > for 2000 and later, (and maybe earlier OSes, I just don’t know), that
is
> > > supposed to help you create a cancel-safe driver IRP queue. It comes
> with
> > > the latest DDKs. I have never used it, because while I was at
BlueWater
> > and
> > > BSquare I used the WinDK toolkit which provided its own cancel-safe
IRP
> > > queues. In fact looking at the new library in the DDK I see that the
> > WinDK
> > > functions and the new DDK library functions are AMAZINGLY similar.
> There
> > is
> > > a sample in the newer DDKs under \src\general\cancel that shows how to
> use
> > > the new DDK library.
> > >
> > > Anyway, I probably told you more than you needed to know here, but I
> hope
> > I
> > > answered your question.
> > >
> > > –
> > > Bill McKenzie
> > >
> > >
> > >
> > > “lists.osr.com” wrote in message
news:xxxxx@ntdev…
> > > >
> > > > Hi Everybody,
> > > >
> > > > In “Windows NT Device Driver Development” by Viscarola and
Mason,
> > > there
> > > > are a couple of statements that say slightly different things to me
> (as
> > I
> > > > read it).
> > > >
> > > > In Chapter 14 with regard to System Queueing it says “Drivers
that
> > use
> > > > System Queueing can have only one operation per device in progress
at
> a
> > > > time.” Right after that there is a note which says “Using system
> > Queueing
> > > > implies that only one IRP can be in progress per device at a time.”
> The
> > > > word “implies” says to me “but not neccesarily”. Thus my confusion.
> > > >
> > > > So my question is, if I use System Queueing why can’t I take the
> IRP
> > > > that is being processed by my StartIo routine, stash the pointer
> away,
> > > > remove it from the queue, and then call IoStartNextPacket, and
> complete
> > > the
> > > > original IRP sometime later when I see fit? What problem’s might I
> > > > eventually run into.
> > > >
> > > > My device is a PCI device and the IRP in question specifically
> does
> > > not
> > > > touch the hardware (but rather is simply waiting for a periodic
event
> > > > message to come back from the device). The OS is NT4.0 SP6a.
> > > >
> > > > Thanks,
> > > > Joe
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>

> Further, I would NEVER use system queuing anymore. The system queue
uses

and abuses the system wide cancel spinlock. This spinlock is a
single
global resource used by every driver in the system. Over use of
this lock
stomps system performance. So, you would be better off using a
driver queue

Another issue is that IoStartPacket queue is not safe from recursion
on OSes prior to XP, and requires a call to IoSetStartIoAttributes to
be safe on XP.

Max

Well, okay, I suppose that would be true of the whole OS. But, my
motivation is not just so I can see what is going on, rather so anyone can
see and see easily. Why make this opaque? Just curious.


Bill McKenzie

“Dan Partelly” wrote in message news:xxxxx@ntdev…
>
> > > Further, I really wish they would have included the source for the
> > library, as I am curious how they solved some of the delicate
> synchronization
> > > problems inherent with these queues. Not saying I don’t trust them,
but
>
>
> The code of cancelapi.obj from csq.lib is really extremly simple, and you
> can revert it yourslef to C source with minimum effort,
> providing you understand x86 assembely.
>
>
>
>
>
> > >
> > > –
> > > Bill McKenzie
> > >
> > >
> > >
> > > “Bill McKenzie” wrote in message
> > > news:xxxxx@ntdev…
> > > >
> > > > It would be perfectly reasonable for you to do this, further, you
> don’t
> > > ever
> > > > have to put this IRP in the queue to begin with if it doesn’t need
to
> be
> > > > queued. It is quite common for a driver to queue a specific type of
> IRP
> > > > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > > > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization methodology,
> but
> > > if
> > > > your IRPs don’t need to be serialized, don’t use a queue for them.
> > > >
> > > > Further, I would NEVER use system queuing anymore. The system queue
> > uses
> > > > and abuses the system wide cancel spinlock. This spinlock is a
single
> > > > global resource used by every driver in the system. Over use of
this
> > lock
> > > > stomps system performance. So, you would be better off using a
driver
> > > queue
> > > > and minimizing the use of this resource. There is a new library
> > available
> > > > for 2000 and later, (and maybe earlier OSes, I just don’t know),
that
> is
> > > > supposed to help you create a cancel-safe driver IRP queue. It
comes
> > with
> > > > the latest DDKs. I have never used it, because while I was at
> BlueWater
> > > and
> > > > BSquare I used the WinDK toolkit which provided its own cancel-safe
> IRP
> > > > queues. In fact looking at the new library in the DDK I see that
the
> > > WinDK
> > > > functions and the new DDK library functions are AMAZINGLY similar.
> > There
> > > is
> > > > a sample in the newer DDKs under \src\general\cancel that shows how
to
> > use
> > > > the new DDK library.
> > > >
> > > > Anyway, I probably told you more than you needed to know here, but I
> > hope
> > > I
> > > > answered your question.
> > > >
> > > > –
> > > > Bill McKenzie
> > > >
> > > >
> > > >
> > > > “lists.osr.com” wrote in message
> news:xxxxx@ntdev…
> > > > >
> > > > > Hi Everybody,
> > > > >
> > > > > In “Windows NT Device Driver Development” by Viscarola and
> Mason,
> > > > there
> > > > > are a couple of statements that say slightly different things to
me
> > (as
> > > I
> > > > > read it).
> > > > >
> > > > > In Chapter 14 with regard to System Queueing it says “Drivers
> that
> > > use
> > > > > System Queueing can have only one operation per device in progress
> at
> > a
> > > > > time.” Right after that there is a note which says “Using system
> > > Queueing
> > > > > implies that only one IRP can be in progress per device at a
time.”
> > The
> > > > > word “implies” says to me “but not neccesarily”. Thus my
confusion.
> > > > >
> > > > > So my question is, if I use System Queueing why can’t I take
the
> > IRP
> > > > > that is being processed by my StartIo routine, stash the pointer
> > away,
> > > > > remove it from the queue, and then call IoStartNextPacket, and
> > complete
> > > > the
> > > > > original IRP sometime later when I see fit? What problem’s might
I
> > > > > eventually run into.
> > > > >
> > > > > My device is a PCI device and the IRP in question specifically
> > does
> > > > not
> > > > > touch the hardware (but rather is simply waiting for a periodic
> event
> > > > > message to come back from the device). The OS is NT4.0 SP6a.
> > > > >
> > > > > Thanks,
> > > > > Joe
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>

No it doesn’t. It just requires that you check to see if the queue is busy,
and if it is queue a DPC to call IoStartPacket.


Bill McKenzie

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>
> > Further, I would NEVER use system queuing anymore. The system queue
> uses
> > and abuses the system wide cancel spinlock. This spinlock is a
> single
> > global resource used by every driver in the system. Over use of
> this lock
> > stomps system performance. So, you would be better off using a
> driver queue
>
> Another issue is that IoStartPacket queue is not safe from recursion
> on OSes prior to XP, and requires a call to IoSetStartIoAttributes to
> be safe on XP.
>
> Max
>
>
>
>

The source code issue is a good question. We are still wrestling with giving
out source code with the appropriate legal formalities here at Microsoft. We
have a lot of technical and other concerns. We brought some of these issues
at the WINHEC BOF.

Having said that the cancel safe queue implementation is pretty simple. It
just follows the standard DDK rules. The only part which might be slightly
new is the use of the DriverContext field in the IRP to associate the IRP
with a piece of context that represents an outstanding IO that’s going to be
completed via timer DPC or interrupt. This ensure that driver does not
accidentally queue the IRP and use the IRP pointer itself as the context
(that’s returned by the timer or DPC) and either reference it directly or
search for the prescence of the IRP in a queue (this is a bug because the
IRP could have been completed and reissued). The cancel safe queue does not
acquire the cancel spinlock at all and hence has no global performance
impact.

“Bill McKenzie” wrote in message
news:xxxxx@ntdev…
>
> Well, okay, I suppose that would be true of the whole OS. But, my
> motivation is not just so I can see what is going on, rather so anyone can
> see and see easily. Why make this opaque? Just curious.
>
> –
> Bill McKenzie
>
>
>
> “Dan Partelly” wrote in message
news:xxxxx@ntdev…
> >
> > > > Further, I really wish they would have included the source for the
> > > library, as I am curious how they solved some of the delicate
> > synchronization
> > > > problems inherent with these queues. Not saying I don’t trust them,
> but
> >
> >
> > The code of cancelapi.obj from csq.lib is really extremly simple, and
you
> > can revert it yourslef to C source with minimum effort,
> > providing you understand x86 assembely.
> >
> >
> >
> >
> >
> > > >
> > > > –
> > > > Bill McKenzie
> > > >
> > > >
> > > >
> > > > “Bill McKenzie” wrote in message
> > > > news:xxxxx@ntdev…
> > > > >
> > > > > It would be perfectly reasonable for you to do this, further, you
> > don’t
> > > > ever
> > > > > have to put this IRP in the queue to begin with if it doesn’t need
> to
> > be
> > > > > queued. It is quite common for a driver to queue a specific type
of
> > IRP
> > > > > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > > > > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization
methodology,
> > but
> > > > if
> > > > > your IRPs don’t need to be serialized, don’t use a queue for them.
> > > > >
> > > > > Further, I would NEVER use system queuing anymore. The system
queue
> > > uses
> > > > > and abuses the system wide cancel spinlock. This spinlock is a
> single
> > > > > global resource used by every driver in the system. Over use of
> this
> > > lock
> > > > > stomps system performance. So, you would be better off using a
> driver
> > > > queue
> > > > > and minimizing the use of this resource. There is a new library
> > > available
> > > > > for 2000 and later, (and maybe earlier OSes, I just don’t know),
> that
> > is
> > > > > supposed to help you create a cancel-safe driver IRP queue. It
> comes
> > > with
> > > > > the latest DDKs. I have never used it, because while I was at
> > BlueWater
> > > > and
> > > > > BSquare I used the WinDK toolkit which provided its own
cancel-safe
> > IRP
> > > > > queues. In fact looking at the new library in the DDK I see that
> the
> > > > WinDK
> > > > > functions and the new DDK library functions are AMAZINGLY similar.
> > > There
> > > > is
> > > > > a sample in the newer DDKs under \src\general\cancel that shows
how
> to
> > > use
> > > > > the new DDK library.
> > > > >
> > > > > Anyway, I probably told you more than you needed to know here, but
I
> > > hope
> > > > I
> > > > > answered your question.
> > > > >
> > > > > –
> > > > > Bill McKenzie
> > > > >
> > > > >
> > > > >
> > > > > “lists.osr.com” wrote in message
> > news:xxxxx@ntdev…
> > > > > >
> > > > > > Hi Everybody,
> > > > > >
> > > > > > In “Windows NT Device Driver Development” by Viscarola and
> > Mason,
> > > > > there
> > > > > > are a couple of statements that say slightly different things to
> me
> > > (as
> > > > I
> > > > > > read it).
> > > > > >
> > > > > > In Chapter 14 with regard to System Queueing it says
“Drivers
> > that
> > > > use
> > > > > > System Queueing can have only one operation per device in
progress
> > at
> > > a
> > > > > > time.” Right after that there is a note which says “Using
system
> > > > Queueing
> > > > > > implies that only one IRP can be in progress per device at a
> time.”
> > > The
> > > > > > word “implies” says to me “but not neccesarily”. Thus my
> confusion.
> > > > > >
> > > > > > So my question is, if I use System Queueing why can’t I take
> the
> > > IRP
> > > > > > that is being processed by my StartIo routine, stash the
pointer
> > > away,
> > > > > > remove it from the queue, and then call IoStartNextPacket, and
> > > complete
> > > > > the
> > > > > > original IRP sometime later when I see fit? What problem’s
might
> I
> > > > > > eventually run into.
> > > > > >
> > > > > > My device is a PCI device and the IRP in question
specifically
> > > does
> > > > > not
> > > > > > touch the hardware (but rather is simply waiting for a periodic
> > event
> > > > > > message to come back from the device). The OS is NT4.0 SP6a.
> > > > > >
> > > > > > Thanks,
> > > > > > Joe
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
>
>
>
>

>this is a bug because the

IRP could have been completed and reissued

This was actually chief of my concerns. I have seen quite a few queue
implementations that have not accounted for the same IRP being completed
coming back and becoming the upfront IRP in the queue, but this is certainly
possible. Actually, I should have noticed this context use in your
implementation, if for no other reason it was mentioned in the NT Insider.
Its like a magic number for the IRP or something right?

I guess my statement about the cross-platform binary was bogus as well, you
just can’t build the same with each DDK. That is no big deal. I think I
checked my brain yesterday, that is fairly obvious this morning :slight_smile:

This library is pretty cool Nar! I love the flexibility you left to the
user, such that the queue can be setup in many different configurations.
Good design.


Bill McKenzie

“Nar Ganapathy [MS]” wrote in message
news:xxxxx@ntdev…
>
> The source code issue is a good question. We are still wrestling with
giving
> out source code with the appropriate legal formalities here at Microsoft.
We
> have a lot of technical and other concerns. We brought some of these
issues
> at the WINHEC BOF.
>
> Having said that the cancel safe queue implementation is pretty simple. It
> just follows the standard DDK rules. The only part which might be slightly
> new is the use of the DriverContext field in the IRP to associate the IRP
> with a piece of context that represents an outstanding IO that’s going to
be
> completed via timer DPC or interrupt. This ensure that driver does not
> accidentally queue the IRP and use the IRP pointer itself as the context
> (that’s returned by the timer or DPC) and either reference it directly or
> search for the prescence of the IRP in a queue (this is a bug because the
> IRP could have been completed and reissued). The cancel safe queue does
not
> acquire the cancel spinlock at all and hence has no global performance
> impact.
>
> “Bill McKenzie” wrote in message
> news:xxxxx@ntdev…
> >
> > Well, okay, I suppose that would be true of the whole OS. But, my
> > motivation is not just so I can see what is going on, rather so anyone
can
> > see and see easily. Why make this opaque? Just curious.
> >
> > –
> > Bill McKenzie
> >
> >
> >
> > “Dan Partelly” wrote in message
> news:xxxxx@ntdev…
> > >
> > > > > Further, I really wish they would have included the source for the
> > > > library, as I am curious how they solved some of the delicate
> > > synchronization
> > > > > problems inherent with these queues. Not saying I don’t trust
them,
> > but
> > >
> > >
> > > The code of cancelapi.obj from csq.lib is really extremly simple, and
> you
> > > can revert it yourslef to C source with minimum effort,
> > > providing you understand x86 assembely.
> > >
> > >
> > >
> > >
> > >
> > > > >
> > > > > –
> > > > > Bill McKenzie
> > > > >
> > > > >
> > > > >
> > > > > “Bill McKenzie” wrote in message
> > > > > news:xxxxx@ntdev…
> > > > > >
> > > > > > It would be perfectly reasonable for you to do this, further,
you
> > > don’t
> > > > > ever
> > > > > > have to put this IRP in the queue to begin with if it doesn’t
need
> > to
> > > be
> > > > > > queued. It is quite common for a driver to queue a specific
type
> of
> > > IRP
> > > > > > say, IRP_MJ_READ, and not queue others, like IRP_MJ_WRITE,
> > > > > > IRP_MJ_DEVICE_CONTROL, etc. Queuing is a serialization
> methodology,
> > > but
> > > > > if
> > > > > > your IRPs don’t need to be serialized, don’t use a queue for
them.
> > > > > >
> > > > > > Further, I would NEVER use system queuing anymore. The system
> queue
> > > > uses
> > > > > > and abuses the system wide cancel spinlock. This spinlock is a
> > single
> > > > > > global resource used by every driver in the system. Over use of
> > this
> > > > lock
> > > > > > stomps system performance. So, you would be better off using a
> > driver
> > > > > queue
> > > > > > and minimizing the use of this resource. There is a new library
> > > > available
> > > > > > for 2000 and later, (and maybe earlier OSes, I just don’t know),
> > that
> > > is
> > > > > > supposed to help you create a cancel-safe driver IRP queue. It
> > comes
> > > > with
> > > > > > the latest DDKs. I have never used it, because while I was at
> > > BlueWater
> > > > > and
> > > > > > BSquare I used the WinDK toolkit which provided its own
> cancel-safe
> > > IRP
> > > > > > queues. In fact looking at the new library in the DDK I see
that
> > the
> > > > > WinDK
> > > > > > functions and the new DDK library functions are AMAZINGLY
similar.
> > > > There
> > > > > is
> > > > > > a sample in the newer DDKs under \src\general\cancel that shows
> how
> > to
> > > > use
> > > > > > the new DDK library.
> > > > > >
> > > > > > Anyway, I probably told you more than you needed to know here,
but
> I
> > > > hope
> > > > > I
> > > > > > answered your question.
> > > > > >
> > > > > > –
> > > > > > Bill McKenzie
> > > > > >
> > > > > >
> > > > > >
> > > > > > “lists.osr.com” wrote in message
> > > news:xxxxx@ntdev…
> > > > > > >
> > > > > > > Hi Everybody,
> > > > > > >
> > > > > > > In “Windows NT Device Driver Development” by Viscarola
and
> > > Mason,
> > > > > > there
> > > > > > > are a couple of statements that say slightly different things
to
> > me
> > > > (as
> > > > > I
> > > > > > > read it).
> > > > > > >
> > > > > > > In Chapter 14 with regard to System Queueing it says
> “Drivers
> > > that
> > > > > use
> > > > > > > System Queueing can have only one operation per device in
> progress
> > > at
> > > > a
> > > > > > > time.” Right after that there is a note which says “Using
> system
> > > > > Queueing
> > > > > > > implies that only one IRP can be in progress per device at a
> > time.”
> > > > The
> > > > > > > word “implies” says to me “but not neccesarily”. Thus my
> > confusion.
> > > > > > >
> > > > > > > So my question is, if I use System Queueing why can’t I
take
> > the
> > > > IRP
> > > > > > > that is being processed by my StartIo routine, stash the
> pointer
> > > > away,
> > > > > > > remove it from the queue, and then call IoStartNextPacket,
and
> > > > complete
> > > > > > the
> > > > > > > original IRP sometime later when I see fit? What problem’s
> might
> > I
> > > > > > > eventually run into.
> > > > > > >
> > > > > > > My device is a PCI device and the IRP in question
> specifically
> > > > does
> > > > > > not
> > > > > > > touch the hardware (but rather is simply waiting for a
periodic
> > > event
> > > > > > > message to come back from the device). The OS is NT4.0 SP6a.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Joe
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>

Nar,

Wanna take a poll about what developers think you should do with the
lawyers? ;^)

I can speak for many of us by saying THANK YOU and also the others at MS who
are sticking your necks out trying to help those of us in the trenches…

-Evan Hillman

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Nar Ganapathy [MS]
Sent: Wednesday, June 19, 2002 10:29 PM
To: NT Developers Interest List
Subject: [ntdev] Re: System Queueing Question

The source code issue is a good question. We are still wrestling
with giving
out source code with the appropriate legal formalities here at
Microsoft. We
have a lot of technical and other concerns. We brought some of
these issues
at the WINHEC BOF.

“Evan Hillman” wrote in message news:xxxxx@ntdev…
>
> I can speak for many of us by saying THANK YOU and also the others at MS
who
> are sticking your necks out trying to help those of us in the trenches…
>

For those of you who might not be aware: Jake and Nar are two of the most
senior developers working on the Windows O/S. We’re very fortunate to have
them participating here.

Peter
OSR

> No it doesn’t. It just requires that you check to see if the queue
is busy,

IoStartNextPacket cannot provide the mean to do this check without the
actual StartIo call.

and if it is queue a DPC to call IoStartPacket.

If a DPC is used, then yes. But what if IoStartNextPacket is called
from StartIo itself?

Max

> > No it doesn’t. It just requires that you check to see if the queue

is busy,

IoStartNextPacket cannot provide the mean to do this check without the
actual StartIo call.

Huh? Anywhere you are going to call IoStartNextPacket, and you are still in
the same context as the original StartIo, first check the queue status and
if it is busy queue a DPC to call IoStartNextPacket.

> and if it is queue a DPC to call IoStartPacket.

If a DPC is used, then yes. But what if IoStartNextPacket is called
from StartIo itself?

That is my point, don’t do that. If you call IoStartNextPacket from StartIo
blindly, you have opened yourself up for a recursion failure. The beauty of
this is, it is a potential failure that could sit there for years without
problem. Using a DPC to call IoStartNextPacket if the queue is busy could
affect performance, but not much likely. Anyway, it is possible to prevent
StartIo recursion without the new instructions, although I like the new
instructions better.


Bill McKenzie

When does the queue get marked as “not busy”? When the current IRP is
completed?

Thanks,
Joe

Bill McKenzie wrote in message
news:xxxxx@ntdev…
>
> No it doesn’t. It just requires that you check to see if the queue is
busy,
> and if it is queue a DPC to call IoStartPacket.
>
> –
> Bill McKenzie
>
>
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntdev…
> >
> > > Further, I would NEVER use system queuing anymore. The system queue
> > uses
> > > and abuses the system wide cancel spinlock. This spinlock is a
> > single
> > > global resource used by every driver in the system. Over use of
> > this lock
> > > stomps system performance. So, you would be better off using a
> > driver queue
> >
> > Another issue is that IoStartPacket queue is not safe from recursion
> > on OSes prior to XP, and requires a call to IoSetStartIoAttributes to
> > be safe on XP.
> >
> > Max
> >
> >
> >
> >
>
>
>
>