A bit of confusion regarding IRP cancellation

On page 355 of Viscarola/Mason, there is a listing of an IRP cancellation
routine. Included in this listing is logic that removes the
IRP-to-be-cancelled from the IRP queue if it is not the IRP-in-progress.
Fine. But on the next page, there is a listing of an IRP start routine.
In this listing there is logic to check whether the IRP-to-be-started has
been cancelled. This IRP has presumably just been removed from the head of
the queue. Well, if it had already been cancelled, wouldn’t it have also
been removed from the queue and hence not be available to be processed
within the start routine?

Thanks.

Dave


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Nope. The queue is protected by one lock (the list lock) and the IRP’s
status is protected by a different lock (the “cancel spin lock”). Thus, the
I/O Manager may acquire the cancel spin lock, mark the IRP as cancelled and
then call the cancel routine. At the same time (on another processor) the
driver acquires the list lock and removes an IRP from the list and then
releases the list lock. Since the cancel routine (running on that first
processor) is blocked waiting for the spin lock, when it grabs the lock it
finds the IRP has already been removed from the queue. That’s why the
cancel code handles the case where the IRP for which it is looking is not on
the queue.

The key here is two locks, two different data structures and the various
states in which the system could exist between them.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
?
Hope to see you at the next OSR file systems class March 11, 2002 in Boston!

-----Original Message-----
From: David Weld [mailto:xxxxx@predev.com]
Sent: Thursday, February 14, 2002 8:01 PM
To: NT Developers Interest List
Subject: [ntdev] A bit of confusion regarding IRP cancellation

On page 355 of Viscarola/Mason, there is a listing of an IRP cancellation
routine. Included in this listing is logic that removes the
IRP-to-be-cancelled from the IRP queue if it is not the IRP-in-progress.
Fine. But on the next page, there is a listing of an IRP start routine.
In this listing there is logic to check whether the IRP-to-be-started has
been cancelled. This IRP has presumably just been removed from the head of
the queue. Well, if it had already been cancelled, wouldn’t it have also
been removed from the queue and hence not be available to be processed
within the start routine?

Thanks.

Dave


You are currently subscribed to ntdev as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

The Irp->Cancel flag is set by the IoManager and is not serialized with any
irp processing by the driver. The flag simply notes that a request has been
made to cancel the irp, not that the irp has completed the cancel process.
There is a race condition between the IoManager’s invocation of your
driver’s cancel routine and your normal irp queue processing functions, as
the IoManager cannot of course acquire the lock you use for serializing
these activities.

The published viscarola/mason method is not the only way, and probably not
the best way, to process irp cancellation, (which is a complicated enough
task that the xp ddk has new facilities to make it easier to get it right.)
You should search this list and the newsgroups for the various discussions
about the available algorithms. They are layed out in quite a bit of detail,
along with the usual flamage, trolling and amusements.

-----Original Message-----
From: David Weld [mailto:xxxxx@predev.com]
Sent: Thursday, February 14, 2002 8:01 PM
To: NT Developers Interest List
Subject: [ntdev] A bit of confusion regarding IRP cancellation

On page 355 of Viscarola/Mason, there is a listing of an IRP
cancellation
routine. Included in this listing is logic that removes the
IRP-to-be-cancelled from the IRP queue if it is not the
IRP-in-progress.
Fine. But on the next page, there is a listing of an IRP
start routine.
In this listing there is logic to check whether the
IRP-to-be-started has
been cancelled. This IRP has presumably just been removed
from the head of
the queue. Well, if it had already been cancelled, wouldn’t
it have also
been removed from the queue and hence not be available to be
processed
within the start routine?

Thanks.

Dave


You are currently subscribed to ntdev as:
xxxxx@stratus.com To unsubscribe send a blank email to
leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> The published viscarola/mason method is not the only way, and probably not

the best way, to process irp cancellation, (which is a complicated enough

There were also some MS recommendations on it.

Max


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hi all,

Various algorithm for implementing IRP cancellation have been discussed in the related news groups many times.
I would like to publish my favorite implementation. I attached the source code of an IRP queue object implementation. It is quite generic and flexible. I used it in many drivers.
The implementation is based on a posting from Jamie Hanrahan in comp.os.ms-windows.programmer.nt.kernel-mode on 29 Jun 1999.

I hope there will be some comments from the experts…


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de

#########################################################
irpqueue.h
#########################################################
/************************************************************************
*
* Module: irpqueue.h
* Long name: IRP Queue
* Description: definition of IRP Queue object
*
* Runtime Env.: Kernel: WinNT 4.0, WDM
*
* Author(s): Udo Eberhardt, xxxxx@thesycon.de
*
* Company: Thesycon GmbH, Ilmenau, Germany
* www.thesycon.de
*
************************************************************************/

#ifndef IRPQUEUE_H
#define IRPQUEUE_H

//
// IRP Queue Object
//
// Implements a queue of IRPs.
// Queued IRPs are hold in a cancellable state.
//
typedef struct _IRP_QUEUE {

// spinlock that protects the lists
KSPIN_LOCK Lock;

// list of queued IRPs
LIST_ENTRY IrpList;

// list of cancelled IRPs
LIST_ENTRY CancelledIrpList;

} IRP_QUEUE;

//
// Initialize the IRP queue object
//
// MUST be called once before any other function
//
void
IrpQueueInit(
IRP_QUEUE* Queue
);

//
// Add an IRP to the queue
//
// The function calls IoMarkIrpPending with the IRP passed in.
// The functions returns always STATUS_PENDING
// The return status of this function should be the return
// status of the dispatch function that handles the IRP.
//
// The IRP could get cancelled immediately after it was passed to this function.
// Therefore, the IRP MUST NOT be accessed after the call.
//
// Note: The field Irp->Tail.Overlay.DriverContext[3] is used
// by this function for internal purposes.
//
NTSTATUS
IrpQueueAddReq(
IRP_QUEUE* Queue,
IRP* Irp,
BOOLEAN AtFront // TRUE: insert at head, FALSE: insert at tail
);

// shortcut macros
#define IrpQueueAdd(Queue,Irp) IrpQueueAddReq(Queue,Irp,FALSE)
#define IrpQueuePutBack(Queue,Irp) IrpQueueAddReq(Queue,Irp,TRUE)

//
// Remove an IRP from the head of queue
//
// The function returns a pointer to the IRP that was
// unqueued or NULL if the queue is empty.
//
IRP*
IrpQueueRemove(
IRP_QUEUE* Queue
);

//
// Returns TRUE if the IRP queue is empty, FALSE otherwise
//
BOOLEAN
IrpQueueEmpty(
IRP_QUEUE* Queue
);

//
// Cleanup the IRP queue
//
// The function removes and completes (with STATUS_CANCELLED) all IRPs that
// are related to the file object identified by CleanupFileObject.
// If CleanupFileObject is NULL all queued IRPs are removed and completed.
//
void
IrpQueueCleanup(
IRP_QUEUE* Queue,
FILE_OBJECT* CleanupFileObject
);

#endif // IRPQUEUE_H
/*************************** EOF **************************************/
#########################################################

#########################################################
irpqueue.c
#########################################################
/************************************************************************
*
* Module: irpqueue.c
* Long name: IRP Queue
* Description: implementation of IRP Queue object
*
* Runtime Env.: Kernel: WinNT 4.0, WDM
*
* Author(s): Udo Eberhardt, xxxxx@thesycon.de
*
* Company: Thesycon GmbH, Ilmenau, Germany
* www.thesycon.de
*
************************************************************************/

#include <wdm.h>
#include “irpqueue.h”

static
void
IrpQueue_IrpCancelRoutine(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
);

//
// Initialize the IRP queue object
//
void
IrpQueueInit(
IRP_QUEUE* Queue
)
{

KeInitializeSpinLock(&Queue->Lock);
InitializeListHead(&Queue->IrpList);
InitializeListHead(&Queue->CancelledIrpList);

} // IrpQueueInit

//
// Add an IRP to the queue
//
NTSTATUS
IrpQueueAddReq(
IRP_QUEUE* Queue,
IRP* Irp,
BOOLEAN AtFront
)
{
NTSTATUS Status;
KIRQL irql;
void OldCancel;

// irp is pending
IoMarkIrpPending(Irp);

// lock lists
KeAcquireSpinLock(&Queue->Lock,&irql);
// install cancel handler
OldCancel = IoSetCancelRoutine(Irp,IrpQueue_IrpCancelRoutine);
ASSERT(OldCancel==NULL);
// check cancel flag to determine if the IRP is already cancelled
// if so remove our cancel routine
if ( Irp->Cancel && (NULL!=IoSetCancelRoutine(Irp,NULL)) ) {
// IRP has been cancelled and IO manager does not call our cancel routine
Status = STATUS_CANCELLED;
// unlock lists
KeReleaseSpinLock(&Queue->Lock,irql);
// complete IRP
Irp->IoStatus.Status = Status;
Irp->IoStatus.Information = 0;
IoCompleteRequest(Irp,IO_NO_INCREMENT);
} else {
// queue IRP now, it will be removed by the CancelRoutine or UnqueueRoutine
// save back pointer to queue inside IRP, it is needed by the cancel routine
Irp->Tail.Overlay.DriverContext[3] = Queue;
// add IRP to list
if ( AtFront ) {
InsertHeadList(&Queue->IrpList,&Irp->Tail.Overlay.ListEntry);
} else {
InsertTailList(&Queue->IrpList,&Irp->Tail.Overlay.ListEntry);
}
// unlock lists
KeReleaseSpinLock(&Queue->Lock,irql);
}

// irp is pending
Status = STATUS_PENDING;
return Status;

} // IrpQueueAddReq

//
// Remove an IRP from the head of queue
//
IRP

IrpQueueRemove(
IRP_QUEUE* Queue
)
{
IRP* Irp;
LIST_ENTRY* ListEntry;
KIRQL irql;

// lock lists
KeAcquireSpinLock(&Queue->Lock,&irql);
// loop until the list is empty or we got an IRP
for (;:wink: {
if ( IsListEmpty(&Queue->IrpList) ) {
// list is empty
Irp = NULL;
break;
} else {
// remove first IRP from list
ListEntry = RemoveHeadList(&Queue->IrpList);
Irp = CONTAINING_RECORD(ListEntry,IRP,Tail.Overlay.ListEntry);
// remove our cancel routine
if ( NULL==IoSetCancelRoutine(Irp,NULL) ) {
// cancel routine is running or will run soon
// put this irp to the cancelled list, it will be removed
// and completed by the cancel routine
InsertTailList(&Queue->CancelledIrpList,&Irp->Tail.Overlay.ListEntry);
} else {
// cancel routine is not running and will not run
// invalidate our back pointer and return this IRP
Irp->Tail.Overlay.DriverContext[3] = NULL;
break;
}
}
} //for
// unlock lists
KeReleaseSpinLock(&Queue->Lock,irql);

return Irp;

} // IrpQueueRemove

//
// Returns TRUE if the IRP queue is empty, FALSE otherwise
//
BOOLEAN
IrpQueueEmpty(
IRP_QUEUE* Queue
)
{
BOOLEAN empty;
KIRQL irql;

KeAcquireSpinLock(&Queue->Lock,&irql);
empty = IsListEmpty(&Queue->IrpList);
KeReleaseSpinLock(&Queue->Lock,irql);

return empty;

} //IrpQueueEmpty

//
// Cleanup the IRP queue
//
void
IrpQueueCleanup(
IRP_QUEUE* Queue,
FILE_OBJECT* CleanupFileObject
)
{
LIST_ENTRY CleanupList; // temp local list head
IRP* Irp;
LIST_ENTRY* ListHead;
LIST_ENTRY* ThisEntry;
LIST_ENTRY* NextEntry;
KIRQL irql;

InitializeListHead(&CleanupList);

// lock lists
KeAcquireSpinLock(&Queue->Lock,&irql);
// scan list
ListHead = &Queue->IrpList;
ThisEntry = ListHead->Flink;
while ( ThisEntry!=ListHead ) {
NextEntry = ThisEntry->Flink;
Irp = CONTAINING_RECORD(ThisEntry,IRP,Tail.Overlay.ListEntry);
if ( (CleanupFileObject==NULL) ||
(CleanupFileObject==IoGetCurrentIrpStackLocation(Irp)->FileObject) ) {
// remove this IRP from queue
RemoveEntryList(ThisEntry);
// remove our cancel routine
if ( NULL==IoSetCancelRoutine(Irp,NULL) ) {
// cancel routine is running or will run soon
// put this irp to the cancelled list, it will be removed
// and completed by the cancel routine
InsertTailList(&Queue->CancelledIrpList,ThisEntry);
} else {
// cancel routine is not running and will not run
// invalidate our back pointer and put this IRP on the local cleanup list
Irp->Tail.Overlay.DriverContext[3] = NULL;
InsertTailList(&CleanupList,ThisEntry);
}
}
ThisEntry = NextEntry;
} //while
// unlock lists
KeReleaseSpinLock(&Queue->Lock,irql);

// now complete all IRPs from cleanup list with STATUS_CANCELLED
while ( !IsListEmpty(&CleanupList) ) {
ThisEntry = RemoveHeadList(&CleanupList);
Irp = CONTAINING_RECORD(ThisEntry,IRP,Tail.Overlay.ListEntry);
Irp->IoStatus.Status = STATUS_CANCELLED;
Irp->IoStatus.Information = 0;
IoCompleteRequest(Irp,IO_NO_INCREMENT);
}

} // IrpQueueCleanup

//
// Cancel routine that is installed on queued IRPs.
//
static
void
IrpQueue_IrpCancelRoutine(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
)
{
IRP_QUEUE* Queue;
KIRQL irql;

// release the global cancel spinlock that IO manager has acquired
IoReleaseCancelSpinLock(Irp->CancelIrql);

// a pointer to the queue is stored within the IRP while we own the IRP
Queue = Irp->Tail.Overlay.DriverContext[3];
ASSERT(Queue!=NULL);
// invalidate back pointer
Irp->Tail.Overlay.DriverContext[3] = NULL;
// lock lists
KeAcquireSpinLock(&Queue->Lock,&irql);
// remove this IRP from its list, it is either in IrpList or in CancelledIrpList
RemoveEntryList(&Irp->Tail.Overlay.ListEntry);
// unlock lists
KeReleaseSpinLock(&Queue->Lock,irql);

// complete the irp with STATUS_CANCELLED
Irp->IoStatus.Status = STATUS_CANCELLED;
Irp->IoStatus.Information = 0;
IoCompleteRequest(Irp,IO_NO_INCREMENT);

} // IrpQueue_IrpCancelRoutine

/ EOF*********** /
#########################################################


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com</wdm.h>

I only looked at it briefly, but this is the general idea.

The XP ddk contains equivalent functionality, including a library for w2k
support, in its ‘cancel safe irp queues’. Oddly enough, the XP documentation
now outlines the viscarola/mason method in its discussion of a general
cancel algorithm. In fact the XP ddk documentation seems to be of two minds
on the subject of cancel processing, on the one hand promoting the use of
IO_CSQ irp queues in several sections of the ddk, but then discussing manual
methods under the specific topic of procesing irp cancellation, without any
mention of the cancel safe queue alternative. Were I using the xp ddk to
decide on a driver design, I would be a bit confused as to what the
recommended approach is here.

-----Original Message-----
From: Udo Eberhardt [mailto:xxxxx@thesycon.de]
Sent: Monday, February 18, 2002 9:12 AM
To: NT Developers Interest List
Subject: [ntdev] RE: A bit of confusion regarding IRP cancellation

Hi all,

Various algorithm for implementing IRP cancellation have been
discussed in the related news groups many times. I would like
to publish my favorite implementation. I attached the source
code of an IRP queue object implementation. It is quite
generic and flexible. I used it in many drivers. The
implementation is based on a posting from Jamie Hanrahan in
comp.os.ms-windows.programmer.nt.kernel-mode on 29 Jun 1999.

I hope there will be some comments from the experts…


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de

#########################################################
irpqueue.h #########################################################
/*************************************************************
***********
*
* Module: irpqueue.h
* Long name: IRP Queue
* Description: definition of IRP Queue object
*
* Runtime Env.: Kernel: WinNT 4.0, WDM
*
* Author(s): Udo Eberhardt, xxxxx@thesycon.de
*
* Company: Thesycon GmbH, Ilmenau, Germany
* www.thesycon.de
*
**************************************************************
**********/

#ifndef IRPQUEUE_H
#define IRPQUEUE_H

//
// IRP Queue Object
//
// Implements a queue of IRPs.
// Queued IRPs are hold in a cancellable state.
//
typedef struct _IRP_QUEUE {

// spinlock that protects the lists
KSPIN_LOCK Lock;

// list of queued IRPs
LIST_ENTRY IrpList;

// list of cancelled IRPs
LIST_ENTRY CancelledIrpList;

} IRP_QUEUE;

//
// Initialize the IRP queue object
//
// MUST be called once before any other function
//
void
IrpQueueInit(
IRP_QUEUE* Queue
);

//
// Add an IRP to the queue
//
// The function calls IoMarkIrpPending with the IRP passed
in. // The functions returns always STATUS_PENDING
// The return status of this function should be the return
// status of the dispatch function that handles the IRP.
//
// The IRP could get cancelled immediately after it was
passed to this function. // Therefore, the IRP MUST NOT be
accessed after the call. // // Note: The field
Irp->Tail.Overlay.DriverContext[3] is used // by this
function for internal purposes. // NTSTATUS IrpQueueAddReq(
IRP_QUEUE* Queue,
IRP* Irp,
BOOLEAN AtFront // TRUE: insert at head, FALSE: insert at tail
);

// shortcut macros
#define IrpQueueAdd(Queue,Irp) IrpQueueAddReq(Queue,Irp,FALSE)
#define IrpQueuePutBack(Queue,Irp) IrpQueueAddReq(Queue,Irp,TRUE)

//
// Remove an IRP from the head of queue
//
// The function returns a pointer to the IRP that was
// unqueued or NULL if the queue is empty.
//
IRP*
IrpQueueRemove(
IRP_QUEUE* Queue
);

//
// Returns TRUE if the IRP queue is empty, FALSE otherwise
//
BOOLEAN
IrpQueueEmpty(
IRP_QUEUE* Queue
);

//
// Cleanup the IRP queue
//
// The function removes and completes (with STATUS_CANCELLED)
all IRPs that // are related to the file object identified by
CleanupFileObject. // If CleanupFileObject is NULL all queued
IRPs are removed and completed. // void IrpQueueCleanup(
IRP_QUEUE* Queue,
FILE_OBJECT* CleanupFileObject
);

#endif // IRPQUEUE_H
/*************************** EOF
**************************************/
#########################################################

#########################################################
irpqueue.c #########################################################
/*************************************************************
***********
*
* Module: irpqueue.c
* Long name: IRP Queue
* Description: implementation of IRP Queue object
*
* Runtime Env.: Kernel: WinNT 4.0, WDM
*
* Author(s): Udo Eberhardt, xxxxx@thesycon.de
*
* Company: Thesycon GmbH, Ilmenau, Germany
* www.thesycon.de
*
**************************************************************
**********/

#include <wdm.h>
> #include “irpqueue.h”
>
>
> static
> void
> IrpQueue_IrpCancelRoutine(
> IN PDEVICE_OBJECT DeviceObject,
> IN PIRP Irp
> );
>
>
>
> //
> // Initialize the IRP queue object
> //
> void
> IrpQueueInit(
> IRP_QUEUE* Queue
> )
> {
>
> KeInitializeSpinLock(&Queue->Lock);
> InitializeListHead(&Queue->IrpList);
> InitializeListHead(&Queue->CancelledIrpList);
>
> } // IrpQueueInit
>
>
>
> //
> // Add an IRP to the queue
> //
> NTSTATUS
> IrpQueueAddReq(
> IRP_QUEUE* Queue,
> IRP* Irp,
> BOOLEAN AtFront
> )
> {
> NTSTATUS Status;
> KIRQL irql;
> void OldCancel;
>
> // irp is pending
> IoMarkIrpPending(Irp);
>
> // lock lists
> KeAcquireSpinLock(&Queue->Lock,&irql);
> // install cancel handler
> OldCancel = IoSetCancelRoutine(Irp,IrpQueue_IrpCancelRoutine);
> ASSERT(OldCancel==NULL);
> // check cancel flag to determine if the IRP is already cancelled
> // if so remove our cancel routine
> if ( Irp->Cancel && (NULL!=IoSetCancelRoutine(Irp,NULL)) ) {
> // IRP has been cancelled and IO manager does not call
> our cancel routine
> Status = STATUS_CANCELLED;
> // unlock lists
> KeReleaseSpinLock(&Queue->Lock,irql);
> // complete IRP
> Irp->IoStatus.Status = Status;
> Irp->IoStatus.Information = 0;
> IoCompleteRequest(Irp,IO_NO_INCREMENT);
> } else {
> // queue IRP now, it will be removed by the CancelRoutine
> or UnqueueRoutine
> // save back pointer to queue inside IRP, it is needed by
> the cancel routine
> Irp->Tail.Overlay.DriverContext[3] = Queue;
> // add IRP to list
> if ( AtFront ) {
> InsertHeadList(&Queue->IrpList,&Irp->Tail.Overlay.ListEntry);
> } else {
> InsertTailList(&Queue->IrpList,&Irp->Tail.Overlay.ListEntry);
> }
> // unlock lists
> KeReleaseSpinLock(&Queue->Lock,irql);
> }
>
> // irp is pending
> Status = STATUS_PENDING;
> return Status;
>
> } // IrpQueueAddReq
>
>
>
> //
> // Remove an IRP from the head of queue
> //
> IRP

> IrpQueueRemove(
> IRP_QUEUE* Queue
> )
> {
> IRP* Irp;
> LIST_ENTRY* ListEntry;
> KIRQL irql;
>
> // lock lists
> KeAcquireSpinLock(&Queue->Lock,&irql);
> // loop until the list is empty or we got an IRP
> for (;:wink: {
> if ( IsListEmpty(&Queue->IrpList) ) {
> // list is empty
> Irp = NULL;
> break;
> } else {
> // remove first IRP from list
> ListEntry = RemoveHeadList(&Queue->IrpList);
> Irp = CONTAINING_RECORD(ListEntry,IRP,Tail.Overlay.ListEntry);
> // remove our cancel routine
> if ( NULL==IoSetCancelRoutine(Irp,NULL) ) {
> // cancel routine is running or will run soon
> // put this irp to the cancelled list, it will be removed
> // and completed by the cancel routine
>
> InsertTailList(&Queue->CancelledIrpList,&Irp->Tail.Overlay.ListEntry);
> } else {
> // cancel routine is not running and will not run
> // invalidate our back pointer and return this IRP
> Irp->Tail.Overlay.DriverContext[3] = NULL;
> break;
> }
> }
> } //for
> // unlock lists
> KeReleaseSpinLock(&Queue->Lock,irql);
>
> return Irp;
>
> } // IrpQueueRemove
>
>
>
> //
> // Returns TRUE if the IRP queue is empty, FALSE otherwise
> //
> BOOLEAN
> IrpQueueEmpty(
> IRP_QUEUE* Queue
> )
> {
> BOOLEAN empty;
> KIRQL irql;
>
> KeAcquireSpinLock(&Queue->Lock,&irql);
> empty = IsListEmpty(&Queue->IrpList);
> KeReleaseSpinLock(&Queue->Lock,irql);
>
> return empty;
>
> } //IrpQueueEmpty
>
>
>
> //
> // Cleanup the IRP queue
> //
> void
> IrpQueueCleanup(
> IRP_QUEUE* Queue,
> FILE_OBJECT* CleanupFileObject
> )
> {
> LIST_ENTRY CleanupList; // temp local list head
> IRP* Irp;
> LIST_ENTRY* ListHead;
> LIST_ENTRY* ThisEntry;
> LIST_ENTRY* NextEntry;
> KIRQL irql;
>
> InitializeListHead(&CleanupList);
>
> // lock lists
> KeAcquireSpinLock(&Queue->Lock,&irql);
> // scan list
> ListHead = &Queue->IrpList;
> ThisEntry = ListHead->Flink;
> while ( ThisEntry!=ListHead ) {
> NextEntry = ThisEntry->Flink;
> Irp = CONTAINING_RECORD(ThisEntry,IRP,Tail.Overlay.ListEntry);
> if ( (CleanupFileObject==NULL) ||
>
> (CleanupFileObject==IoGetCurrentIrpStackLocation(Irp)->FileObject) ) {
> // remove this IRP from queue
> RemoveEntryList(ThisEntry);
> // remove our cancel routine
> if ( NULL==IoSetCancelRoutine(Irp,NULL) ) {
> // cancel routine is running or will run soon
> // put this irp to the cancelled list, it will be removed
> // and completed by the cancel routine
> InsertTailList(&Queue->CancelledIrpList,ThisEntry);
> } else {
> // cancel routine is not running and will not run
> // invalidate our back pointer and put this IRP on
> the local cleanup list
> Irp->Tail.Overlay.DriverContext[3] = NULL;
> InsertTailList(&CleanupList,ThisEntry);
> }
> }
> ThisEntry = NextEntry;
> } //while
> // unlock lists
> KeReleaseSpinLock(&Queue->Lock,irql);
>
> // now complete all IRPs from cleanup list with STATUS_CANCELLED
> while ( !IsListEmpty(&CleanupList) ) {
> ThisEntry = RemoveHeadList(&CleanupList);
> Irp = CONTAINING_RECORD(ThisEntry,IRP,Tail.Overlay.ListEntry);
> Irp->IoStatus.Status = STATUS_CANCELLED;
> Irp->IoStatus.Information = 0;
> IoCompleteRequest(Irp,IO_NO_INCREMENT);
> }
>
> } // IrpQueueCleanup
>
>
>
>
> //
> // Cancel routine that is installed on queued IRPs.
> //
> static
> void
> IrpQueue_IrpCancelRoutine(
> IN PDEVICE_OBJECT DeviceObject,
> IN PIRP Irp
> )
> {
> IRP_QUEUE* Queue;
> KIRQL irql;
>
> // release the global cancel spinlock that IO manager has acquired
> IoReleaseCancelSpinLock(Irp->CancelIrql);
>
> // a pointer to the queue is stored within the IRP while we
> own the IRP
> Queue = Irp->Tail.Overlay.DriverContext[3];
> ASSERT(Queue!=NULL);
> // invalidate back pointer
> Irp->Tail.Overlay.DriverContext[3] = NULL;
> // lock lists
> KeAcquireSpinLock(&Queue->Lock,&irql);
> // remove this IRP from its list, it is either in IrpList
> or in CancelledIrpList
> RemoveEntryList(&Irp->Tail.Overlay.ListEntry);
> // unlock lists
> KeReleaseSpinLock(&Queue->Lock,irql);
>
> // complete the irp with STATUS_CANCELLED
> Irp->IoStatus.Status = STATUS_CANCELLED;
> Irp->IoStatus.Information = 0;
> IoCompleteRequest(Irp,IO_NO_INCREMENT);
>
> } // IrpQueue_IrpCancelRoutine
>
>
> / EOF
>
*********** /
> #########################################################
>
>
> —
> You are currently subscribed to ntdev as:
> xxxxx@stratus.com To unsubscribe send a blank email to
> leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com</wdm.h>

And furthermore…

The XP implementation provides callouts for locking/unlocking and cancelled
irp completion, which generalizes the facility a bit, and seems like a good
idea. (I would of course write this as a c++ class and virtualize these
methods with default implementations :slight_smile: Also it was only clear after
reading the implementation that your cleanup function uses a null fileobject
as a flag to cleanup all irps unconditionally.


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Thanks for reviewing the code. Your comments are appreciated.

Sure, it could be more generalized. My original intention was to keep it small and simple. A C++ class would provide a higher level of flexibility in an elegant way, I agree. But one would be forced to develop the whole driver in C++.

Of course, one could use the objects provided by the XP DDK. But this would work on XP only, and 2000 eventually. But what about the other (older) systems? I prefer to use my own implementations, so I can support NT4, 98, ME, 2000, XP, whatever. Note that many of our customers are still requesting support of the older systems.

I agree that the DDK documentation is a bit confusing regarding IRP cancellation issues, especially for newcomers.


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
Sent: Monday, February 18, 2002 4:02 PM
To: NT Developers Interest List
Subject: [ntdev] RE: A bit of confusion regarding IRP cancellation

And furthermore…

The XP implementation provides callouts for locking/unlocking and
cancelled
irp completion, which generalizes the facility a bit, and seems
like a good
idea. (I would of course write this as a c++ class and virtualize these
methods with default implementations :slight_smile: Also it was only clear after
reading the implementation that your cleanup function uses a null
fileobject
as a flag to cleanup all irps unconditionally.


You are currently subscribed to ntdev as: xxxxx@thesycon.de
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

>Of course, one could use the objects provided by the XP DDK. But this would work on XP

only, and 2000 eventually. But what about the other (older) systems?

Reverse-engineer their code and port it back to w2k and NT4, like people do with IoCancelFileOpen.

I agree that the DDK documentation is a bit confusing regarding IRP cancellation issues,
especially for newcomers.

There was an MS article on this, I have the HTML somewhere on my disks.

Max


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> >I agree that the DDK documentation is a bit confusing regarding

IRP cancellation issues,
>especially for newcomers.

There was an MS article on this, I have the HTML somewhere on my disks.

Can you please point us to the source, or post it.


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

>Of course, one could use the objects provided by the XP DDK. But this would
work on XP

only, and 2000 eventually. But what about the other (older) systems?

As Mr. Roddy correctly pointed out, the XP functions are provided in a
library for use on Win2K. There’s no reason the same library wouldn’t work
on NT V4, to the best of my knowledge.

The DDK is INCREBILY confusing on cancel. However, everyone seems to
overlook the REAL wisdom in the DDK on this topic: In MANY drivers, it’s not
even necessary to implement a cancel routine. So, the key question to ask
isn’t “How do I support cancel”, it’s “does my driver even NEED to support
cancel”? In most cases, when you can guarantee that any request arriving at
your driver will complete “real soon” (like, within a couple of seconds)
implementing cancel is just asking for trouble.

Udo’s functions look to me to be pretty straight-forward. I *do* wonder if
there isn’t the possibility of a race on the “cancelled IRP list”, since it
appears that this list can be guarded by different spin locks… I WILL say
that the use of the “cancelled IRP list” is a MUCH better solution than
calling InitializeListHead(…) on the IRP to allow the RemoveEntryList to
succeed (which I’ve seen “recommended” at least one place).

Peter
OSR


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> The DDK is INCREBILY confusing on cancel. However, everyone seems to

overlook the REAL wisdom in the DDK on this topic: In MANY
drivers, it’s not
even necessary to implement a cancel routine. So, the key question to ask
isn’t “How do I support cancel”, it’s “does my driver even NEED to support
cancel”? In most cases, when you can guarantee that any request
arriving at
your driver will complete “real soon” (like, within a couple of seconds)
implementing cancel is just asking for trouble.

I know your article in the NT Insider on this topic. I agree, very often it is unnecessary to worry about cancel. But for some kind of functionality it is just required. For example, I used my IRP_QUEUE object many times to implement notifications from a kernel mode driver to a user mode app. The app submits several IRPs that are hold in a pending state…

Udo’s functions look to me to be pretty straight-forward. I *do*
wonder if
there isn’t the possibility of a race on the “cancelled IRP
list”, since it
appears that this list can be guarded by different spin locks…

Both lists, the IrpList and the CancelledIrpList are protected by the same lock. Everytime one of the lists is accessed this lock is held. Or do I miss some point?

I WILL say
that the use of the “cancelled IRP list” is a MUCH better solution than
calling InitializeListHead(…) on the IRP to allow the RemoveEntryList to
succeed (which I’ve seen “recommended” at least one place).

Agreed. As I already mentioned, I found the original idea of using a cancelled list in a posting from Jamie Hanrahan.


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

“Udo Eberhardt” wrote in message news:xxxxx@ntdev…

>> Udo’s functions look to me to be pretty straight-forward. I do
>> wonder if
>> there isn’t the possibility of a race on the “cancelled IRP
>> list”, since it
>> appears that this list can be guarded by different spin locks…
>>
> Both lists, the IrpList and the CancelledIrpList are protected by the same
lock. Everytime one of the lists
> accessed this lock is held. Or do I miss some point?
>

Ah, you’re quite right! You have one CancelledIrpList per IRP_QUEUE
structure. Reading quickly, I had assumed there was ONE (global to your
driver) CancelledIrpList.

So, no race…

p


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

“Roddy, Mark” wrote in message news:xxxxx@ntdev…
>
> The Irp->Cancel flag is set by the IoManager and is not serialized with
any
> irp processing by the driver. The flag simply notes that a request has
been
> made to cancel the irp, not that the irp has completed the cancel process.
If you read the DDK carefully, you can prove that Irp->Cancel must be set
to TRUE before the driver’s cancel routine is called.

There are a couple of cases that any correct cancel functionality must
handle.
Irp->Cancel must be used to handle the race where the IRP was cancelled
before
the driver set the cancel routine. I’d also assert that it is not possible
to use the
Interlocked list functions to protect the list and to also correctly handle
cancel from the same list due to this race.

Setting the cancel routine to NULL does not guarantee that the cancel
handler is
not about to be called for the IRP on a different CPU => by the time your
cancel
routine has been called, you may have already completed the IRP. Therefore
you must never try to remove the IRP being cancelled from your list before
validating
that the IRP is on the list (unless you use the cancel spin lock to protect
your list [but please don’t]).

Given all of this, the simplest way to handle cancel is to have the cancel
routine IGNORE
the IRP parameter. Instead, it can release the cancel lock, then search all
of the
driver’s queues for all IRPs that have the Cancel flag set, and cancel it.
To handle the race with the IRP is cancelled as soon as your driver gets it,
in the dispatch routine, get your lock, check Irp->Cancel, add to
queue/set cancel routine only if FALSE.

-DH

> There is a race condition between the IoManager’s invocation of your
> driver’s cancel routine and your normal irp queue processing functions, as
> the IoManager cannot of course acquire the lock you use for serializing
> these activities.
>
> The published viscarola/mason method is not the only way, and probably not
> the best way, to process irp cancellation, (which is a complicated enough
> task that the xp ddk has new facilities to make it easier to get it
right.)
> You should search this list and the newsgroups for the various discussions
> about the available algorithms. They are layed out in quite a bit of
detail,
> along with the usual flamage, trolling and amusements.
>
> > -----Original Message-----
> > From: David Weld [mailto:xxxxx@predev.com]
> > Sent: Thursday, February 14, 2002 8:01 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] A bit of confusion regarding IRP cancellation
> >
> >
> > On page 355 of Viscarola/Mason, there is a listing of an IRP
> > cancellation
> > routine. Included in this listing is logic that removes the
> > IRP-to-be-cancelled from the IRP queue if it is not the
> > IRP-in-progress.
> > Fine. But on the next page, there is a listing of an IRP
> > start routine.
> > In this listing there is logic to check whether the
> > IRP-to-be-started has
> > been cancelled. This IRP has presumably just been removed
> > from the head of
> > the queue. Well, if it had already been cancelled, wouldn’t
> > it have also
> > been removed from the queue and hence not be available to be
> > processed
> > within the start routine?
> >
> > Thanks.
> >
> > Dave
> >
> > —
> > You are currently subscribed to ntdev as:
> > xxxxx@stratus.com To unsubscribe send a blank email to
> > leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
> >
>
> —
> You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
> To unsubscribe send a blank email to
leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
>

Ignoring the IRP parameter passed to the IRP cancel routine and searching all the driver’s queues for cancelled IRPs is an approach that can cause performance problems if a driver queues a large number of IRPs. Note that cancel does not only gets called for single IRPs. In some situations (for example thread termination in user app) all IRPs queued in a driver will be cancelled by a sequence of cancel routine calls. Then a peak in CPU load will appear when the driver does a linear search in its queues.

This problem was described in an MS article in the Win2000 DDK. The article is titled “Common Driver Reliability Problems” and contains other interesting topics as well.

There are better cancel implementations. There will be no problems with race conditions if the implementation takes care of that.


Udo Eberhardt
Thesycon GmbH, Germany
xxxxx@thesycon.de
www.thesycon.de

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dave Harvey
Sent: Monday, February 25, 2002 3:03 AM
To: NT Developers Interest List
Subject: [ntdev] Re: A bit of confusion regarding IRP cancellation

“Roddy, Mark” wrote in message
> news:xxxxx@ntdev…
> >
> > The Irp->Cancel flag is set by the IoManager and is not serialized with
> any
> > irp processing by the driver. The flag simply notes that a request has
> been
> > made to cancel the irp, not that the irp has completed the
> cancel process.
> If you read the DDK carefully, you can prove that Irp->Cancel must be set
> to TRUE before the driver’s cancel routine is called.
>
> There are a couple of cases that any correct cancel functionality must
> handle.
> Irp->Cancel must be used to handle the race where the IRP was cancelled
> before
> the driver set the cancel routine. I’d also assert that it is not possible
> to use the
> Interlocked list functions to protect the list and to also
> correctly handle
> cancel from the same list due to this race.
>
> Setting the cancel routine to NULL does not guarantee that the cancel
> handler is
> not about to be called for the IRP on a different CPU => by the time your
> cancel
> routine has been called, you may have already completed the IRP.
> Therefore
> you must never try to remove the IRP being cancelled from your list before
> validating
> that the IRP is on the list (unless you use the cancel spin lock
> to protect
> your list [but please don’t]).
>
>
> Given all of this, the simplest way to handle cancel is to have the cancel
> routine IGNORE
> the IRP parameter. Instead, it can release the cancel lock, then
> search all
> of the
> driver’s queues for all IRPs that have the Cancel flag set, and cancel it.
> To handle the race with the IRP is cancelled as soon as your
> driver gets it,
> in the dispatch routine, get your lock, check Irp->Cancel, add to
> queue/set cancel routine only if FALSE.
>
> -DH
>
> > There is a race condition between the IoManager’s invocation of your
> > driver’s cancel routine and your normal irp queue processing
> functions, as
> > the IoManager cannot of course acquire the lock you use for serializing
> > these activities.
> >
> > The published viscarola/mason method is not the only way, and
> probably not
> > the best way, to process irp cancellation, (which is a
> complicated enough
> > task that the xp ddk has new facilities to make it easier to get it
> right.)
> > You should search this list and the newsgroups for the various
> discussions
> > about the available algorithms. They are layed out in quite a bit of
> detail,
> > along with the usual flamage, trolling and amusements.
> >
> > > -----Original Message-----
> > > From: David Weld [mailto:xxxxx@predev.com]
> > > Sent: Thursday, February 14, 2002 8:01 PM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] A bit of confusion regarding IRP cancellation
> > >
> > >
> > > On page 355 of Viscarola/Mason, there is a listing of an IRP
> > > cancellation
> > > routine. Included in this listing is logic that removes the
> > > IRP-to-be-cancelled from the IRP queue if it is not the
> > > IRP-in-progress.
> > > Fine. But on the next page, there is a listing of an IRP
> > > start routine.
> > > In this listing there is logic to check whether the
> > > IRP-to-be-started has
> > > been cancelled. This IRP has presumably just been removed
> > > from the head of
> > > the queue. Well, if it had already been cancelled, wouldn’t
> > > it have also
> > > been removed from the queue and hence not be available to be
> > > processed
> > > within the start routine?
> > >
> > > Thanks.
> > >
> > > Dave
> > >
> > > —
> > > You are currently subscribed to ntdev as:
> > > xxxxx@stratus.com To unsubscribe send a blank email to
> > > %%email.unsub%%
> > >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@thesycon.de
> > To unsubscribe send a blank email to
> %%email.unsub%%
> >
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@thesycon.de
> To unsubscribe send a blank email to %%email.unsub%%
>
>

Now I more confused. The example in the online DDK (Oct 3, 2001) under Synchronizing IRP Cancellation/
Using a Driver-Supplied Spin Lock seems to be based on the assumption that IoCancelIrp
will atomically set the cancel routine to NULL, and call the cancel handler, but IoCancelIrp
says that will set Irp->Cancel then call the cancel routine, but says nothing about clearing the cancel routine.

-DH

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dave Harvey
Sent: Monday, February 25, 2002 3:03 AM
To: NT Developers Interest List
Subject: [ntdev] Re: A bit of confusion regarding IRP cancellation

“Roddy, Mark” wrote in message
> news:xxxxx@ntdev…
> >
> > The Irp->Cancel flag is set by the IoManager and is not serialized with
> any
> > irp processing by the driver. The flag simply notes that a request has
> been
> > made to cancel the irp, not that the irp has completed the
> cancel process.
> If you read the DDK carefully, you can prove that Irp->Cancel must be set
> to TRUE before the driver’s cancel routine is called.
>
> There are a couple of cases that any correct cancel functionality must
> handle.
> Irp->Cancel must be used to handle the race where the IRP was cancelled
> before
> the driver set the cancel routine. I’d also assert that it is not possible
> to use the
> Interlocked list functions to protect the list and to also
> correctly handle
> cancel from the same list due to this race.
>
> Setting the cancel routine to NULL does not guarantee that the cancel
> handler is
> not about to be called for the IRP on a different CPU => by the time your
> cancel
> routine has been called, you may have already completed the IRP.
> Therefore
> you must never try to remove the IRP being cancelled from your list before
> validating
> that the IRP is on the list (unless you use the cancel spin lock
> to protect
> your list [but please don’t]).
>
>
> Given all of this, the simplest way to handle cancel is to have the cancel
> routine IGNORE
> the IRP parameter. Instead, it can release the cancel lock, then
> search all
> of the
> driver’s queues for all IRPs that have the Cancel flag set, and cancel it.
> To handle the race with the IRP is cancelled as soon as your
> driver gets it,
> in the dispatch routine, get your lock, check Irp->Cancel, add to
> queue/set cancel routine only if FALSE.
>
> -DH
>
> > There is a race condition between the IoManager’s invocation of your
> > driver’s cancel routine and your normal irp queue processing
> functions, as
> > the IoManager cannot of course acquire the lock you use for serializing
> > these activities.
> >
> > The published viscarola/mason method is not the only way, and
> probably not
> > the best way, to process irp cancellation, (which is a
> complicated enough
> > task that the xp ddk has new facilities to make it easier to get it
> right.)
> > You should search this list and the newsgroups for the various
> discussions
> > about the available algorithms. They are layed out in quite a bit of
> detail,
> > along with the usual flamage, trolling and amusements.
> >
> > > -----Original Message-----
> > > From: David Weld [mailto:xxxxx@predev.com]
> > > Sent: Thursday, February 14, 2002 8:01 PM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] A bit of confusion regarding IRP cancellation
> > >
> > >
> > > On page 355 of Viscarola/Mason, there is a listing of an IRP
> > > cancellation
> > > routine. Included in this listing is logic that removes the
> > > IRP-to-be-cancelled from the IRP queue if it is not the
> > > IRP-in-progress.
> > > Fine. But on the next page, there is a listing of an IRP
> > > start routine.
> > > In this listing there is logic to check whether the
> > > IRP-to-be-started has
> > > been cancelled. This IRP has presumably just been removed
> > > from the head of
> > > the queue. Well, if it had already been cancelled, wouldn’t
> > > it have also
> > > been removed from the queue and hence not be available to be
> > > processed
> > > within the start routine?
> > >
> > > Thanks.
> > >
> > > Dave
> > >
> > > —
> > > You are currently subscribed to ntdev as:
> > > xxxxx@stratus.com To unsubscribe send a blank email to
> > > %%email.unsub%%
> > >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@thesycon.de
> > To unsubscribe send a blank email to
> %%email.unsub%%
> >
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@thesycon.de
> To unsubscribe send a blank email to %%email.unsub%%
>
>