IoQueueWorkItem asserts on checked build

Hi,
I’m trying to use the work item mechanism on *Windows XP*, but have been unsuccessfull so far. I’m using the general idea from this place: http://www.microsoft.com/whdc/driver/tips/workitem.mspx
Here is my code snippet for queueing work item:

IndicateWorkItemContext *iwic = MEMALLOC_S(context->handles.adapter, IndicateWorkItemContext);

if (iwic != NULL)
{
/* setup the context */
iwic->adapter = context->handles.adapter;
iwic->packet = packet;

/* queue packet for indication and check if we need to queue work item */
if (NdisInterlockedInsertTailList(&_wi.queue, &iwic->contexts, &_wi.lock) == NULL)
{
DBGPRINT(DL_LOG8, " queueing work item %p, context %p", iwic->item, iwic);

IoQueueWorkItem(_wi.handle, indicateWorkItemRoutine, DelayedWorkQueue, &_wi);
}

[…]
}

and for work item routine:

static VOID indicateWorkItemRoutine(
__in PDEVICE_OBJECT DeviceObject,
__in_opt PVOID Context
)
{
WorkItem *wi = (WorkItem *)Context;
PLIST_ENTRY e = NdisInterlockedRemoveHeadList(&wi->queue, &wi->lock);

/* walk through the context queue */
for (; e != NULL; e = NdisInterlockedRemoveHeadList(&wi->queue, &wi->lock))
{
/* get work item context */
IndicateWorkItemContext *iwic = CONTAINING_RECORD(e, IndicateWorkItemContext, contexts);

/* indicate */
NdisMIndicateReceivePacket(iwic->adapter, &iwic->packet, 1);

/* release work item context memory */
MEMFREE_S(iwic);
}
}

MS documentation states that by the time the work item is executed it’s out of the work items queue, so it could be queued again. Apparently it’s not so (or at least I think it is not) because the checked build of Windows XP asserts in IoQueueWorkItem:

Assertion failed: WorkItem->List.Flink == NULL
*** Source File: d:\xpsp\base\ntos\ex\worker.c, line 523

So the question is if it’s still in the queue (according to the assert)? Maybe there is a bug in the code above I don’t see or the whole concept behind this code is wrong?
Would it be better to allocate new work item on every request and queue it and free it in the work item routine?
Thank you in advance!
Regards,
Igor

Where are your calls to allocate or, in you case, initialize the
IO_WORKITEM?

Wouldn’t you need an IO_WORKITEM per packet?

Thomas F. Divine
http://www.pcausa.com


From:
Sent: Wednesday, December 01, 2010 8:20 AM
To: “Windows System Software Devs Interest List”
Subject: [ntdev] IoQueueWorkItem asserts on checked build

> Hi,
> I’m trying to use the work item mechanism on Windows XP, but have been
> unsuccessfull so far. I’m using the general idea from this place:
> http://www.microsoft.com/whdc/driver/tips/workitem.mspx
> Here is my code snippet for queueing work item:
>
> IndicateWorkItemContext iwic = MEMALLOC_S(context->handles.adapter,
> IndicateWorkItemContext);
>
> if (iwic != NULL)
> {
> /
setup the context /
> iwic->adapter = context->handles.adapter;
> iwic->packet = packet;
>
> /
queue packet for indication and check if we need to queue work item
> */
> if (NdisInterlockedInsertTailList(&_wi.queue, &iwic->contexts,
> &_wi.lock) == NULL)
> {
> DBGPRINT(DL_LOG8, " queueing work item %p, context %p",
> iwic->item, iwic);
>
> IoQueueWorkItem(_wi.handle, indicateWorkItemRoutine,
> DelayedWorkQueue, &_wi);
> }
>
> […]
> }
>
> and for work item routine:
>
> static VOID indicateWorkItemRoutine(
> in PDEVICE_OBJECT DeviceObject,
>
in_opt PVOID Context
> )
> {
> WorkItem *wi = (WorkItem )Context;
> PLIST_ENTRY e = NdisInterlockedRemoveHeadList(&wi->queue, &wi->lock);
>
> /
walk through the context queue /
> for (; e != NULL; e = NdisInterlockedRemoveHeadList(&wi->queue,
> &wi->lock))
> {
> /
get work item context */
> IndicateWorkItemContext iwic = CONTAINING_RECORD(e,
> IndicateWorkItemContext, contexts);
>
> /
indicate /
> NdisMIndicateReceivePacket(iwic->adapter, &iwic->packet, 1);
>
> /
release work item context memory */
> MEMFREE_S(iwic);
> }
> }
>
> MS documentation states that by the time the work item is executed it’s
> out of the work items queue, so it could be queued again. Apparently it’s
> not so (or at least I think it is not) because the checked build of
> Windows XP asserts in IoQueueWorkItem:
>
> Assertion failed: WorkItem->List.Flink == NULL
> *** Source File: d:\xpsp\base\ntos\ex\worker.c, line 523
>
> So the question is if it’s still in the queue (according to the assert)?
> Maybe there is a bug in the code above I don’t see or the whole concept
> behind this code is wrong?
> Would it be better to allocate new work item on every request and queue it
> and free it in the work item routine?
> Thank you in advance!
> Regards,
> Igor
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

The assertion means you are queueing a work item that is already queued. You are assuming that once you queue the work item that the callbackc will be invoked immediately…not true. You need to track queued state

d

dent from a phpne with no keynoard

-----Original Message-----
From: Thomas F. Divine
Sent: December 01, 2010 7:22 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] IoQueueWorkItem asserts on checked build

Where are your calls to allocate or, in you case, initialize the
IO_WORKITEM?

Wouldn’t you need an IO_WORKITEM per packet?

Thomas F. Divine
http://www.pcausa.com

--------------------------------------------------
From:
Sent: Wednesday, December 01, 2010 8:20 AM
To: “Windows System Software Devs Interest List”
Subject: [ntdev] IoQueueWorkItem asserts on checked build

> Hi,
> I’m trying to use the work item mechanism on Windows XP, but have been
> unsuccessfull so far. I’m using the general idea from this place:
> http://www.microsoft.com/whdc/driver/tips/workitem.mspx
> Here is my code snippet for queueing work item:
>
> IndicateWorkItemContext iwic = MEMALLOC_S(context->handles.adapter,
> IndicateWorkItemContext);
>
> if (iwic != NULL)
> {
> /
setup the context /
> iwic->adapter = context->handles.adapter;
> iwic->packet = packet;
>
> /
queue packet for indication and check if we need to queue work item
> */
> if (NdisInterlockedInsertTailList(&_wi.queue, &iwic->contexts,
> &_wi.lock) == NULL)
> {
> DBGPRINT(DL_LOG8, " queueing work item %p, context %p",
> iwic->item, iwic);
>
> IoQueueWorkItem(_wi.handle, indicateWorkItemRoutine,
> DelayedWorkQueue, &_wi);
> }
>
> […]
> }
>
> and for work item routine:
>
> static VOID indicateWorkItemRoutine(
> in PDEVICE_OBJECT DeviceObject,
>
in_opt PVOID Context
> )
> {
> WorkItem *wi = (WorkItem )Context;
> PLIST_ENTRY e = NdisInterlockedRemoveHeadList(&wi->queue, &wi->lock);
>
> /
walk through the context queue /
> for (; e != NULL; e = NdisInterlockedRemoveHeadList(&wi->queue,
> &wi->lock))
> {
> /
get work item context */
> IndicateWorkItemContext iwic = CONTAINING_RECORD(e,
> IndicateWorkItemContext, contexts);
>
> /
indicate /
> NdisMIndicateReceivePacket(iwic->adapter, &iwic->packet, 1);
>
> /
release work item context memory */
> MEMFREE_S(iwic);
> }
> }
>
> MS documentation states that by the time the work item is executed it’s
> out of the work items queue, so it could be queued again. Apparently it’s
> not so (or at least I think it is not) because the checked build of
> Windows XP asserts in IoQueueWorkItem:
>
> Assertion failed: WorkItem->List.Flink == NULL
> *** Source File: d:\xpsp\base\ntos\ex\worker.c, line 523
>
> So the question is if it’s still in the queue (according to the assert)?
> Maybe there is a bug in the code above I don’t see or the whole concept
> behind this code is wrong?
> Would it be better to allocate new work item on every request and queue it
> and free it in the work item routine?
> Thank you in advance!
> Regards,
> Igor
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Hmmm …

I agree that he is queuing an already queued workitem, but he seems to
be tracking the queued state correctly:

if (NdisInterlockedInsertTailList(&_wi.queue, &iwic->contexts,
&_wi.lock) == NULL)

he only queues the work item for the first ndis packet on his
“_wi.queue”, and he drains that same queue in the worker thread
callback. Perhaps he drains the queue elsewhere? Otherwise the test
for queue empty seems to be sufficient.

What am I missing?

Mark Roddy

On Wed, Dec 1, 2010 at 10:25 AM, Doron Holan wrote:
> The assertion means you are queueing a work item that is already queued. You are assuming that once you queue the work item that the callbackc will be invoked immediately…not true. You need to track queued state
>
> ?d
>
> dent from a phpne with no keynoard
>
> -----Original Message-----
> From: Thomas F. Divine
> Sent: December 01, 2010 7:22 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] IoQueueWorkItem asserts on checked build
>
>
> Where are your calls to allocate or, in you case, initialize the
> IO_WORKITEM?
>
> Wouldn’t you need an IO_WORKITEM per packet?
>
> Thomas F. Divine
> http://www.pcausa.com
>
> --------------------------------------------------
> From:
> Sent: Wednesday, December 01, 2010 8:20 AM
> To: “Windows System Software Devs Interest List”
> Subject: [ntdev] IoQueueWorkItem asserts on checked build
>
>> Hi,
>> I’m trying to use the work item mechanism on Windows XP, but have been
>> unsuccessfull so far. I’m using the general idea from this place:
>> http://www.microsoft.com/whdc/driver/tips/workitem.mspx
>> Here is my code snippet for queueing work item:
>>
>> IndicateWorkItemContext iwic = MEMALLOC_S(context->handles.adapter,
>> IndicateWorkItemContext);
>>
>> if (iwic != NULL)
>> {
>> ? ?/
setup the context /
>> ? ?iwic->adapter = context->handles.adapter;
>> ? ?iwic->packet = packet;
>>
>> ? ?/
queue packet for indication and check if we need to queue work item
>> */
>> ? ?if (NdisInterlockedInsertTailList(&_wi.queue, &iwic->contexts,
>> &_wi.lock) == NULL)
>> ? ?{
>> ? ? ? ?DBGPRINT(DL_LOG8, " ?queueing work item %p, context %p",
>> iwic->item, iwic);
>>
>> ? ? ? ?IoQueueWorkItem(_wi.handle, indicateWorkItemRoutine,
>> DelayedWorkQueue, &_wi);
>> ? ?}
>>
>> ? ?[…]
>> }
>>
>> and for work item routine:
>>
>> static VOID indicateWorkItemRoutine(
>> ? in ? ? ?PDEVICE_OBJECT DeviceObject,
>> ?
in_opt ?PVOID Context
>> )
>> {
>> ? ?WorkItem *wi = (WorkItem )Context;
>> ? ?PLIST_ENTRY e = NdisInterlockedRemoveHeadList(&wi->queue, &wi->lock);
>>
>> ? ?/
walk through the context queue /
>> ? ?for (; e != NULL; e = NdisInterlockedRemoveHeadList(&wi->queue,
>> &wi->lock))
>> ? ?{
>> ? ? ? ?/
get work item context */
>> ? ? ? ?IndicateWorkItemContext iwic = CONTAINING_RECORD(e,
>> IndicateWorkItemContext, contexts);
>>
>> ? ? ? ?/
indicate /
>> ? ? ? ?NdisMIndicateReceivePacket(iwic->adapter, &iwic->packet, 1);
>>
>> ? ? ? ?/
release work item context memory */
>> ? ? ? ?MEMFREE_S(iwic);
>> ? ?}
>> }
>>
>> MS documentation states that by the time the work item is executed it’s
>> out of the work items queue, so it could be queued again. Apparently it’s
>> not so (or at least I think it is not) because the checked build of
>> Windows XP asserts in IoQueueWorkItem:
>>
>> Assertion failed: WorkItem->List.Flink == NULL
>> ***? Source File: d:\xpsp\base\ntos\ex\worker.c, line 523
>>
>> So the question is if it’s still in the queue (according to the assert)?
>> Maybe there is a bug in the code above I don’t see or the whole concept
>> behind this code is wrong?
>> Would it be better to allocate new work item on every request and queue it
>> and free it in the work item routine?
>> Thank you in advance!
>> Regards,
>> Igor
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

Thomas:
I allocate it just once in DriverEntry. This approach is used in the link I gave. When one has many requests they are being queued for processing by single work item (it may still run when new requests arrive).
Doron:
Yes, this is what I suspected. When I queue work item I first insert new work item context into the queue of contexts so the NdisInterlockedInsertTailList should return NULL (the queue is empty). If the callback has not been called by the time another request arrives the above function will not return NULL so the work item will not be queued. But if the callback is running the documentation says that it’s no longer in the work item queue so I suppose (as well as the example in the link) that it can be queued again. The new IoQueueWorkItem will be called only if the previous one has processed all the items from the context list. Or do I totally get it wrong?

Regards,
Igor

Mark, thank you for the support!
I do not remove items from the “_wi.queue” anywhere else in the code besides indicateWorkItemRoutine.
I had some crazy idea that when the work item is removed from the work item queue just before calling the callback the kernel might actually “forget” to clear links in its LIST_ENTRY (at least in Windows XP)…

Regards,
Igor

>calling the callback the kernel might actually “forget” to clear links in its LIST_ENTRY (at least in

Windows XP)…

What will the later OS say?


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Maxim:
I’m not sure what OS it’s applied to. I’ve found the above quote in MSDN Library -> Windows Development -> WDK -> KMDA -> Design Guide -> Driver Programming Techniques -> Synchronization Techniques -> Kernel Dispatch Objects -> Thread Objects -> System Worker Threads
I presume it’s something general and not directly related to any OS version.