IO Queue Stops Processing Requests

I have a basic GPIO driver that doesn’t do much more than process IO Control requests. Almost all of the IO requests process immediately and return a completion status before the next request is sent. I have one request that requires an unknown amount of time to complete. I move this request to a second manual queue for completion at a later time. My default queue is configured for parallel operation as seen in the code below. The problem I have is that my default queue stops processing requests until the request from the manual queue is completed. Maybe I am wrong but shouldn’t the default queue continue to function even if the manual queue holds an uncompleted request? Thanks in advance for any insights.

Queue configs in my code:

// configure default queue of incoming requests
WDF_IO_QUEUE_CONFIG_INIT_DEFAULT_QUEUE(&ioQueueConfig,
WdfIoQueueDispatchParallel);

// maximum number of simultaneous requests
ioQueueConfig.Settings.Parallel.NumberOfPresentedRequests = 2;

// io event processing callbacks
ioQueueConfig.EvtIoDeviceControl = uio48IoDeviceControl;

// no power managment
ioQueueConfig.PowerManaged = WdfFalse;

status = WdfIoQueueCreate(uio48Device,
&ioQueueConfig,
WDF_NO_OBJECT_ATTRIBUTES,
NULL);

if (!NT_SUCCESS(status)) {
KdPrint((FUNCTION" : uio48Queue failed %!STATUS!\n"));
return status;
}

// configure manual queue for IOCTL_WAIT_INT
WDF_IO_QUEUE_CONFIG_INIT(&irqQueueConfig,
WdfIoQueueDispatchManual);

irqQueueConfig.EvtIoCanceledOnQueue = uio48IoCanceledOnQueue;

status = WdfIoQueueCreate(uio48Device,
&irqQueueConfig,
WDF_NO_OBJECT_ATTRIBUTES,
&uio48DevContext->IrqQueue);

if (!NT_SUCCESS(status)) {
KdPrint((FUNCTION" : uio48IrqQueue failed %!STATUS!\n"));
return status;
}

xxxxx@winsystems.com wrote:

I have a basic GPIO driver that doesn’t do much more than process IO Control requests. Almost all of the IO requests process immediately and return a completion status before the next request is sent. I have one request that requires an unknown amount of time to complete. I move this request to a second manual queue for completion at a later time. My default queue is configured for parallel operation as seen in the code below. The problem I have is that my default queue stops processing requests until the request from the manual queue is completed. Maybe I am wrong but shouldn’t the default queue continue to function even if the manual queue holds an uncompleted request?

Yes. In fact, even if you had specified serial dispatching, as soon as
you put the request in another queue, it’s no longer considered part of
the first queue, and it would dispatch the next request.

MY guess is that your application is not using FILE_FLAG_OVERLAPPED. In
that case, the I/O system will only allow you to submit one request at a
time on a file handle. You either need a second file handle, or you
need to use overlapped I/O.

// maximum number of simultaneous requests
ioQueueConfig.Settings.Parallel.NumberOfPresentedRequests = 2;

Why are you doing that? Was that just an attempt to solve this
problem? In general, this is never a good idea. Either your driver can
handle an arbitrary number of requests, or it can only handle one at a time.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Thanks Tim! That was the magic bullet. Queues are now functioning as expected.

What flags would be set to allow multiple file handles for a driver?

Concerning the ‘NumberOfPresentedRequests = 2’ setting, that was added as a possible fix. I have returned to sequential operation.

xxxxx@winsystems.com wrote:

Thanks Tim! That was the magic bullet. Queues are now functioning as expected.

What flags would be set to allow multiple file handles for a driver?

That’s the default situation. Assuming you have not called
WdfDeviceInitSetExclusive to limit the driver to one client, no flags
are required. You just call CreateFile again in the application. I
have used that trick on occasion when my usage was mostly sequential,
but I needed to have one request submitted asynchronously. That way, I
didn’t have to change all of the app code to use overlapped I/O.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

The queues are operating as expected. The delayed request is on the manual queue and the default queue once again processes requests. Problem is that the framework is calling my CanceledOnQueue function before it starts the next request. I have not been able to find any reason why the request is being cancelled. What is the criteria for cancellation? How can I stop this?

Ignore Message 5. I was given wrong information. What is actually happening:

The request that requires a delay before completion is sent to the manual queue. When the next IO request is sent to the default queue and completed, the request in the manual queue completes and queues another wait request. It appears that a single WdfRequestCompleteWithInformation call is completing two different requests?

Additional information:

Even though the IOCTL call appears to have completed, the request still remains on the manual queue and is cancelled when the application is terminated.

I have the following thread running. At the same time the main application is also accessing the driver via IOCTL calls. I would expect the IOCTL_WAIT_INT function to never exit since I am generating no interrupts which is how the request is completed. But when the main app accesses the driver and completes an IOCTL call, the wait function in my thread is also terminated.

DWORD WINAPI MyThreadFunction(LPVOID lpParam)
{
IRQ_PARAMS irq_params;
BOOL IoCtlResults;
ULONG ReturnedLen;
BYTE irqBit = 0;
int count = 0;

// device 1 only
irq_params.devNum = 1;

while (1)
{
// put THIS process to sleep until interrupt occurs
IoCtlResults = DeviceIoControl(hDevice,
IOCTL_WAIT_INT,
&irq_params,
sizeof(irq_params),
&irq_params,
sizeof(irq_params),
&ReturnedLen,
NULL);

// exit loop if program is terminating
if (exit_flag) {
break;
}

// determine interrupt bit
if (irq_params.irq[0]) {
while (!(irq_params.irq[0] & (1 << count)))
count++;

irqBit = count + 1;
}
else if (irq_params.irq[1]) {
while (!(irq_params.irq[1] & (1 << count)))
count++;

irqBit = count + 9;
}
else if (irq_params.irq[2]) {
while (!(irq_params.irq[2] & (1 << count)))
count++;

irqBit = count + 17;
}

if (irqBit)
printf(“Interrupt on bit %d\n”, irqBit);
else
printf(“No Interrupt\n”);

count = 0;
irq_count++;
}

return 0;
}

xxxxx@winsystems.com wrote:

Even though the IOCTL call appears to have completed, the request still remains on the manual queue and is cancelled when the application is terminated.

Then the request was not completed. What makes you think it was? You
certainly can’t tell anything from the code you posted, because you
aren’t checking the error returns.

You have added FILE_FLAG_OVERLAPPED in the CreateFile call now, right?
But did you read any of the documentation on using overlapped I/O? You
should be supplying an OVERLAPPED structure in every I/O call. Your
initial call will return ERROR_IO_PENDING, then you call
WaitForSingleObject or GetOverlappedResult to wait for the request to
complete.

Technically, you’re not allowed to pass a null OVERLAPPED structure when
you have FILE_FLAG_OVERLAPPED. The 21st Century Windows systems do
their best to simulate synchronous access in that case, but it’s not
impossible for me to believe that you might accidentally get an
ERROR_IO_PENDING return here.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

As you can tell, I am fairly new at this. All I know is that the function that should be waiting is completing and resuming execution. I will look into the OVERLAPPED structure and its use.

I created this quick test code and didn’t add the IOCTL checks. My DLL does check the return values and I think I am getting garbage when I run GetLastError - the last value I got was EA38C4A0h.

I might be able to get around this if I could open two handles to the driver as you suggested earlier. My driver is not allowing a second handle. You mentioned that this should be the default behavior but I am doing something to disable it. Any idea what I could be doing?

xxxxx@winsystems.com wrote:

As you can tell, I am fairly new at this. All I know is that the function that should be waiting is completing and resuming execution. I will look into the OVERLAPPED structure and its use.

At the very least, you need to check for failure from DeviceIoControl
and fetch GetLastError when it fails.

I created this quick test code and didn’t add the IOCTL checks. My DLL does check the return values and I think I am getting garbage when I run GetLastError - the last value I got was EA38C4A0h.

I have to think you’re doing something wrong. GetLastError is pretty
danged reliable, and it’s only about 4 lines of code. You are calling
it as a function, right? This mistake might produce the results you
describe:

printf( “%08x\n”, GetLastError );

I might be able to get around this if I could open two handles to the driver as you suggested earlier. My driver is not allowing a second handle. You mentioned that this should be the default behavior but I am doing something to disable it. Any idea what I could be doing?

Is this a KMDF driver? Is it a function driver, or is it a filter for
another driver?


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Thanks much for your help! I was able to get this working with the OVERLAPPED structure.

Once I added a valid OVERLAPPED structure to my DeviceIoControl() function, the error code received is now ERROR_IO_PENDING which I believes is good in this instance. I then added a wait for HasOverlappedIoComplete(). Code is below.

I also added a timeout so the code doesn’t lock up. If I do reach my timeout, will a CancelIo() call cancel the request on the manual queue? I read that the framework is responsible for cancelling any requeued request.

DECLDIR int WaitForInterrupt(unsigned int *irqArray, unsigned int timeout)
{
IRQ_PARAMS irqp;
int status = SUCCESS;
OVERLAPPED lOverlapped = {0};
HANDLE hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);

irqp.devNum = 1;
if (!DeviceIoControl(uio48Handle,
IOCTL_WAIT_INT,
&irqp,
sizeof(irqp),
&irqp,
sizeof(irqp),
&OutBytesReturned,
&lOverlapped))
{
if (GetLastError() == ERROR_IO_PENDING)
{
// wait until overlapped request is complete
// add a timeout to prevent lock up
while (!HasOverlappedIoCompleted(&lOverlapped))
{
// delay 1 ms
Sleep(1);

// if timeout expires, return error
if (!timeout–)
{
CancelIo(uio48Handle);
status = TIMEOUT_ERROR;
break;
}
}
}
else
return status = DRIVER_ERROR;
}

*(irqArray) = (unsigned int) irqp.irq[0] & 0xff;
*(irqArray + 1) = (unsigned int) irqp.irq[1] & 0xff;
#if MAX_INT_PORTS > 2
* (irqArray + 2) = (unsigned int) irqp.irq[2] & 0xff;
#endif

return status;
}

About opening a second handle to the driver, it is a KMDF function driver.

You have to realize that once a request is queued into a manual queue, it is there forever unless your driver dequeue it again and either process it directly or requeue it in another queue (configured as parallel or sequential) so that it is presented to a request handler (EvtIoRead, EvtIoWrite?).

This means that your driver must regain execution by some mean. My guess is that your driver sets this request aside because the hardware cannot process it immediately but if that is the case, how is your driver notified that the device is ready ?

Here you have a nice article treating WDF queues.

http://www.osronline.com/article.cfm?article=587

J. S.