calling DeviceIoControl from multiply threads

Hello,

I developed a WDF driver. In order to R/W 32bit word from PCIe, the application is sending IOCTL which is answered by the driver.
The attached code handles 2 cases:

  1. In case Timeout is required (overlapped IO). This is for sending IOCTL that is answered by the driver from ISR DPC upon interrupt.
  2. INFINITE timeout - No event. For R/W of 32bit registers from/to PCIe

I’v noticed that is this code is called from more than one thread in the application, I’m getting an exception after some time.
Why can’t I call DeviceIoControl to the same device from multiple threads ?

Thank you,
Zvika

You certainly can call DeviceIoControl simultaneously from multiple threads. There are three minor issues in your code, although none is likely to lead to problems.

(1) If DeviceIoControl returns non-zero, you’re done. You’re not supposed to wait for the event.

(2) If DeviceIoControl returns zero and the error is ERROR_IO_PENDING, you’re supposed to call GetOverlappedResult to get the completion information. WaitForSingleObject tells you that the request completed, but not what the final error status or the final byte count was. And GetOverlappedResult can do the wait for you.

There are three possible paths for an overlapped request: API returns non-zero – request is finished, no wait required. API returns zero and last error is not ERROR_IO_PENDING – API failed, no wait required. API returns zero and last error is ERROR_IO_PENDING – you must eventually call GetOverlappedResult.

(3) Technically speaking, if you open a file for overlapped processing, EVERY request on that handle must have an OVERLAPPED structure. Your “else” clause there is not valid. In the long past, this caused an error. I believe the kernel now handles it, but that’s not the contract.

Hi Tim,

Thank you very much for your reply.

If my client uses multiple threads, each thread should use a different device handle (and event) and call CreateFile.
Am I right ?
If yes - Is there a way to make sure that each thread is using different device handle ?

Best regards,
Zvika

You do not need a different file handle for each thread. You do need a different event for each request. Typically, you create a pool of OVERLAPPED-derived objects, each with its own event handle and its own context information. As one request finishes, you resubmit it using the same event handle. Your “synchronous simulation” can just reuse the same event over and over, since there will only be one of those at a time.

Hi Tim,

In the driver, the queue that handles IOCTL requests from application is initialized by:
WDF_IO_QUEUE_CONFIG_INIT (&queueConfig, WdfIoQueueDispatchSequential)

Should I use WdfIoQueueDispatchSequential in this case ?

Thank you very much,
Zvika

I would like you read the documentation and answer this question for yourself.

1 Like

To correct Tim slightly, it is not necessary to use different events for multiple IO completions, but it is highly desirable. Otherwise the completion of any, will signal all waiting threads (you must not use an auto reset even in this case) and GetOverlappedResult will report ERROR_IO_PENDING again. In the days of 16 MB of RAM, a desire to limit the number of KM objects led to some designs like this, but in 2020 do not attempt this.

The purpose of OVERLAPPED IO is so that multiple operations from one or more threads can be pending on a single handle simultaneously. The idea of having 1 handle per thread goes against this diametrically. And the 1 thread per operation is inherently inefficient, so I suggest you look at the thread pool APIs for a better UM threading model.

Hi Tim,

According to documentation, in case of multiple threads, I should use: WdfIoQueueDispatchParallel

WdfIoQueueDispatchSequential: The I/O queue’s requests are presented to the driver’s request handlers one at a time. The framework does not deliver the next request until a driver has called WdfRequestComplete to complete the current request
This is not what I need. I have multiple asynchronous threads.

Can you please confirm ?

Thank you,
Zvika

You’re mixing up two concepts.

How many I/O operations can my thread have in progress simultaneously? This is determined by using overlapped I/O and the number of times you call ReadFile or whatever.

The Requests go from you app to a WDF Queue.

The number of Requests your driver can have in progress from a given Queue at one time is determined by the Dispatch type. Sequential dispatching, you get a Request from a Queue and you must “deal with it” (typically, complete it) before you’lll be called by that Ququemwith another Request. Parallel dispatching, you get to set your own limit on the maximum number of in-progress operations from the Queue… by default the limit is infinite.

So…,They are two different things.

You use parallel dispatching if your device can process multiple I/O Requests from that Queue simultaneously. You use sequential dispatching when your device can only process one Request at a time of the type presented by the Queue.

In both cases it can be useful for the user app to have multiple requests pending in the driver at the same time.

Peter

Hi Tim,

I would like to thank you for your helpץ
Your tip to create a pool of events works great.

**Peter **- Thank you very much for the detailed explanation on the dispatch type.

One thread is sending an IOCTL which is completed upon interrupt. When there are no interrupts - WaitForSingleObject in the application blocks forever.

At the same I have a thread which sends ~20 IOCTL requests every 1 sec.
Those requests are answered immidately.

But the queue in the driver is set to sequential.
How does it work ?

Best regards,
Zvika

How does it work ?

Over the past… what… two or more years, it feels to me like we’ve been trying to teach you how to write WDF drivers one forum post at a time. This isn’t a good way to learn how to write drivers.

You really need to (a) take a seminar, (b) spend some dedicated time reading and studying for yourself. Either one will work. There is no substitute for actually understanding the basic concepts of this discipline. You’ve repeatedly asked us to expend effort to help you with various issues, but – and I mean this with all due respect – you don’t seem to be putting the time or effort in yourself to investigate these concepts and improve your knowledge and skills.

Peter

But the queue in the driver is set to sequential. How does it work ?

A sequential queue processes another request as soon as the current request is (a) completed, (b) canceled, or (c) placed in another queue. I assume option “c” is what you’re seeing.

Hi Tim,

Thank you very much !
Best regards,
Zvika