O.S: windows 10 IOT LTSC
My FPGA is creating an MSI interrupt every 20msec.
The application is sending an IOCTL request to the driver.
In the driver, this request is forwarded to internal queue (WdfRequestForwardToIoQueue)
Upon interrupt, interrupt handler DPC pulls the request from the internal queue and calls to : WdfRequestCompleteWithInformation to answer the IOCTL in the application.
This answer contains an interrupt counter.
The above mechanism works OK for some time (e.g 30 minutes).
and then Iâm getting in the application an interrupt miss error. The application missed an interrupt (according to the counter).
There are 2 other processes that work with this driver at the same time.
Each of them also waits for an interrupt. But each application (out of 3) waits for a different interrupt.
Can this cause the problem ?
Is there an alternative to the above mechanism so there will be no interrupt miss ?
Thank you for your reply.
Currently: Application is sending **one ** IOCTL which is blocked till interrupt (or timeout)
You suggest: During application initialization, application will send **N ** IOCTL requests and not wait for reply.
Then, in the forever while loop, application will send **one **request and wait for reply.
Did I understand correctly ?
With all due respect, it IS too much to ask from me.
A proper code review is time consuming, complex, and expensive. For me to take a brief look, and toss some random comments to you based on general impressions gained while eating breakfast, would risk my not giving you the right answers⌠which is worse than giving you no answers at all. I know this from experience.
Perhaps some of our other colleagues here will be able to help better than I can.
I can see one flaw right off the bat. Every WDFREQUEST you get must either be completed or forwarded to another I/O queue. The âdefaultâ branch in your ioctl handler doesnât do that. In most samples, you donât complete the request in each switch case, you just set âstatusâ and âinformationâ and complete the request after the switch. The âdefaultâ case can then just âbreakâ, and the STATUS_INVALID_DEVICE_REQUEST will happen automatically. Your case that forwards to an I/O queue would then âreturnâ instead of âbreakâ.
Youâre also not checking for errors in the WdfRequestRetrieveXxx calls, nor are you checking that the BAR offsets are within range. Thatâs extremely dangerous. You should never trust the user-mode app.
Without seeing source code for SendDeviceIoctl, thereâs little we can do. Nothing in the code you posted actually sends an ioctl or waits for a result.
You still havenât described how you know interrupts are being âmissedâ. I assume youâre getting them all in kernel mode, so is it your user-mode accounting that comes up short?
while (1)
{
WaitForDwellEvent (INFINITE, &Counter);
//Check Counter
WaitForDwellEvent (0, &Temp); //Send IOCTL - Do not wait
}
looking at nothing else in your code, this does not follow any valid pattern.
for overlapped IO, there are two general patterns - top down or bottom up. The âtop downâ pattern is the classic one where the application determines what to do and the kernel does it and tells you when its done. A typical example is writing to a file. The timing of when a certain activity should start is entirely under the control of the UM process.
the âbottom upâ pattern is where the what to do is still controlled by the UM application, but the when to do it is under the control of the KM component. either within itself, or in response to some hardware or network event.
with the top down pattern, IO is initiated from UM when work needs to be done. you do not expect any special queueing on the driver side because the driverâs job is to do the work ânowâ. This loop cannot implement that pattern.
with the bottom up pattern, IO is initiated from UM in advance of when the work is to be done by the driver. in this pattern the driver is expected to queue and pend multiple IOCTLs from UM, and then complete them âlaterâ. This loop also does not implement this pattern, as further IOCTLs submitted from UM to KM should happen in the completion handler and not be in any kind of loop.
but in my shop, while(1) would be a gigantic red flag. While(true) and proper code formatting (a matter of opinion)
reading between the lines, I am almost sure that when you say that events are being âmissedâ, they happen during times where you driver does not have a pending IOCTL to complete because this loop cannot ever sent enough.
The attached Gwbr.cpp.txt contains also the code for SendDeviceIoctl
The WaitForDwellEvent sends an IOCTL to the driver which is handled in Control.c.txt (line 119)
The IOCTL is forwarded to a queue allocated only for this IOCTL code and the request is completed in IsrDps.c.txt (line 210)
In IsrDpc.c.txt, upon getting a âdwellâ interrupt, a request is pulled from this queue (line 200) and completed.
The method for checking a missed interrupt in the application:
Upon âdwellâ interrupt, a counter is incremented. This counter is sent back to the application upon an interrupt.
The reply is built in IsrDpc.c.txt (line 207)
I guess we can infer that you have fixed your problem. we can also infer that your pattern is the âbottom upâ pattern and you have correctly identified that pending multiple IRPs allows your driver to do something useful when these interrupts happen during times when your previous solution did not have any pending IO.
next you need to consider how many pending IRPs you should have. This is a problem that you likely cannot exactly solve, but it depends on how important it is to âneverâ miss an event and what happens when the system is under load and the UM components cannot process completions and send new IRPs for a âlongâ time
An interrupt-driven driver always needs to be able to handle the case where a new interrupt arrives before the previous interrupt has finished processing, especially if interrupts arrive quickly. Are you handling that case? You canât assume a nice linear progression from ISR to DPC to user-mode.
IOW, you are not guaranteed a 1:1 correspondence between calling WdfInterruptQueueDpcForIsr and calls to your DPC. Not to mention, your DPC can run in parallel on multiple CPUs simultaneously.
Can you please offer a mechanism (or a sample code) to handle a case in which new interrupt arrives before previous interrupt finished processing ?
How should I handle a not 1:1 correspondence between calling WdfInterruptQueueDpcForIsr and calls to your DPC ?
How should I handle a not 1:1 correspondence between calling WdfInterruptQueueDpcForIsr and calls to your DPC ?
I get this question almost every time I teach our WDF seminar. And I frankly never know how to answer it. The answer is: âYou write some code. That is, you write whatever code you need to write to handle this situation, in your driver, based on your hardware. I canât tell you what that is.â
This code might be something as simple as keeping a count of unserviced interrupts in your ISR, and decrementing that count to zero in your DPC.
Or, maybe that code entails you saving some state away (say, the latched state of your deviceâs registers) on each ISR invocation⌠and retrieving that saved state in your DPC and processing it. This save/retrieve might be something as simple as non-destructively ORâing register status into copy of one or more of your hardware registers (we call these âshadow registersâ) that you maintain in (for example) your Interrupt context. Or the save/retrieve might be something as complex as storing a bunch of stuff into a data area and then making an entry on a linked-list (with ExInterlockedInsertTailList) for each ISR invocation, and retrieving and processing all those entries (using ExInterlockedRemoveHeadList) once you get to your DPC.
Only you know what you need to do, based on your hardware and driver architecture. The key thing is you cannot assume that your calls to WdfInterruptQueueDpcForIsr and calls to your DpcForIsr will be 1:1⌠you have to understand that these can be N:1⌠and write whatever code you need to write to ensure that your driver âdoes the right thingâ under this constraint.
The interrupt tells you âsomething interesting happenedâ. It should not be giving you any more detail than that. It is up to the driver to determine what happened. If some data finished transferring, then your driver needs to be able to find out how much data there was. If your driver has to assume that each interrupt means 64k bytes transferred, then your design is badly broken.
In addition to all of the coalescing that can happen in software, interrupts can be coalesced in hardware too. Message signaled interrupts have been the norm for many years now and the hardware will queue exactly 1 and discard any duplicates.
the point is the same: no matter how you get to your DPC, n things of interest have happened on your hardware, and you have n things to report back up to UM or wherever. and if you donât have at least n pending IRPs to complete, then you have to decide what to do. Throw the data away or queue it up (to some limit). but as Tim points out, if you canât figure out how to calculate n, or get the details of all n things, you are sure to have data loss before you even worry about how the UM program calls the driver.
certain conditions will inevitably result in data loss. so then it becomes an engineerâs job to determine acceptable thresholds. If you anticipate an interrupt frequency on the order of 1 Hz, than just about any design that works at all will work in nearly every case. But even then, there is still no reason to do it deliberately badly
As I wrote, during init, Iâm sending 10 IOCTL requests with the code: DWELL_INTERRUPT_REQUEST_CODE
I checked that those requests are handled.
For each of them, the message is sent to a queue.
Upon interrupt, a message is read from this queue in DPC using WdfIoQueueRetrieveNextRequest
In the application, after this IOCTL is answered, another IOCTL is sent.
In case of âinterrupt missâ, WdfIoQueueRetrieveNextRequest returnes with 0x8000001A which means: STATUS_NO_MORE_ENTRIES
How can this happen ?
Is it possible that the queue is too small ?