Hi,
I’m writing a UMDF driver, based on the FX2 sample, which supports multiple applications (the design was discussed in the “Distributing data to different queues in UMDF” thread in this forum). In short, several applications send read requests, a thread in the driver keeps sending read requests to the device, and when the driver completes them, it completes the requests sent by the applications, so all applications get all data sent by the device.
My driver works fine on single-core cpus, but when it runs on dual-core cpus, a few bugs appear which I suspect are related to synchronization issues.
I have a thread which sends read requests in a loop to the device.
I don’t want the next request to be sent until I complete the previous request.
So in order to synch between OnCompletion and OnRead I use a shared variable, a HANDLE, which OnCompletion sets, and the thread which sends the requests waits for it to be signaled, resets it and sends the next request.
I’ve protected this variable with the following lock mechanism:
CRITICAL_SECTION m_Crit;
__inline void Lock(){::EnterCriticalSection(&m_Crit);}
__inline void Unlock(){::LeaveCriticalSection(&m_Crit);}
But I think it is not enough for dual core systems.
For instance, I see that sometimes it gets stuck on the WaitForSingleObject in the thread which sends request, and it doesn’t get to OnCompletion, even though the packet has actually arrived (I check it with a USB analyzer). So it might not have been signaled properly. Using TraceView I see that different threads on different CPUs are setting and resetting this variable.
Should I use another method of synchronizing for dual-core cpus? If not, and the method mentioned above is sufficient, how could it be that a message arrived to the host (as logged by a usb analyzer), but OnCompletion wasn’t called? (Only the next packet from the device caused OnCompletion to be called)
Thanks,
Gadi
Most likely your synchronization system is wrong and you don’t see it in
single core CPU because two threads never execute concurrently. You might
have a deadlock caused by out of order resource acquisition for example.
By the way if your variable is just an int you can use interlocked
operations.
–
EA
Hi,
I’m writing a UMDF driver, based on the FX2 sample, which supports
multiple applications (the design was discussed in the “Distributing data
to different queues in UMDF” thread in this forum). In short, several
applications send read requests, a thread in the driver keeps sending read
requests to the device, and when the driver completes them, it completes
the requests sent by the applications, so all applications get all data
sent by the device.
My driver works fine on single-core cpus, but when it runs on dual-core
cpus, a few bugs appear which I suspect are related to synchronization
issues.
I have a thread which sends read requests in a loop to the device.
I don’t want the next request to be sent until I complete the previous
request.
So in order to synch between OnCompletion and OnRead I use a shared
variable, a HANDLE, which OnCompletion sets, and the thread which sends
the requests waits for it to be signaled, resets it and sends the next
request.
I’ve protected this variable with the following lock mechanism:
CRITICAL_SECTION m_Crit;
__inline void Lock(){::EnterCriticalSection(&m_Crit);}
__inline void Unlock(){::LeaveCriticalSection(&m_Crit);}
But I think it is not enough for dual core systems.
For instance, I see that sometimes it gets stuck on the
WaitForSingleObject in the thread which sends request, and it doesn’t get
to OnCompletion, even though the packet has actually arrived (I check it
with a USB analyzer). So it might not have been signaled properly. Using
TraceView I see that different threads on different CPUs are setting and
resetting this variable.
Should I use another method of synchronizing for dual-core cpus? If not,
and the method mentioned above is sufficient, how could it be that a
message arrived to the host (as logged by a usb analyzer), but
OnCompletion wasn’t called? (Only the next packet from the device caused
OnCompletion to be called)
Thanks,
Gadi
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
One caveat- I’m just beginning to understand UMDF internals (although I’ve written KM-> UM redirectors in the past, so I have some idea how it probably works), so perhaps I missed something. But as I see it, this is the picture.
Given the behavior, I assume the handle is for a notification (aka manual reset) event (but it doesn’t matter all that much).
The mechanism you describe (shared event among all threads) doesn’t work- it actually provides none of the protection you wish to provide. You’ve described several of the expected symptoms of its use:
* Any thread can signal the event no matter who is waiting.
* When the event is signaled, ALL threads waiting on it are now free to go (not just the one that thinks it reset it originally).
* You have a gap between sending the request and resetting the event where the thread could be suspended, and while suspended, the I/O completes and the event would be set before you reset it and wait on it. You thus wait for a signal that won’t arrive until you send a read on another thread and it completes.
Using critical sections has no real effect on your problem. If your synchronization scheme was sound, critical sections would be superfluous- the kernel already synchronizes the state of events and threads using them.
If I understand what you want to achieve (each thread waits for one I/O to complete then submits another), then you should use a synchronization (auto-reset), and not a notification (manual reset) event per thread. But synchronous I/O would be easier, and it does exactly what you seem to want to do.
Hi,
Thanks for the quick replies.
Following your post, I tried auto-reset but it didn’t help.
Does UMDF (based on the FX2 skeleton) support synchronous I/O? What changes do I have to do in my design?
Thanks,
Gadi
Gadi-
Well, in my case, I may have replied too quickly [before I fully remembered exactly what you were doing before).
With respect to what I had said, changing the event type is not enough (in fact, it isn’t a necessity [I missed that earlier], but it makes things easier)- you also have to use an event for each thread, not a shared event across all threads. You also have to associate that event with the completion context so that you then signal the proper event when an I/O completes so you wake up the thread that sent it.
That’s how synchronous I/O works [although there a notification event is OK], which is why I suggested it.
When you call IWdfIoRequest::Send, set the WDF_REQUEST_SEND_OPTION_FLAG_SYNCHRONOUS bit in the flags parameter. You also no longer need a completion routine, when you do this, so the resulting code should be much simpler.
*BUT* I’m having some problems reconciling this with what I understood of your approach from your earlier questions. At that time you were taking requests from multiple threads, holding them all while you did an I/O to the device, and then posted the results of that I/O back to all of the original callers. In that case, you don’t always send a request down for each one you receive, so it shouldn’t even be working the way I understood your original question on this thread. Which means what I’ve been talking about above won’t help.
If I were doing that [and I’ve understood what you want- still not sure of that], then I’d have a state machine for the underlying I/O pump whose state is synchronized with a synchronization event. On an incoming I/O I’d acquire it and check the state- if Idle, then I mark my request as the accumulation point for I/Os arriving while I/O is in progress, set the state to I/O in progress, and signal the event. Otherwise, I add my request to a list anchored at said accumulation point and signal the event (no state change). In the first case, I then send a synchronous I/O down [I’d use a suitable timeout on the request]. In the second, I just exit. When my synchronous I/O completes (i.e., the call returns), I again acquire the event, post the data [or failed state, as the case may be] and complete all the accumulated user I/Os, set the state to Idle and release the event. That would pretty much be my queue callback function.
Cancellation of the User I/O might take some further thinking [particularly since UMDF isn’t something I’d call myself an expert in, and I don’t want to take all day thinking about this], but I don’t see it breaking an approach like that.
So if what you’re doing [or trying to do] is something like that and this is where you are having synchronization issues, then I think more detail may be needed. If I’ve got it completely wrong, my apologies, but that’s the best I Can come up with from what I’ve seen and can recall.