xxxxx@wideband-sys.com wrote:
Re: memory alignment… We are currently allocating twice the buffer sizes needed and explicitly aligning the base address the driver will pass the to calling applications. I see Peter’s comment above that the KMDF provides the alignment and so the Windows version is moot. We are building our driver on Win10 using KMDF 1.11; we had previously tried using KMDF 1.9 – which is what we used for our WES7 driver – but neither KMDF version seems to provide the proper alignment.
It’s really hard to believe that. I looked at the code – there just
isn’t any way for it to go wrong. I’d love to see your corrected code
and the corrected debug output that shows this.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Peter,
Actually, we did hire a consultant – who is on this forum – back in 2011 to help with the original XPe development and he saved us a lot of time. We aren’t planning any additional development, just trying to port the existing from XPe (to WES7 which we did a couple of years ago) to Win10.
There seems to be disagreement – or perhaps two different but viable approaches – between you and Alex as to where to call MmMapLockedPagesSpecifyCache.
The interesting thing is the driver code has essentially been unchanged thru these three versions of Windows and we haven’t noticed any glaring issues. Perhaps we’re just lucky or there is a potential for problems. This may explain why once in a blue moon (say at least as rarely as every 100 times) we have misaligned data when our app is cycled WITHOUT power cycling the system. Our system ALWAYS comes up fine off a fresh power cycle.
Jim
Tim,
Thanks for looking into this. I will email that to you early next week. I don’t have that handy now.
Jim
Not “disagreement”… All there presented alternatives are architecturally valid. Which callback to use depends on what you’re specifically trying to accomplish. Look at the docs for the callbacks, see which sounds like it fits your needs best, and go with that one.
Good plan. Perhaps you pay them another few bucks to update the driver now and save yourself the time and trouble?
It is a bit concerning that an experienced kernel dev would code a call to MmMapLockedPages in an EvtIoDeviceControl callback. That’s never been either architecturally correct. OTOH, it definitely works in specific narrowly defined cases…
I am *very* curious to see what Tim finds in terms of the alignment issue. Wouldn’t it be cool if there was some really subtle KMDF bug lurking. It’s not likely but stranger things have happened.
Peter
OSR
@OSRDrivers
Tim Roberts wrote:
xxxxx@wideband-sys.com wrote:
> Answers Needed
> - Is there a limit on the boundaries one can align a common buffer?
> - Is there a way to translate a physical address to a virtual address?
I don’t see how this can fail. I checked the source code.
WdfCommonBufferCreate handles the alignment itself, using the same
mechanism you use. It doesn’t rely on WDM. It gets the alignment from
the WDFDMAENABLER. The WDFDMAENABLER gets its default value from the
DEVICE_OBJECT during WdfDmaEnablerCreate, and
WdfDeviceSetAlignmentRequirement clearly sets the value in the
DEVICE_OBJECT.
After receiving some source code from Mr. Brown, I did some deeper
investigation into this today. I now believe this is a bug in KMDF.
A driver calls WdfDeviceSetAlignmentRequirement. That stores a value in
the AlignmentRequirement field in the FDO. The driver then calls
WdfDmaEnablerCreate, specifying an alignment requirement in the
WDF_DMA_ENABLER_CONFIG structure, which stores it in the WDM DMA_ENABLER
object.
In the KMDF source code, we get to FxDmaEnabler::Initialize. For a
bus-master device, we call FxDmaEnabler::ConfigureBusMasterAdapters.
That computes the larger of the two alignment values and stores the
maximum in m_CommonBufferAlignment, which gets used when creating a
common buffer. The PROBLEM is that this computation is only done if the
DMA type is set to scatter/gather. If the DMA type is not
scatter/gather, then m_CommonBufferAlignment is left at 0, so no
alignment gets done, as Mr. Brown discovered.
The easy workaround is for him to use WdfCommonBufferCreateWithConfig,
where we can pass the specific alignment value for this common buffer,
but I would argue that it is a bug for KMDF not to set
m_CommonBufferAlignment in the non-scatter/gather case. There’s
certainly nothing in the documentation that implies such a connection.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Mr. Roberts… You deserve some sort of award for your dedicated research into this issue and assistance of our mutual colleague. Let me be the first to publicly say Thank You. Your actions aren’t surprising, given how generous and patient you are with some of our posters here.
OK… on to the analysis.
Let me start by saying that I am as surprised by what you discovered as you are. I would not have casually expected that I need to use a scatter/gather profile for my alignment requirement to be enforced.
By way of background: Our members will recall that the way “System S/G” works is that if your device does not support S/G, the system will allocate a big physically contiguous intermediate buffer, do the transfer for you to tha contiguous buffer, and then copy the data back to the original non-contiguous, buffer you specify. Sooooo… I suspect the Framework dev’s thinking might have been “if the profile isn’t a s/g type, then the DMA will be intermediately buffered. Thus, the driver’s buffer need not be aligned.” Fair enough, right?
But I think the Framework dev either outsmarted himself here, or else one piece of code was written without knowledge of how the other piece works. The KEY here is that common buffers are always logically contiguous… and are thus prime targets for use by drivers that do not have a s/g profile.
So the common buffer alignment ALWAYS has to be honored, regardless of the profile selected.
So I vote “Bug”… even if there’s a subtle reason this implementation is “as designed” (and I would VERY much like to hear the story), I think the chance of causing pain to a driver dev pretty handily outweighs the value of the optimization as I currently understand it.
Again, outstanding work Mr Roberts! Bravo! And many thanks… your work benefits the entire community… and you have just demonstrated the value of having the WDF source code made public in a very real way.
Peter
OSR
@OSRDrivers
1 Like