Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

MmProbeAndLockPages() reporting STATUS_ACCESS_VIOLATION

omriomri Member Posts: 6

Hey All,
So I've been scratching my head for days trying to figure out this problem with no success. I'm writing a KMDF PCIe driver. I communicate with it though a bunch of custom DeviceIoControl() calls (since my device has quite a unique purpose and interface) and it uses buffered I/O. One of the custom I/O control codes is passed in a C struct (using buffered I/O) that includes a member variable that is a pointer to a user mode buffer that I'd like to make accessible and map into kernel space. This mapping needs to persist for longer than the just duration of this DeviceIoControl() call so I can't use WdfRequestProbeAndLockUserBufferForRead(). I'm trying to call MmProbeAndLockPages() to map this memory and it consistently fails with STATUS_ACCESS_VIOLATION. Here is the relevant code:

mappedMemory->_mdl = IoAllocateMdl(parameters->_virtualAddress, (ULONG)parameters->_size, FALSE, FALSE, NULL);
if(mappedMemory->_mdl == NULL)
{
PDEBUGERROR("IoAllocateMdl() failed in mapMemory()!");
return(STATUS_NONE_MAPPED);
}

__try { MmProbeAndLockPages(mappedMemory->_mdl, UserMode, (parameters->_usage == opFtxMappedMemoryFromDeviceUsage) ? IoWriteAccess : IoReadAccess); }
__except(EXCEPTION_EXECUTE_HANDLER)
{
PDEBUGERROR("MmProbeAndLockPages() failed in mapMemory() with exception code %u!", (unsigned int)GetExceptionCode());
IoFreeMdl(mappedMemory->_mdl);
return(STATUS_NONE_MAPPED);
}

This code is called by an EVT_WDF_IO_IN_CALLER_CONTEXT callback so I should be in the correct process' context. I'm not 100% sure how to check (if anybody knows how, please let me know), but I can't see why I wouldn't be the top of the driver stack and have a filter driver above me ruining my process context guarantee. I have checked that parameters->_virtualAddress and parameters->_size is correct and matches what I have passed in from user space so the input buffer's buffered I/O mechanism is working. The code fails regardless of whether I use it with IoWriteAccess or IoReadAccess. I have tried both KernelMode and UserMode as the parameter to MmProbeAndLockPages(). On the user space side, I have tried this with memory allocated both via VirtualAlloc(NULL, size, MEM_COMMIT, PAGE_READWRITE) and from an array local variable directly on the call stack. I have tried with and without locking the memory from the user space side via VirtualLock(). Nothing seems to make a difference. Does anybody know what could potentially be the problem? what else to check? perhaps another approach to map user space memory into kernel space for extended periods of time outside of the context of a single request/IRP?

Thanks in advance,
Omri

Comments

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,002
    via Email
    omri wrote:
    > I'm writing a KMDF PCIe driver. I communicate with it though a bunch of custom DeviceIoControl() calls (since my device has quite a unique purpose and interface) and it uses buffered I/O. One of the custom I/O control codes is passed in a C struct (using buffered I/O) that includes a member variable that is a pointer to a user mode buffer that I'd like to make accessible and map into kernel space.

    That's a design flaw.  The correct design would be to have one
    METHOD_xx_DIRECT ioctl where you pass the buffer, then keep that buffer
    pending in a manual queue for the life of the need.  That way, the I/O
    manager takes care of the mapping and validation. You can follow up with
    a buffered ioctl to trigger whatever I/O operation needs to use the
    buffer.  When you're done, you complete the request.


    > I'm trying to call MmProbeAndLockPages() to map this memory and it consistently fails with STATUS_ACCESS_VIOLATION. Here is the relevant code:

    Unless you're doing this in an InProcessContext callback, this is
    probably failing because you're no longer in that processes context, so
    the address space belongs to another process.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Pavel_APavel_A Member Posts: 2,674
    edited May 23

    One of the custom I/O control codes is passed in a C struct (using buffered I/O) that includes a member variable that is a pointer to a user mode buffer that I'd like to make accessible and map into kernel space.

    As you likely know, Windows ioctls can pass two buffers. So, make the 1st buffer contain your "C struct" and the 2nd buffer be the other buffer.
    The system will automatically verify and map it for you.
    Let the driver to manage the pointer, instead of passing it in the C struct.
    Pend this ioctl and do not complete until done with the buffer (for example, the app can cancel it when done).
    Bingo, done.

    -- pa

  • omriomri Member Posts: 6

    Hi Tim,
    Thanks, unfortunately this is actually a port of a working Linux driver where the cross platform user space C++ code has the map(), doSomethingWithTheMappedMemory(), unmap() structure that will be quite a pain in to refactor.
    You mentioned InProcessContext callback. What is that? As I understood the documentation that's what EVT_WDF_IO_IN_CALLER_CONTEXT is (providing you are at the top of the driver stack and don't have a filter driver above you).
    One alternative approach I've considered (which won't require major refactoring to existing cross platform code) is I can allocate paged memory in kernel space and map it into user space as opposed to vice versa. That way when the user space asks to map it into the kernel and allow the kernel to do something with it, all the kernel has to do is lock the pages. What I'm unsure about is that there could potentially be up to a gigabyte of these buffers around at any given time and I don't know how much memory I can allocate in the kernel side.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,251
    >I can allocate paged memory in kernel space and map it into user space as opposed to vice versa.

    That’s close to what I did recently in a similar situation (Linux driver that used mmap, etc). But I allocated non-paged memory using MmAllocatePagesForMdl , mapped it into user virtual address space with MmMapLockedPagesSpexifyCache, and returned the pointer to the user app.

    Just don’t forget to do the unmap before the user app is allowed to exit.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • omriomri Member Posts: 6
    edited May 23

    FYI, I just tried calling ProbeForRead() with the exact same buffer address and size right before I call MmProbeAndLockPages() and it succeeds just fine, yet MmProbeAndLockPages() still fails with STATUS_ACCESS_VIOLATION. What could that possibly mean?

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,251

    Either the virtual address or the length must be wrong. You’ve validated that they’re correct, and match what is passed in user mode?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,002
    via Email
    On May 23, 2019, at 3:53 PM, omri wrote:
    >
    > Thanks, unfortunately this is actually a port of a working Linux driver where the cross platform user space C++ code has the map(), doSomethingWithTheMappedMemory(), unmap() structure that will be quite a pain in to refactor.

    map() and unmap() are not cross-platform and never have been. However, it is certainly possible for you to write a map() wrapper that does exactly what I described.


    > You mentioned InProcessContext callback. What is that? As I understood the documentation that's what EVT_WDF_IO_IN_CALLER_CONTEXT is (providing you are at the top of the driver stack and don't have a filter driver above you).

    Yes. Is that what you're using?


    > ...all the kernel has to do is lock the pages. What I'm unsure about is that there could potentially be up to a gigabyte of these buffers around at any given time and I don't know how much memory I can allocate in the kernel side.

    Page pool is plenty large.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • anton_bassovanton_bassov Member Posts: 5,003

    I can allocate paged memory in kernel space and map it into user space as opposed to vice versa.

    This approach is generally frowned upon in the Windows world because of the extra complications that may arise. For example, consider what happens if your app terminates abnormally....

    Anton Bassov

  • omriomri Member Posts: 6

    Peter_Viscarola_(OSR), yes I have verified that both the user space virtual address of the buffer and the size (passed in as members of a struct passed in via the lpInBuffer parameter) are exactly the same once they reach my kernel side driver as they were when I passed them in via DeviceIoControl().

    Tim_Roberts, sorry when I said map() and unmap(), I didn't mean platform specific functions. I meant that is how the existing logic flow of the user space cross platform C++ code for my project interacts with the kernel side driver. First, you pass in a user space buffer address and size to the kernel as either an input buffer or output buffer for future operations (the map() step), the kernel returns an ID to reference this buffer with from now on. Than there are multiple operations you can do with it, passing the ID to reference which buffer of data you are using as input or output. Once you are done, you pass the buffer ID to an IO control code telling it to unmap the buffer from kernel space and that the buffer ID is no longer valid and may later be reused to reference another buffer (the unmap() step).
    Tim_Roberts, yes also I am calling MmProbeAndLockPages() from an EVT_WDF_IO_IN_CALLER_CONTEXT callback. Still very odd why it doesn't succeed especially when ProbeForRead() works fine.

  • Pavel_APavel_A Member Posts: 2,674

    Still very odd why it doesn't succeed especially when ProbeForRead() works fine.

    There is someone's bug there: ether yours or Microsoft's. Guess which.
    Mr. Robert's advice lets you skip all these mappings - the system will do everything. Even tell you if the user process dies unexpectedly (by canceling the ioctl). And you don't need to pass any pointer to the buffer in the first struct.
    My advice was just same, slightly rephrased.

    -- pa

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,251

    It should go without saying that Mr. Roberts’ and Mr. Pavel_A’s advice represent the primary recommended design pattern for what you want to do. In the same driver that I pinned the buffers, as described earlier, I also used the design pattern described by Mr. Pavel_A.

    Sure you can’t change your IOCTL implementation?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • anton_bassovanton_bassov Member Posts: 5,003

    ... I've been scratching my head for days trying to figure out this problem with no success.
    .....
    I'm trying to call MmProbeAndLockPages() to map this memory and it consistently fails with STATUS_ACCESS_VIOLATION.

    You seem to assume that the userland-provided parameters are correct, so that you don't even question their validity. However, there is a good chance that this assumption is just a way too bold. If you take the approach that Mr.Roberts suggests (I think our hosts call it " a big honking IRP" and promote it as a new "good coding practice" in their classes) , this problem will be gone

    Anton Bassov

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,251
    edited May 26
    Yup.

    The “big Honkin’ Hangin’ Request” design pattern.

    We even teach it in our Advanced WDF seminar.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Developing Minifilters 29 July 2019 OSR Seminar Space
Writing WDF Drivers 23 Sept 2019 OSR Seminar Space
Kernel Debugging 21 Oct 2019 OSR Seminar Space
Internals & Software Drivers 18 Nov 2019 Dulles, VA