Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


MmGetMdlByteCount greater than TransferBufferLength

matt_sykesmatt_sykes Member - All Emails Posts: 290
URB bulk out transfer coming from USBSTOR, usbccgp.
URB TransferBuffer is NULL.

MDL flags are 3 (mapped to system and locked)
MDL address from get system safe is valid.
MDL byte offset is zero
MDL byte count is 0x1000

URB transfer length is 0x200

Using byte count causes an exception, so clearly it is invalid, yet according to MSDN "MmGetMdlByteCount macro returns the length, in bytes, of the buffer described by the specified MDL"

Anyone else seen this?

Comments

  • Slava_ImameevSlava_Imameev Member Posts: 480
    MDL describes a buffer a part of which is being transferred. Other way would be incorrect. If you have TransferBufferLength use it instead of MDL's buffer size.
  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    By the way there is one MDL in the chain.

    So what you are saying is the byte count in the MDL is wrong.

    Dont you think it more likely that the MDL was created correctly and its the URB transfer length thats wrong?
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,450
    On Feb 2, 2017, at 5:50 AM, [email protected] wrote:
    >
    > By the way there is one MDL in the chain.
    >
    > So what you are saying is the byte count in the MDL is wrong.
    >
    > Dont you think it more likely that the MDL was created correctly and its the URB transfer length thats wrong?

    Why do you think either one of them is "wrong"? The MDL describes a buffer, possibly used for many different things. This USB transaction happens to be copying into a portion of that buffer. Is that really so hard to believe?

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    Yes, I would expect an MDL sent to a device to describe data for that device, and not have a byte count that was irrelevant.


    And if it wasnt I would expect it write large, "IGNORE THE BYTE COUNT", all across the DDK documentation, and it isnt.
  • Slava_ImameevSlava_Imameev Member Posts: 480
    <QUOTE>
    And if it wasnt I would expect it write large, "IGNORE THE BYTE COUNT", all
    across the DDK documentation, and it isnt.
    </QUOTE>

    Actually, it is - "TransferBufferLength Specifies the length, in bytes, of the buffer specified in TransferBuffer or described in TransferBufferMDL."
  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    Oh come on, thats not the same at all.

    One part of the documentaiton saying that TransferBufferLength specifies "the length, in bytes, of the buffer" while another part documents a function that returns the "the length, in bytes, of the buffer" (to quote the MSDN) and to have two different lengths is ridiculous.


    It needs sorting out.
  • Slava_ImameevSlava_Imameev Member Posts: 480
    Okay, use MmGetMdlByteCount and blame the MSDN documentation for BSODs.
  • Chris_AseltineChris_Aseltine Member Posts: 1,228
    Tim Roberts wrote:

    > This USB transaction happens to be copying into a portion of
    > that buffer.

    Just to be clear, it's copying *from* a portion of that buffer, right? The guy said this was an OUT transaction from usbstor?
  • Alex_GrigAlex_Grig Member Posts: 3,238
    I've been through that. This happens when an application buffer is too big for a single transfer.

    Some class drivers (Tape?) make partial MDLs. Some class drivers just adjust TransferBuffer and TransferBufferLength, which you're supposed to pass to GetScatterGatherList. In the end, you're supposed to use these from SRB/URB, not what's in MDL.

    MDL just describes memory. The buffer is described by values in SRB/URB.
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,450
    On Feb 2, 2017, at 9:50 AM, [email protected] wrote:
    >
    > Oh come on, thats not the same at all.
    >
    > One part of the documentaiton saying that TransferBufferLength specifies "the length, in bytes, of the buffer" while another part documents a function that returns the "the length, in bytes, of the buffer" (to quote the MSDN) and to have two different lengths is ridiculous.
    >
    > It needs sorting out.

    It certainly does not. Consider this code:

    char buffer[1024];

    ReadFile( hDevice, &buffer[20], 128, &dwActual, NULL );

    My buffer is 1024 bytes long, but I only want to read 128 bytes into it. This is the EXACT same situation as what you are describing. The MDL describes a buffer, not an I/O transaction. The TransferBufferLength describes a transaction, which may be into or from PART of a buffer.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    @Alex, thanks for confirming that, and suggesting to use the URB value.

    @Tim, sorry, I disagree. Both values are documented as being the buffer length, yet have, in this case, different values.

    Its either bad documentation or a bug in the driver that prepped the URB.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    If ByteOffset is 0 than the buffer is page-aligned and the whole page should be accessible without any page fault if the page is locked.

    So if the page size is 0x1000, you should not have an exception.

    According to WDM.H version 10 (1607):

    //++
    //
    // PVOID
    // MmGetMdlVirtualAddress (
    // _In_ PMDL Mdl
    // )
    //
    // Routine Description:
    //
    // The MmGetMdlVirtualAddress returns the virtual address of the buffer
    // described by the Mdl.
    //
    // Arguments:
    //
    // Mdl - Pointer to an MDL.
    //
    // Return Value:
    //
    // Returns the virtual address of the buffer described by the Mdl
    //
    //--

    #define MmGetMdlVirtualAddress(Mdl) \
    ((PVOID) ((PCHAR) ((Mdl)->StartVa) + (Mdl)->ByteOffset))

    //++
    //
    // PVOID
    // MmGetMdlStartVa (
    // _In_ PMDL Mdl
    // )
    //
    // Routine Description:
    //
    // The MmGetMdlBaseVa returns the virtual address of the buffer
    // described by the Mdl rounded down to the nearest page.
    //
    // Arguments:
    //
    // Mdl - Pointer to an MDL.
    //
    // Return Value:
    //
    // Returns the returns the starting virtual address of the MDL.
    //
    //
    //--

    #define MmGetMdlBaseVa(Mdl) ((Mdl)->StartVa)
  • anton_bassovanton_bassov Member Posts: 5,159
    > Yes, I would expect an MDL sent to a device to describe data for that device,
    > and not have a byte count that was irrelevant.




    ....and I would expect a reasonable Windows driver writer to get all the relevant information about the IO request from its corresponding request descriptor(i.e. IRP/URB/whatever descriptor that may be in use) , rather than from MDL that describes a buffer used for this request. After all, someone who sends IO requests to your device may simply reuse the same buffers from its locally managed custom memory pool for all the requests that don't exceed their sizes instead of going for the whole trouble of buffer allocation and MDL setup upon every single request. I really hope you are not going to tell us that such an approach constitutes a bug simply because Mr.Sykes et al are unable to see such a possibility.....


    Anton Bassov
  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    @Anton And how are you supposed to know which is the correct length? Both values are documented as being the length of the write buffer.

    Of course its possible to use the values are you and Tim suggest, no one is saying it isnt, what I am saying is that there is no documentation to support this.

    Now, I have been writing all kinds of windows drivers for about 20 years now. Not as intensively as some, in Europe the demand isnt as high, so please save the cheap jibes would you, its not deserved and not welcomed and if you cant keep a civil toungue when you reply to posts please dont bother replying to any more.
  • anton_bassovanton_bassov Member Posts: 5,159
    > And how are you supposed to know which is the correct length? Both values are
    >documented as being the length of the write buffer.


    Please note that the length of the buffer and the amount of valid data in it are two different things that may or may not match - the only requirement here is that the latter does not exceed the former.
    Once you happen to be a data recipient on this particular occasion I would suggest looking at the latter.


    > Now, I have been writing all kinds of windows drivers for about 20 years now.

    To be honest, I don't think you are making yourself a compliment by saying this - after all, something that one may not necessarily understand after having written drivers for, say, six month should become pretty obvious after having done so for 20 years. Never mind.....


    > .....so please save the cheap jibes would you, its not deserved and not welcomed
    > and if you cant keep a civil toungue when you reply to posts please dont bother replying
    > to any more.


    Actually,on this particular occasion I don't seem (IMHO, of course) to be doing anything particularly wrong. I DO see your point - I am obviously neither the friendliest nor best-behaving character that one may encounter on NTDEV, but in this particular case I am not THAT "uncivil"....


    Anton Bassov
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    There is an inconsistency here or I will need to go back to the CPU manuals and learn again how logical to physical memory address translation is done.

    You said that the system VA was valid so print it to see if it is page-aligned. If it is not the case than ByteOffset should not be 0 and you have an indication that the MDL is not usable.

    By the way what was the exception code ? Did the driver touched an invalid VA (no underlying physical memory) or was it an invalid protection ?

    My guess is that the VA is not page-aligned and with 0x1000 as the length you crossed the page boundary and reached either an invalid VA or a VA with a different protection.
  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    @Anton Of all the drivers I have written I have never lower filtered a USB stick driver, looked at the datam, and seen these two values differ. Is that so surprising in 20 years? I did say there is less occasion in Europe to write drivers, so dont be. How many CPU filters driver have you written, how many WLAN NDIS drivers, custom Protocol drivers, TCPIP upper filters to adjust the window size, USB bridging drivers, the weird and the wonderful and many more of all kinds of stuff to interface PCs with military equipment.

    I expect you have some overlap with that list, can I l;augh at you because you dont know everything I do?

    Of course not, Have some respect please.
  • matt_sykesmatt_sykes Member - All Emails Posts: 290
    @D.T. I am no longer lower filtering the USB stick driver, it isn't a part of the product, I was just looking for some male to male USB cables (bulk pipes) to do siome testing. Turns out the ones we got were USB storage devices, hence my query, but I have moved on to other cables to use for proof of concept and wont debug this issue any more.

    Let it remain then as originally stated, that the documentation on the two buffer lengths, which apparently mean exactly the same thing, needs clarifying in the documentaiton. It must say that the URB transfer buffer length should be used as the MDL buffer size might be larger and indicate a bigger bufer that is only partly used. In accordance with Tim and Antons remarks.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 15 Jun 2020 LIVE ONLINE
Writing WDF Drivers 22 June 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA