MmGetMdlByteCount greater than TransferBufferLength

URB bulk out transfer coming from USBSTOR, usbccgp.
URB TransferBuffer is NULL.

MDL flags are 3 (mapped to system and locked)
MDL address from get system safe is valid.
MDL byte offset is zero
MDL byte count is 0x1000

URB transfer length is 0x200

Using byte count causes an exception, so clearly it is invalid, yet according to MSDN “MmGetMdlByteCount macro returns the length, in bytes, of the buffer described by the specified MDL”

Anyone else seen this?

MDL describes a buffer a part of which is being transferred. Other way would be incorrect. If you have TransferBufferLength use it instead of MDL’s buffer size.

By the way there is one MDL in the chain.

So what you are saying is the byte count in the MDL is wrong.

Dont you think it more likely that the MDL was created correctly and its the URB transfer length thats wrong?

On Feb 2, 2017, at 5:50 AM, xxxxx@hotmail.com wrote:

By the way there is one MDL in the chain.

So what you are saying is the byte count in the MDL is wrong.

Dont you think it more likely that the MDL was created correctly and its the URB transfer length thats wrong?

Why do you think either one of them is “wrong”? The MDL describes a buffer, possibly used for many different things. This USB transaction happens to be copying into a portion of that buffer. Is that really so hard to believe?

Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Yes, I would expect an MDL sent to a device to describe data for that device, and not have a byte count that was irrelevant.

And if it wasnt I would expect it write large, “IGNORE THE BYTE COUNT”, all across the DDK documentation, and it isnt.

Actually, it is - “TransferBufferLength Specifies the length, in bytes, of the buffer specified in TransferBuffer or described in TransferBufferMDL.”

Oh come on, thats not the same at all.

One part of the documentaiton saying that TransferBufferLength specifies “the length, in bytes, of the buffer” while another part documents a function that returns the “the length, in bytes, of the buffer” (to quote the MSDN) and to have two different lengths is ridiculous.

It needs sorting out.

Okay, use MmGetMdlByteCount and blame the MSDN documentation for BSODs.

Tim Roberts wrote:

This USB transaction happens to be copying into a portion of
that buffer.

Just to be clear, it’s copying *from* a portion of that buffer, right? The guy said this was an OUT transaction from usbstor?

I’ve been through that. This happens when an application buffer is too big for a single transfer.

Some class drivers (Tape?) make partial MDLs. Some class drivers just adjust TransferBuffer and TransferBufferLength, which you’re supposed to pass to GetScatterGatherList. In the end, you’re supposed to use these from SRB/URB, not what’s in MDL.

MDL just describes memory. The buffer is described by values in SRB/URB.

On Feb 2, 2017, at 9:50 AM, xxxxx@hotmail.com wrote:

Oh come on, thats not the same at all.

One part of the documentaiton saying that TransferBufferLength specifies “the length, in bytes, of the buffer” while another part documents a function that returns the “the length, in bytes, of the buffer” (to quote the MSDN) and to have two different lengths is ridiculous.

It needs sorting out.

It certainly does not. Consider this code:

char buffer[1024];

ReadFile( hDevice, &buffer[20], 128, &dwActual, NULL );

My buffer is 1024 bytes long, but I only want to read 128 bytes into it. This is the EXACT same situation as what you are describing. The MDL describes a buffer, not an I/O transaction. The TransferBufferLength describes a transaction, which may be into or from PART of a buffer.

Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

@Alex, thanks for confirming that, and suggesting to use the URB value.

@Tim, sorry, I disagree. Both values are documented as being the buffer length, yet have, in this case, different values.

Its either bad documentation or a bug in the driver that prepped the URB.

If ByteOffset is 0 than the buffer is page-aligned and the whole page should be accessible without any page fault if the page is locked.

So if the page size is 0x1000, you should not have an exception.

According to WDM.H version 10 (1607):

//++
//
// PVOID
// MmGetMdlVirtualAddress (
// In PMDL Mdl
// )
//
// Routine Description:
//
// The MmGetMdlVirtualAddress returns the virtual address of the buffer
// described by the Mdl.
//
// Arguments:
//
// Mdl - Pointer to an MDL.
//
// Return Value:
//
// Returns the virtual address of the buffer described by the Mdl
//
//–

#define MmGetMdlVirtualAddress(Mdl) \
((PVOID) ((PCHAR) ((Mdl)->StartVa) + (Mdl)->ByteOffset))

//++
//
// PVOID
// MmGetMdlStartVa (
// In PMDL Mdl
// )
//
// Routine Description:
//
// The MmGetMdlBaseVa returns the virtual address of the buffer
// described by the Mdl rounded down to the nearest page.
//
// Arguments:
//
// Mdl - Pointer to an MDL.
//
// Return Value:
//
// Returns the returns the starting virtual address of the MDL.
//
//
//–

#define MmGetMdlBaseVa(Mdl) ((Mdl)->StartVa)

> Yes, I would expect an MDL sent to a device to describe data for that device,

and not have a byte count that was irrelevant.

…and I would expect a reasonable Windows driver writer to get all the relevant information about the IO request from its corresponding request descriptor(i.e. IRP/URB/whatever descriptor that may be in use) , rather than from MDL that describes a buffer used for this request. After all, someone who sends IO requests to your device may simply reuse the same buffers from its locally managed custom memory pool for all the requests that don’t exceed their sizes instead of going for the whole trouble of buffer allocation and MDL setup upon every single request. I really hope you are not going to tell us that such an approach constitutes a bug simply because Mr.Sykes et al are unable to see such a possibility…

Anton Bassov

@Anton And how are you supposed to know which is the correct length? Both values are documented as being the length of the write buffer.

Of course its possible to use the values are you and Tim suggest, no one is saying it isnt, what I am saying is that there is no documentation to support this.

Now, I have been writing all kinds of windows drivers for about 20 years now. Not as intensively as some, in Europe the demand isnt as high, so please save the cheap jibes would you, its not deserved and not welcomed and if you cant keep a civil toungue when you reply to posts please dont bother replying to any more.

> And how are you supposed to know which is the correct length? Both values are

documented as being the length of the write buffer.

Please note that the length of the buffer and the amount of valid data in it are two different things that may or may not match - the only requirement here is that the latter does not exceed the former.
Once you happen to be a data recipient on this particular occasion I would suggest looking at the latter.

Now, I have been writing all kinds of windows drivers for about 20 years now.

To be honest, I don’t think you are making yourself a compliment by saying this - after all, something that one may not necessarily understand after having written drivers for, say, six month should become pretty obvious after having done so for 20 years. Never mind…

…so please save the cheap jibes would you, its not deserved and not welcomed
and if you cant keep a civil toungue when you reply to posts please dont bother replying
to any more.

Actually,on this particular occasion I don’t seem (IMHO, of course) to be doing anything particularly wrong. I DO see your point - I am obviously neither the friendliest nor best-behaving character that one may encounter on NTDEV, but in this particular case I am not THAT “uncivil”…

Anton Bassov

There is an inconsistency here or I will need to go back to the CPU manuals and learn again how logical to physical memory address translation is done.

You said that the system VA was valid so print it to see if it is page-aligned. If it is not the case than ByteOffset should not be 0 and you have an indication that the MDL is not usable.

By the way what was the exception code ? Did the driver touched an invalid VA (no underlying physical memory) or was it an invalid protection ?

My guess is that the VA is not page-aligned and with 0x1000 as the length you crossed the page boundary and reached either an invalid VA or a VA with a different protection.

@Anton Of all the drivers I have written I have never lower filtered a USB stick driver, looked at the datam, and seen these two values differ. Is that so surprising in 20 years? I did say there is less occasion in Europe to write drivers, so dont be. How many CPU filters driver have you written, how many WLAN NDIS drivers, custom Protocol drivers, TCPIP upper filters to adjust the window size, USB bridging drivers, the weird and the wonderful and many more of all kinds of stuff to interface PCs with military equipment.

I expect you have some overlap with that list, can I l;augh at you because you dont know everything I do?

Of course not, Have some respect please.

@D.T. I am no longer lower filtering the USB stick driver, it isn’t a part of the product, I was just looking for some male to male USB cables (bulk pipes) to do siome testing. Turns out the ones we got were USB storage devices, hence my query, but I have moved on to other cables to use for proof of concept and wont debug this issue any more.

Let it remain then as originally stated, that the documentation on the two buffer lengths, which apparently mean exactly the same thing, needs clarifying in the documentaiton. It must say that the URB transfer buffer length should be used as the MDL buffer size might be larger and indicate a bigger bufer that is only partly used. In accordance with Tim and Antons remarks.