Replacing WDFMEMORY and/or buffer in write callback

NTDEV
1

A (long) while back, in this thread:

https://www.osronline.com/ShowThread.cfm?link=216562

… Doron alluded to being able to replace a WDFREQUEST’s WDFMEMORY ala:

“Then all you need to do is allocate a WDFMEMORY with the new expanded size, for
a write operation copy the caller’s buffer content into the new buffer, format
the request with your WDFMEMORY and adjusted size, and send the request down the
stack.”

I needed to do something similar, so I tried to follow that approach. When I reformat the request using my new WDFMEMORY, it returns STATUS_SUCCESS, but turning around and obtaining the request’s WDFMEMORY again returns the original WDFMEMORY, as if nothing was changed. There are no errors returned, and the wdf logs from windbg only show power operations (eg, no “read only” buffers or other errors).

Here’s what I’m doing (some irrelevant and error handling code removed for brevity):

//
// Write path function
//

WDF_REQUEST_PARAMETERS params;
WDFMEMORY origmem, newmem, newmem2;
NTSTATUS status;
PVOID origbuf, newbuf, newbuf2;

WDF_REQUEST_PARAMETERS_INIT(&params);
WdfRequestGetParameters(Request, &params);

status = WdfRequestRetrieveInputMemory(Request, &origmem);

if (NT_SUCCESS(status)) {
origbuf = WdfMemoryGetBuffer(origmem, NULL);
} else {
// handle failure (code omitted)
}

status = WdfMemoryCreate(WDF_NO_OBJECT_ATTRIBUTES, NonPagedPool, ‘MyPl’, params.Parameters.Write.Length, &newmem, &newbuf);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

//
// buffer modification code here (copy and modify from origbuf -> newbuf) omitted.
// this code works fine, in the debugger I can see both the original and modified data
// correctly, in origbuf and newbuf, respectively.
//

status = WdfIoTargetFormatRequestForWrite(WdfDeviceGetIoTarget(device), Request, newmem, NULL, NULL);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

status = WdfRequestRetrieveInputMemory(Request, &newmem2);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

newbuf2 = WdfMemoryGetBuffer(newmem2, NULL);

At this point, even though I think I’ve replaced “Request”'s WDFMEMORY with “newmem”, newmem2 still is == to origmem and newbuf2 == origbuf. None of the failure paths are being hit.

This makes it difficult to modify the write buffer before sending the request down.

I’ve temporarily worked around this by rolling a new WDFREQUEST and WDFMEMORY, copying all the parameters and info from the original request into the new one, and sending the new request down. But that seems wasteful and is quite a bit more code.

This is using VS2015 and a WS2016 TP4 KMDF target.

Cluestick? My guess is it’s related to the use of WdfIoTargetFormatRequestForWrite, but I re-read the docs a few more times and this seems like the correct function to use here.

Thanks.

-ml

2

On Mon, Feb 08, 2016 at 02:40:16PM -0500, xxxxx@azathoth.net wrote:

A (long) while back, in this thread:

https://www.osronline.com/ShowThread.cfm?link=216562

… Doron alluded to being able to replace a WDFREQUEST’s WDFMEMORY ala:

“Then all you need to do is allocate a WDFMEMORY with the new expanded size, for
a write operation copy the caller’s buffer content into the new buffer, format
the request with your WDFMEMORY and adjusted size, and send the request down the
stack.”

I needed to do something similar, so I tried to follow that approach. When I reformat the request using my new WDFMEMORY, it returns STATUS_SUCCESS, but turning around and obtaining the request’s WDFMEMORY again returns the original WDFMEMORY, as if nothing was changed. There are no errors returned, and the wdf logs from windbg only show power operations (eg, no “read only” buffers or other errors).

Here’s what I’m doing (some irrelevant and error handling code removed for brevity):

//
// Write path function
//

WDF_REQUEST_PARAMETERS params;
WDFMEMORY origmem, newmem, newmem2;
NTSTATUS status;
PVOID origbuf, newbuf, newbuf2;

WDF_REQUEST_PARAMETERS_INIT(&params);
WdfRequestGetParameters(Request, &params);

status = WdfRequestRetrieveInputMemory(Request, &origmem);

if (NT_SUCCESS(status)) {
origbuf = WdfMemoryGetBuffer(origmem, NULL);
} else {
// handle failure (code omitted)
}

status = WdfMemoryCreate(WDF_NO_OBJECT_ATTRIBUTES, NonPagedPool, ‘MyPl’, params.Parameters.Write.Length, &newmem, &newbuf);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

//
// buffer modification code here (copy and modify from origbuf -> newbuf) omitted.
// this code works fine, in the debugger I can see both the original and modified data
// correctly, in origbuf and newbuf, respectively.
//

status = WdfIoTargetFormatRequestForWrite(WdfDeviceGetIoTarget(device), Request, newmem, NULL, NULL);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

status = WdfRequestRetrieveInputMemory(Request, &newmem2);

if (!NT_SUCCESS(status)) {
// handle failure (code omitted)
}

newbuf2 = WdfMemoryGetBuffer(newmem2, NULL);

At this point, even though I think I’ve replaced “Request”'s WDFMEMORY with “newmem”, newmem2 still is == to origmem and newbuf2 == origbuf. None of the failure paths are being hit.

This makes it difficult to modify the write buffer before sending the request down.

I’ve temporarily worked around this by rolling a new WDFREQUEST and WDFMEMORY, copying all the parameters and info from the original request into the new one, and sending the new request down. But that seems wasteful and is quite a bit more code.

This is using VS2015 and a WS2016 TP4 KMDF target.

Cluestick? My guess is it’s related to the use of WdfIoTargetFormatRequestForWrite, but I re-read the docs a few more times and this seems like the correct function to use here.

Thanks.

-ml

PS - I have the ‘reverse’ operation (eg, putting the original WDFMEMORY back in the request) in the completion path, but obviously if this first change fails, the completion path doesn’t make much sense :slight_smile:

-ml

3

xxxxx@azathoth.net wrote:

… Doron alluded to being able to replace a WDFREQUEST’s WDFMEMORY ala:

“Then all you need to do is allocate a WDFMEMORY with the new expanded size, for
a write operation copy the caller’s buffer content into the new buffer, format
the request with your WDFMEMORY and adjusted size, and send the request down the
stack.”

I needed to do something similar, so I tried to follow that approach. When I reformat the request using my new WDFMEMORY, it returns STATUS_SUCCESS, but turning around and obtaining the request’s WDFMEMORY again returns the original WDFMEMORY, as if nothing was changed. There are no errors returned, and the wdf logs from windbg only show power operations (eg, no “read only” buffers or other errors).

A WDFREQUEST has essentially two sets of state: the stuff it got from
above, and the stuff it is preparing to send below. When you format the
request for write, you are preparing things for the NEXT driver.
WdfRequestRetrieveInputMemory is fetching the buffer you were handed.
You’re going to need that eventually, since you have to copy the results
back at some point.

You’ll have to keep track of the buffer in a context somewhere.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

4

On Mon, Feb 08, 2016 at 12:07:43PM -0800, Tim Roberts wrote:

xxxxx@azathoth.net wrote:
> … Doron alluded to being able to replace a WDFREQUEST’s WDFMEMORY ala:
>
> “Then all you need to do is allocate a WDFMEMORY with the new expanded size, for
> a write operation copy the caller’s buffer content into the new buffer, format
> the request with your WDFMEMORY and adjusted size, and send the request down the
> stack.”
>
> I needed to do something similar, so I tried to follow that approach. When I reformat the request using my new WDFMEMORY, it returns STATUS_SUCCESS, but turning around and obtaining the request’s WDFMEMORY again returns the original WDFMEMORY, as if nothing was changed. There are no errors returned, and the wdf logs from windbg only show power operations (eg, no “read only” buffers or other errors).

A WDFREQUEST has essentially two sets of state: the stuff it got from
above, and the stuff it is preparing to send below. When you format the
request for write, you are preparing things for the NEXT driver.
WdfRequestRetrieveInputMemory is fetching the buffer you were handed.
You’re going to need that eventually, since you have to copy the results
back at some point.

Thanks Tim, for the quick reply! I’m doing this (and keeping track in a context as you mention next). The way you’ve described it explains what may be going on here.

You’ll have to keep track of the buffer in a context somewhere.

The issue I’m seeing is that the *original* buffer content is being written, not what I replaced it with. I can see this in the read path (eg, I see a write at offset X, which I change using the code I shared, then a subsequent read of offset X). In the read routine, I inspect the buffer and see the original, unmodified buffer content.

There could be some other error, at least now I know the buffer I’m being handed is the upstream one. I’ll do some more digging and reply here if I find out what happened.

Thanks again.

-ml

5

To close this out - the problem was that there were a couple of paths through the driver that weren’t properly reformatting the request. Once those were fixed, the buffer swap worked as expected.

-ml

6

I know this is a pretty old thread, but since OP seems to have managed to come around this - I will go ahead and ask the question.
What’s the appropriate way of doing “I have the ‘reverse’ operation (eg, putting the original WDFMEMORY back in the request) in the completion path” ?

Using one of the format functions or changing the MdlAddress yourself in the completion callback?

7

The format functions are for sending the irp to the next io target, not for reformatting a request on the way back up the stack. you are probably best off re-setting the MdlAddress explicitly in the driver

8

Thanks Doron, I’ll give that a shot.

9

While it makes sense, to restore MdlAddress explicitly in the completion routine, it still hasn’t solved issue that I’m facing.

As an upper disk filter, all I’m trying to do essentially is change buffer data in the write callback and do the reverse in the read callback. I attach my filter to an empty/unallocated .vhd disk in the system even before operations like initializing disk, creating and formatting volume on it are taken.

Steps taken in the Write path:
create new wdfmemory
copy original request buffer
// modify new buffer - not doing this at the moment, just copying original data in new WDFMEMORY
reformat request with new WDFMEMORY
send request

Write completion path:
restore Irp->MdlAddress
free wdfmemory allocated earlier
complete request

But the formatting step always comes back saying “it failed to format”.

Any obvious step that I’m missing out here ?

I did a comparison of IRP fields just before sending it downstream (with new WDFMEMORY) and in the write completion callback, and I could see that 2 fields weren’t same. One is MdlAddress which I restore and another one is PendingReturned = 1. Perhaps this should be handled?

Also is there any tool/utility like FIleSpy at this level that might help in understanding what driver is failing the request and with what status.
This will go a long way in troubleshooting.

Thanks

10

all I’m trying to do essentially is change buffer data in the write callback and do the reverse in the read callback

Sorry… can you explain further? Maybe I’m not following your follow-on to this ANCIENT thread (which is why we ask you to NEVER NECROPOST)…

If somebody issues a READ to you, you can’t change the data buffer you return to them. It’s their data buffer, after all. If somebody does “ReadFile” and gives a pointer and length for the data buffer, you can’t return the data to someplace else, right?

In a decryption scenario, you get the user’s read, you send down YOUR OWN data buffer into which the encrypted data is returned, you decrypt the data, and return the DECRYPTED data into the data buffer the user specified.

I’m sure I’m missing something,

Peter

11

Hi Peter,

At the moment, I’m just looking to get the steps right for the encryption path. Once that’s in place, I can figure out the decryption path with the info you’ve mentioned above. In-fact, I haven’t even registered read request handler at the moment since I’m not really changing any data (sending same data as received but with new WDFMEMORY) in the write callback. The steps I follow in the encryption path are:

Write Callback:
create new wdfmemory → “NewMemory”
copy contents of original request buffer into NewMemory
// modify NewMemory - not doing this at the moment as mentioned before
maintain Irp->MdlAddress and NewMemory in context
reformat request with new WDFMEMORY
send request

Write completion path:
restore Irp->MdlAddress from context
free NewMemory allocated earlier
complete request with status, info etc.

With this in place, I head into diskmgmt.msc and initialize my device (a vhd disk, right-click initialize with either MBR / GPT). Driver receives write callback, takes steps mentioned above - disk is initialized, so far so good. However, when I go a step ahead and “create simple volume” on the disk and format that volume using NTFS/FAT32, diskmgmt just shows dialog box saying it failed to format the volume. There is no specific error thrown at me, don’t see any Wdfxxxxx apis failing in the driver. Don’t see anything in the event viewer also.

I’m sure I’m missing something subtle.

If you reckon this is an odd issue and it would be helpful to create a new post, I can go ahead and do that.

Thanks for your help already.

12

You shouldn’t have to fool with MDLs and such.

OK, let’s take this a step at a time. Try it with creating a NEW Request, and sending that. This is well described in the docs for WdfIoTargetFormatRequestForWrite. If this works, you’ll know you’ve got all the basic mechanics right.

Once that is working… Then just switch out the Request handle in the format call to that of the previous (existing) Request.

See where you get with that.

Hope that helps,

Peter

13

Hi Peter,

Creating a new request and then sending that downstream did not work either. However, for this approach the problem is little different.
With this approach, I see that after certain number of requests, couple of requests are never completed and diskmgmt shows “Formatting volume” forever (again there is no specific error code thrown anywhere).

To give you an idea I see this in WinDbg,

KmdfDiskFilter: DriverEntry
KmdfDiskFilter: PropertyName: \Device\00000024, PropertyLength: 34
KmdfDiskFilter: PropertyName: \Device\00000041, PropertyLength: 34
KmdfDiskFilter: Attached to → \Device\00000041

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 0; Length → 512; ----> printing from Write callback
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 512; Status → [0x0] -----> printing from Write Completion Routine

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 65536; Length → 512;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 512; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 98304; Length → 512;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 512; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 177971200; Length → 16384;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 16384; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 65536; Length → 16384;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 16384; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 98304; Length → 512;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 512; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 533790208; Length → 512;
KmdfDiskFilter: IO → ENCRYPTED WRITE; BytesWritten → 512; Status → [0x0]

KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 173608960; Length → 1048576; ------> CompletionRoutine is never called
KmdfDiskFilter: IO → ORIGINAL WRITE; DeviceOffset → 65536; Length → 983040; ------> CompletionRoutine is never called

14

To keep it short I’m removing error handling code.
There must be something going wrong in the Write Callback rather than in write’s CompletionRoutine since its never called for those last 2 requests above?

Steps in the Write Callback:
// Creating new request. Doesn’t matter how its created, this problem of CompletionRoutine not getting called persists.
WdfRequestCreate(WDF_NO_OBJECT_ATTRIBUTES, IoTarget, &EncryptedRequest);
OR
WdfRequestCreateFromIrp(WDF_NO_OBJECT_ATTRIBUTES, WdfRequestWdmGetIrp(Request), FALSE, &EncryptedRequest);

// Creating new memory.
WdfMemoryCreate(WDF_NO_OBJECT_ATTRIBUTES, NonPagedPoolNx, POOL_TAG, Length, &EncryptedMemory, &EncryptedDataBuffer);

// Retrieving buffer from existing request and copying it to new memory.
WdfRequestRetrieveInputMemory(Request, &InputMemory);
InputBuffer = WdfMemoryGetBuffer(InputMemory, NULL);
WdfMemoryCopyFromBuffer(EncryptedMemory, 0, InputBuffer, Length);

// Formatting the new request.
WdfIoTargetFormatRequestForWrite(IoTarget, EncryptedRequest, EncryptedMemory, NULL, NULL);

// Setting up context, so cleanup of new memory and completion of original request can be done.
CompletionCtx->OrgRequest = Request;
CompletionCtx->EncryptedMemory = EncryptedMemory;

WdfRequestSetCompletionRoutine(EncryptedRequest, FilterEncryptedWriteComplete, (WDFCONTEXT)CompletionCtx);

WdfRequestSend(EncryptedRequest, IoTarget, WDF_NO_SEND_OPTIONS)

15

Are you checking the return from WdfRequestSend? Rememebr, it’s a Boolean, not an NTSTATUS.

Peter

16

indeed I’m. I removed that piece to keep it short.

17

However, just noticed, I do see below in the debugger:
Thread 0xFFFFBF8FBFA22080 is waiting for all inflight requests to be acknowledged on WDFQUEUE 0x000040703DEE8F88

18

The I/O Target you’re using is your local I/O Target…WdfDeviceGetIoTarget?

Peter

19

yes, it’s a local I/O target. Partition gets created fine, its the “format” command that never returns. Probably there is some special handling that needs to be added for the format command to go through ? Not sure if it bears any relevance but the requests that are not getting completed are supposed write significant amount of data (1048576 & 983040 bytes) compared to earlier requests.

20

You’re passing through everything else? IOCTLs, and reads, particularly?

I gotta say… I’m stumped.

Peter