Reading volume offset from KMDF volume filter driver during volume attach

Anand_A · February 13, 2017, 7:42am

Hi All,

I have a KMDF volume class upper filter driver (for change block tracking) that is writing some information to the volume at system shutdown time (using volume offsets of a known file). The offset is saved in registry. Now, when the system is booting, I want to read at the known offset. This approach has been suggested in some earlier discussions like this one :
http://www.osronline.com/showThread.cfm?link=270963

Although the above link also says that “I was able to read volume blockwise in EVT_WDF_DEVICE_PREPARE_HARDWARE”, I am not able to do so and further, from other responses on the same thread, it seems that is not possible anyway. (To be more elaborate, issuing reads on the volume using WdfIoTargetSendReadSynchronously() returns STATUS_NO_SUCH_DEVICE in EvtDriverDeviceAdd and returns STATUS_DEVICE_OFF_LINE in EvtDevicePrepareHardware)

So, in KMDF, since the framework handles IRP_MN_START_DEVICE with a sequence of callbacks, how do I hook my driver into the completion callback ? Do I need to call WdfDeviceInitAssignWdmIrpPreprocessCallback() and set a completion routine ?

On some other discussions for achieving what I am trying to, I also see suggestions to do the processing of persisted data when the first IO operation on the filtered device is received. This leads to queuing the work to another thread or queuing a workitem (since EvtIoWrite() is called at <= DISPATCH) and the driver has to be ready to track writes that happen before the decision on whether to continue tracking the changes on the volume across reboot can be taken

Can you please suggest if I should still pursue IRP_MN_START_DEVICE path (and if so, how) or should I switch over to the other approach as IRP_MN_START_DEVICE is not the correct way ?

Thanks,
Anand.

Scott_Noone_OSR · February 13, 2017, 1:09pm

The volume stack has its own (undefined) rules for when the first I/O can occur. The rules also vary between system and data volumes.

You’ll need to come up with a heuristic for when it’s safe to read. Ideally you can coordinate with something higher in the stack to know (e.g. if a file system filter initializes you know that you can read or, better, have it read for you), but that’s usually too late. Needing to be the “first” thing to read is usually more complex, you need to key off some incoming I/O request and deal with other threads also trying to read (or write).

Ultimately there’s no one answer, it depends on your design and requirements. This is the one of the painful (fun?) parts of trying to do something non-standard in Windows.

-scott
OSR

Mark_Roddy · February 14, 2017, 6:54pm

I give up. Why isn’t (ab)using the first read to the volume you are
filtering (other than the boot volume) a fine way to both know when it is
safe to read and a when to do so before anyone else gets to read?

Mark Roddy

On Mon, Feb 13, 2017 at 1:09 PM, wrote:

> The volume stack has its own (undefined) rules for when the first I/O can
> occur. The rules also vary between system and data volumes.
>
> You’ll need to come up with a heuristic for when it’s safe to read.
> Ideally you can coordinate with something higher in the stack to know (e.g.
> if a file system filter initializes you know that you can read or, better,
> have it read for you), but that’s usually too late. Needing to be the
> “first” thing to read is usually more complex, you need to key off some
> incoming I/O request and deal with other threads also trying to read (or
> write).
>
> Ultimately there’s no one answer, it depends on your design and
> requirements. This is the one of the painful (fun?) parts of trying to do
> something non-standard in Windows.
>
> -scott
> OSR
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: http:> showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:></http:>

Scott_Noone_OSR · February 15, 2017, 7:59am

Keying off the first read means you’re having faith that someone above you knows something you don’t. That’s fine, but it’s still a heuristic and not the rule established by the volume manager (i.e. “someone else’s first read is OK, it just can’t be yours”). You could try looking for the related IOCTLs and key off that, but you’re still using a heuristic based on behavior.

The only way to truly know that the volume is ready for reading is to be higher in the stack when the file system is available or be the file system.

I know that you (Mark) understand the choices here, but the OP needs to understand that these aren’t architecturally defined rules. I have keyed off the first read in the past but I’m not going to say that’s the right answer forever and all time in all situations with all third parties present (by way of example but not limitation)

-scott
OSR

Mark_Roddy · February 15, 2017, 9:23am

Fair enough.

Mark Roddy

On Wed, Feb 15, 2017 at 7:58 AM, wrote:

> Keying off the first read means you’re having faith that someone above you
> knows something you don’t. That’s fine, but it’s still a heuristic and not
> the rule established by the volume manager (i.e. “someone else’s first read
> is OK, it just can’t be yours”). You could try looking for the related
> IOCTLs and key off that, but you’re still using a heuristic based on
> behavior.
>
> The only way to truly know that the volume is ready for reading is to be
> higher in the stack when the file system is available or be the file system.
>
> I know that you (Mark) understand the choices here, but the OP needs to
> understand that these aren’t architecturally defined rules. I have keyed
> off the first read in the past but I’m not going to say that’s the right
> answer forever and all time in all situations with all third parties
> present (by way of example but not limitation)
>
> -scott
> OSR
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: http:> showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:></http:>

amitr0-2 · February 15, 2017, 7:17pm

how about first write, because it is a change block tracking driver., yes
the first IO might suffer some performance penalty. also attach to FS, as
Scott said. you will need both.

On Wed, Feb 15, 2017 at 6:22 AM, Mark Roddy wrote:

> Fair enough.
>
> Mark Roddy
>
> On Wed, Feb 15, 2017 at 7:58 AM, wrote:
>
>> Keying off the first read means you’re having faith that someone above
>> you knows something you don’t. That’s fine, but it’s still a heuristic and
>> not the rule established by the volume manager (i.e. “someone else’s first
>> read is OK, it just can’t be yours”). You could try looking for the related
>> IOCTLs and key off that, but you’re still using a heuristic based on
>> behavior.
>>
>> The only way to truly know that the volume is ready for reading is to be
>> higher in the stack when the file system is available or be the file system.
>>
>> I know that you (Mark) understand the choices here, but the OP needs to
>> understand that these aren’t architecturally defined rules. I have keyed
>> off the first read in the past but I’m not going to say that’s the right
>> answer forever and all time in all situations with all third parties
>> present (by way of example but not limitation)
>>
>> -scott
>> OSR
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> Visit the list online at: http:>> lists.cfm?list=ntdev>
>>
>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>> software drivers!
>> Details at http:
>>
>> To unsubscribe, visit the List Server section of OSR Online at <
>> http://www.osronline.com/page.cfm?name=ListServer>
>>
>
> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
> on crash dump analysis, WDF, Windows internals and software drivers!
> Details at To unsubscribe, visit the List Server section of OSR Online at

–

- ab</http:></http:>

Jamey_Kirby · February 15, 2017, 7:26pm

You do not needed to attach to the FS. However, first write IS the best
solution for initialization.

On Wed, Feb 15, 2017 at 7:17 PM Amitrajit B wrote:

> how about first write, because it is a change block tracking driver., yes
> the first IO might suffer some performance penalty. also attach to FS, as
> Scott said. you will need both.
>
> On Wed, Feb 15, 2017 at 6:22 AM, Mark Roddy wrote:
>
> Fair enough.
>
> Mark Roddy
>
> On Wed, Feb 15, 2017 at 7:58 AM, wrote:
>
> Keying off the first read means you’re having faith that someone above you
> knows something you don’t. That’s fine, but it’s still a heuristic and not
> the rule established by the volume manager (i.e. “someone else’s first read
> is OK, it just can’t be yours”). You could try looking for the related
> IOCTLs and key off that, but you’re still using a heuristic based on
> behavior.
>
> The only way to truly know that the volume is ready for reading is to be
> higher in the stack when the file system is available or be the file system.
>
> I know that you (Mark) understand the choices here, but the OP needs to
> understand that these aren’t architecturally defined rules. I have keyed
> off the first read in the past but I’m not going to say that’s the right
> answer forever and all time in all situations with all third parties
> present (by way of example but not limitation)
>
> -scott
> OSR
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
>
>
> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
> on crash dump analysis, WDF, Windows internals and software drivers!
> Details at To unsubscribe, visit the List Server section of OSR Online at
>
>
>
>
> –
>
> - ab
> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
> on crash dump analysis, WDF, Windows internals and software drivers!
> Details at To unsubscribe, visit the List Server section of OSR Online at</http:>

Anand_A · February 16, 2017, 2:45am

Thanks everyone for the helpful responses

The volume stack has its own (undefined) rules for when the first I/O can occur.
The rules also vary between system and data volumes.
@Scott : Are you asserting that the system and / or boot volumes may be written to before a volume class upper filter driver is loaded ? I understand that they reads will happen but in general reboot scenarios (lets not consider writes that can happen, say, because of BSOD), can there be writes too ?

If writes can indeed happen, how do I filter out those volumes ? And should I also consider all volumes having special file (page, hibernation and dump) ?

For system and / or boot volume, I do see a few threads :
http://www.osronline.com/showThread.cfm?link=111292

[quote] I think you need to look at the device, not the volume. I’ve not tried it,
but this should work [/quote]

http://www.osronline.com/showthread.cfm?link=141744

[quote] It is a Disk Class upper filter so the
device objects I’m looking at are disk device objects, not volume device
objects. I’m guessing thats why I’m not seeing that DO_SYSTEM_BOOT_PARTITION
flag set. [/quote]

So, both threads are contradicting each other. I think I saw another thread that was asking to open SystemRoot and backtrack from there but I have lost that thread. Will try to see which of the 2 approaches above work for at least system and boot volumes.

Scott_Noone_OSR · February 16, 2017, 1:43pm

NTFS typically writes to the volume as part of processing the mount request
so I’m not sure using the first write is really a performance benefit.

-scott
OSR
@OSRDrivers

“Jamey Kirby” wrote in message news:xxxxx@ntdev…
You do not needed to attach to the FS. However, first write IS the best
solution for initialization.

On Wed, Feb 15, 2017 at 7:17 PM Amitrajit B wrote:

how about first write, because it is a change block tracking driver., yes
the first IO might suffer some performance penalty. also attach to FS, as
Scott said. you will need both.

On Wed, Feb 15, 2017 at 6:22 AM, Mark Roddy wrote:

Fair enough.

Mark Roddy

On Wed, Feb 15, 2017 at 7:58 AM, wrote:
Keying off the first read means you’re having faith that someone above you
knows something you don’t. That’s fine, but it’s still a heuristic and not
the rule established by the volume manager (i.e. “someone else’s first read
is OK, it just can’t be yours”). You could try looking for the related
IOCTLs and key off that, but you’re still using a heuristic based on
behavior.

The only way to truly know that the volume is ready for reading is to be
higher in the stack when the file system is available or be the file system.

I know that you (Mark) understand the choices here, but the OP needs to
understand that these aren’t architecturally defined rules. I have keyed off
the first read in the past but I’m not going to say that’s the right answer
forever and all time in all situations with all third parties present (by
way of example but not limitation)

-scott
OSR

—
NTDEV is sponsored by OSR

Visit the list online at:
http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at
http:

— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on
crash dump analysis, WDF, Windows internals and software drivers! Details at
To unsubscribe, visit the List Server section of OSR Online at

–

- ab
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on
crash dump analysis, WDF, Windows internals and software drivers! Details at
To unsubscribe, visit the List Server section of OSR Online at</http:></http:></http:>

Scott_Noone_OSR · February 16, 2017, 2:07pm

No, my point is that with system/boot volumes you don’t need to wait for
some state change from above before you can perform I/O on the volume.
Whether or not you choose to take advantage of this is up to you (I usually
create a generic mechanism and then don’t care about special casing the
system/boot volume).

How you deal with paging files is up to your design. You could choose to
ignore them in the filter or in the application that performs the backup.
Crash dump and hibernate files are not writing to using the standard
mechanism and bypass volume filters.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntdev…

Thanks everyone for the helpful responses

The volume stack has its own (undefined) rules for when the first I/O can
occur.
The rules also vary between system and data volumes.
@Scott : Are you asserting that the system and / or boot volumes may be
written to before a volume class upper filter driver is loaded ? I
understand that they reads will happen but in general reboot scenarios (lets
not consider writes that can happen, say, because of BSOD), can there be
writes too ?

If writes can indeed happen, how do I filter out those volumes ? And should
I also consider all volumes having special file (page, hibernation and dump)
?

For system and / or boot volume, I do see a few threads :
http://www.osronline.com/showThread.cfm?link=111292

[quote] I think you need to look at the device, not the volume. I’ve not
tried it,
but this should work [/quote]

http://www.osronline.com/showthread.cfm?link=141744

[quote] It is a Disk Class upper filter so the
device objects I’m looking at are disk device objects, not volume device
objects. I’m guessing thats why I’m not seeing that
DO_SYSTEM_BOOT_PARTITION
flag set. [/quote]

So, both threads are contradicting each other. I think I saw another thread
that was asking to open SystemRoot and backtrack from there but I have lost
that thread. Will try to see which of the 2 approaches above work for at
least system and boot volumes.