Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTFSD
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Minifilter | Validity/Scope & Reusability of Objects in the IRP

AaronKAaronK Member Posts: 5

Hello everyone!

I'm Currently writing a File Replication filter for VHDs - Since I am rather new to the topic I have a few questions about the Scope of certain parameters of the IRP.
Would be great if someone had the answers - or point me to a source/google search where I can expand my understanding about the Topics.

  1. Could I use the IOPB.TargetFileObject from the PFLT_CALLBACK_DATA for my own read needs?
    1.1. If no, even If I did an FltObjectReference beforehand? Is this even a sensible question or am I completely mistaken with how I think about this?
    1.2. If yes, would It make more sense to Open the File for myself? I kind of don't want to have to compete with shared Acces rights

  2. Can I Use the Pointer Address to the IOPB.TargetFileObject to find out if further IRPs write to the same file?
    2.1. Could I miss IRPs from other Processes since they might use another File Object Addres - Or does Windows use the same Address to reference the File globally and toss it if nothing is referencing to it anymore?

  3. Am I correct in my assumption that the WriteBuffer / MDL in a writing IRP is only valid until the IRP is done?
    3.1. And if I want to write the same changes to another File I have to copy said buffer?

  4. Each write/read from any Programm etc has to Open, do, Close the File. Is there any way to know if the IRPs I'm filtering are from this Chain/transaction?
    4.1 I think the FileObject Pointer from the Open till the Close should be the same. Can someone Verify that?

  5. The MDL in the FLTWriteFile seems to be Optional? What advantages do I have if I create one for My Write?

  6. FileObjects/Buffer/MDLs/Handles can only be dereferenced/closed/freed as soon as the Operation which uses them is done, not just after I have called said operation. correct?

That's it - a big Thank you in advance!

Comments

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,336

    @AaronK said:
    1. Could I use the IOPB.TargetFileObject from the PFLT_CALLBACK_DATA for my own read needs?

    In general yes that is possible and is a common pattern. You'll see this commonly referred to as "borrowing" the caller's File Object.

    1.1. If no, even If I did an FltObjectReference beforehand? Is this even a sensible question or am I completely mistaken with how I think about this?

    As long as the CALLBACK_DATA (CBD) is not complete you have a reference to the File Object. If you wanted to asynchronously borrow the File Object (e.g. queue a work item and complete the CBD) you would need to reference it. Two things though:

    1. The File Object is an Ob structure and not a Flt structure, so you use ObReferenceObject.
    2. Keeping the File Object alive outside of an I/O operation can cause weird side effects if you keep the reference too long. The problem is that things like delete can fail due to the reference. It gets worse if you're doing this over the network because the reference can also cause sharing violations.

    1.2. If yes, would It make more sense to Open the File for myself? I kind of don't want to have to compete with shared Acces rights

    Yes, you can open it yourself. If you're only working with local files you can bypass share access with IO_IGNORE_SHARE_ACCESS_CHECK.

    1. Can I Use the Pointer Address to the IOPB.TargetFileObject to find out if further IRPs write to the same file?

    No. You want to use a Stream Context for that.

    2.1. Could I miss IRPs from other Processes since they might use another File Object Addres - Or does Windows use the same Address to reference the File globally and toss it if nothing is referencing to it anymore?

    The File Object is an open instance of a Stream. Each call to CreateFile is going to create a new, unique File Object.

    1. Am I correct in my assumption that the WriteBuffer / MDL in a writing IRP is only valid until the IRP is done?
      3.1. And if I want to write the same changes to another File I have to copy said buffer?

    You can send the same IRP (CBD if we're talking FltMgr) multiple times. You just need to reclaim the IRP in the completion routine (Post Operation) and not let it complete to the layers above you're done.

    1. Each write/read from any Programm etc has to Open, do, Close the File. Is there any way to know if the IRPs I'm filtering are from this Chain/transaction?
      4.1 I think the FileObject Pointer from the Open till the Close should be the same. Can someone Verify that?

    Yes, they're the same File Object across all I/O operations to the same open instance. There are exceptions though. For example, if they're using cached I/O you'll see two writes: a cached and then a paging. These can come from different File Objects. Memory mapped I/O is worse because you only see the paging write and can't always definitively know which process wrote to the page.

    1. The MDL in the FLTWriteFile seems to be Optional? What advantages do I have if I create one for My Write?

    If you don't pass an MDL one will eventually be built for you. If you already have an MDL you can pass it in as an optimization.

    1. FileObjects/Buffer/MDLs/Handles can only be dereferenced/closed/freed as soon as the Operation which uses them is done, not just after I have called said operation. correct?

    Correct.

    -scott
    OSR

  • AaronKAaronK Member Posts: 5

    Hi Scott,

    Really, thank you for your great Response :)

    Follow Up to your Answer - about Reissueing the CBD.

    1. I had thought about doing this. Currently im calling an asyncronus FltWriteFile in the preop with a copy of the IOPB.Buffer or MDL since this Is something I was sure would work. Are there any reasons against doing it like this / benefits of reusing the CDB?

    2. I googled a bit about reusing CBDs - From what I have gathered there seem to be two methods to do this,
      2.1 Returning FLT_PREOP_SYNCHRONIZE in the Preop and then in the postop changing the TargetFileObject, setting the data to dirty and then using FltReissueSynchronousIo, correct? (How would I reissue AsyncIOs then, also - what even are Async IOs in this Layer? (ref- question 3))
      2.2 Using FltReuseCallbackData for the CBD, then calling FltPerformSynchronousIo or FltPerformAsynchronousIo, while the MS Docs dont seem to like the Idea of me Using the PerformIo Functions and also warn against using the Reuse for any CBDs I haven't allocated myself... Guess the first way would be better then.
      2.3 - Anything better I should look into?

    3. One thing I don't really get is the Synchronous I/O vs. Asynchronous IO part. The Minifilter filters Writes - Why would it matter if the write was called to be Async/sync? Or is this Synchronous part referencing something else?

    4. Is it even possible to reissue the CBD to an Network Path?

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,336

    You're better off just doing your own FltWriteFile. There are fewer things to worry about, it's conceptually easier, and there's no real downsides. Reissuing CDBs can be handy but best done if you're not changing File Objects.

    WRT sync vs async writes, it doesn't much matter if it's a non-paging I/O. If the caller wants sync and the drivers process it async, the I/O Manager fixes it by blocking the requestor. If the caller wants async and the drivers process it sync, everything works fine (though you'll find that performance tanks if you do this too much).

    Things change when it comes to paging I/O though. If the paging I/O comes in sync then it's very likely that the Mm/Cc have pre-acquired file system locks before sending the I/O. If you aynchronously process this request and send it to the file system from another thread you'll deadlock.

    If you don't already have it you should find a copy of the old Nagar book:

    https://www.amazon.com/Windows-File-System-Internals-Developers/dp/1565922492

    It's OLD so I wouldn't follow any practical advice in it. However, chapters 4-8 give you a good idea about how file system I/O works in Windows. Also spend some time watching Process Monitor output (preferably on a secondary drive in your system) so you can see the patterns.

    -scott
    OSR

  • AaronKAaronK Member Posts: 5

    Hi Scott.

    Guess I will keep my current concept then. No more questions to ask from me as of now.

    You have helped me a lot! Thank you. :smiley:

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 15 Jun 2020 LIVE ONLINE
Writing WDF Drivers 22 June 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA