Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTFSD
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Detect specific Alternate Data Stream

Doug_NDoug_N Member - All Emails Posts: 101
edited June 17 in NTFSD

In PreCreate I'd like to know if a particular Alternate Data Stream exists. The only way I can think to check for this is FltCreateFile, but I'm concerned about the performance impact this might have when doing this on every file open (not directories, not paging file, but just about everything else).

Is there a better way?

Thanks
Doug

Comments

  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,131

    I'm not aware of of any.

     
    Last time I checked:
    1. You cannot send only send down a FileQueryStator a FileQueryStatLx to FltQueryInformationByName() (so not FileQueryStreamInformation which would tell you)
    2. Neither of those two information classes give you stream count (a stream count of 1 would mean it doesn't exist).

     
    On the other hand sending a create which will mostly fail might not hurt too much - and it might have the side effect of charging enough of NTFS's caches to make the following MJ_CREATE faster.

     
    Of course file system type is a good predictor but I'm assuming you have already factored that out.

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 314
    via Email
    > 1. You cannot send only send down a FileQueryStator a FileQueryStatLx to
    > FltQueryInformationByName() (so not FileQueryStreamInformation which would
    > tell you)
    What is FileQueryStat/Lx?
    There isn't even a single Google result on it ;)

    > On the other hand sending a create which will mostly fail might not hurt too
    > much - and it might have the side effect of charging enough of NTFS's caches
    > to make the following MJ_CREATE faster.
    No. The biggest impact is the actual call to open/create/replace a
    file. Disk I/O plays a minor role in general, and especially in the
    case of opening the ADS.

    At best, you can hope that the parent directory is cached, and the the
    disk I/O impact is minimal, but absolutely nothing else.

    And the impact is not trivial. It would not be noticed by a typical
    home user, surfing the web. But we had cases where a simple Photoshop
    session had abysmal performance because of this.

    What we did as the only possible mitigation, that we could think of,
    is to query existence of an ADS only in certain folders, e.g. document
    folders - that made a major difference.
    Granted, this was a decade ago, but we were using SLC SSDs for testing
    at the time, so the impact is still big. Factor in Windows 10
    bloatness, and how big apps are today (especially how heavy file I/O
    intensive they are!), and it might even be worse today than back then.
    At the time, I could even figure the entire Office installation within
    seconds - today, I cannot even figure typing a character in the Start
    Menu's Search box (i.e. there is SO much f***ing open/close, that it
    is ridicolous).
  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,131

    What is FileQueryStat/Lx?

    Sorry, I should have been more clear. These have been a bit of a thorn in my side for long enough that I assumed that my shorthand would be 'understood by everyone' which of course is ridiculous.

     
    You can see FileStatInformation and FileStatLxInformation in recent (Win10 on?) WDM.H

    They can be used in a new syscall called something like NtQueryInformationByName() or some such and appear to be there to allow WSL to do the Un*x syscall stat().

     
    As far as file system minifilters are concerned you could think of term as being like a different way to do IRP_MJ_NETWORK_QUERY_OPEN complete with a complete different way of handling it (right down to a different return status to indicate 'I do not handle it').

     
    If you cannot handle this call then the IoSubsystem will in turn issue a Create/QueryInfo/Close triple.

     
    Fun fact. When I looked (and it may still be true) WSL used them a great deal and in particular it liked to send down an query with a null name and a handle in the OBJECT_ATTRIBUTES......

  • Doug_NDoug_N Member - All Emails Posts: 101

    Thanks for the comments Rod and Dejan.

    I'm mostly trying to do special handling during reads of some files with the ADS attached, and I figured detecting 'my' files in PreCreate would be optimal so I could immediately ignore everything else if they weren't mine. But Dejan is right - there are a heck of a lot of opens constantly occurring. Maybe I need to instead find all possible read paths and do my detection there instead so it ignores all the other opens that never actually read the file. In that case, is FltCreateFile on the ADS the best option, or would FltQueryInformationFile(FileStreamInformation) be more performant?

  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,131

    If this only impacts in the read path, any reason why you cannot do the work in Post Create? Or have I missed something?

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 314
    via Email
    For any more specific idea, you need to tell us more as to what your
    intent/needs are.

    Whether you send IRO_MJ_QUERY_OPEN or FileStatInformation is unimportant.
    Both require the same OPEN processing and that is the perfornance
    bottleneck.

    I see no other way to minimize the impact other than not checking all opens.
  • Doug_NDoug_N Member - All Emails Posts: 101

    Rod, I could do it in PostCreate, but was trying to get out of the way of as much I/O as possible. If I really need to pass through Pre to Post, then might as well defer until Read (I guess?) to avoid the many open/query attribute/close (etc) scenarios that go through. Now to come up with a good way to ensure I don't need to check on every read of every file (i.e. check once, record it somewhere, then check my recorded info on subsequent reads).

    Both require the same OPEN processing and that is the performance bottleneck.

    Dejan, you kind of answered my question - either way probably has a similar hit.

    I appreciate both of your feedback.

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 314
    via Email
    IIRC, you cannot defer to a read. FltCreateFile will not always work
    during reads.

    Kind regards, Dejan.
  • Doug_NDoug_N Member - All Emails Posts: 101

    That's good to know.

    I'm leaning towards using FltQueryInformationFile(FileStreamInformation) during the read - that way the filter doesn't need to keep track of or lookup the filename (I think - haven't gotten that far yet). Also leaning towards using a file stream context to track whether this is a file of interest or not.

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 314
    via Email
    Same issue - not callable during some reads.
    But you might be onto something.. if you do this during Post-create,
    you need not open the file, as you already have a handle (and that was
    the the most performance killing part of the task).
    The FileStreamInformation should be nothing compared to a read, much
    less vs. open processing.

    Pitty I could not apply this as I needed the results in pre-create.

    > I'm leaning towards using FltQueryInformationFile(FileStreamInformation)
    > during the read - that way the filter doesn't need to keep track of or
    > lookup the filename (I think - haven't gotten that far yet). Also leaning
    > towards using a file stream context to track whether this is a file of
    > interest or not.
  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,300

    @Doug_N can you describe the problem you're trying to solve? I understand that you want to know if a file has a particular ADS, but why? Just want to make sure there isn't a better way to do what you're trying to accomplish (i.e. if an ADS is actually the right solution).

    -scott
    OSR

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 15 Jun 2020 LIVE ONLINE
Writing WDF Drivers 22 June 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA