Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

FsContext is NULL on NTFS volume for some files

Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
Hi

My purpose is;
to track write and read operations for some specific files. For that reason, I wrote a post create callback and I am creating a Stream Context for some files and attach it to that file object. This works for many files, but not for some others. I got STATUS_NOT_SUPPORTED. All files are on same NTFS volume. I looked at the FILE_OBJEST's FSContext variable. They are NULL. These are the temp files with "tmp" extension and used by CorelDraw application during file saving. Interestingly, context are successfuly attached to some temp files.

My question is;
1. I need to track all read and writes to some specific files. I can not use file name in read and write callback. How can I distinguish that request if I cannot attach a context them ?

2. If FsContext is Null, can I set them to any value. Is this cause a problem ?

3. Ms office also saves files as temp file then rename it. Coupd you please provide me some information about temp files (which are used during saving).

Target platform is Windows 10 64bit. Development tool is Visual Studio 2015.

Regards

Comments

  • rod_widdowsonrod_widdowson Member - All Emails Posts: 972
    > This works for many files, but not for some others. I got
    > STATUS_NOT_SUPPORTED. All files are
    > on same NTFS volume. I looked at the FILE_OBJEST's FSContext variable.
    > They are NULL.

    Yup. Quite a lot of pseudo files do it (paging files do IIRC)

    > These are the temp files with "tmp" extension and used by CorelDraw
    > application during file saving.

    Your surprise me significantly. Are you sure that you really are in a path
    that has the file successfully created/opened?

    STATUS_REPARSE & STATUS_PENDING are both success values so you have to
    (pretty much uniquely) test explicitly against STATUS_SUCCESS;

    > 2. If FsContext is Null, can I set them to any value.
    No.
    > Is this cause a problem ?
    Only if you don't mind crashes:-)

    > 3. Ms office also saves files as temp file then rename it. Coupd you
    > please provide me some information
    > about temp files (which are used during saving).

    Only that Office does lots of weird things and said weird things are more
    than likely to change between releases (or these days on a Wednesday morning
    after a windows update). Further, if you write any security code which
    depends on observed application behavior you are 99% to having coded an
    exploit for your code - all the exploit has to do is something which isn't
    what your code expects. In Filesystems, as in all security subsystems, you
    cannot rely on things being well behaved.

    R
  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    > Yup. Quite a lot of pseudo files do it (paging files do IIRC)

    At the moment, I do not know exactly what pseudo files are (for Windows). Is there any specific attribute to identify them ? Or do you mean They are just a normal/flat file used by application for trivial tasks.
    I googled for IIRC but cannot find a usefull information. What is IIRC ?

    >Your surprise me significantly. Are you sure that you really are in a path that has the file successfully created/opened?
    I do context attachment after file extension comprasion in Post Create callback routine which is synchorized with pre-create callback. In post-create callback, I am provied with a not-null FILE_OBJECT variable. I do not exactly check whether the underlying file system successfully created/opened the file or not. Should I look at Data->IoStatus.Status variable for that ?

    I want to give more details what my purpose is;

    I focused on writing a file encryption/decryption driver (XOR bytes) for some specific files. I intercepts files by their extension. I Know that File Extention Is Not A Robust Way To Do That but it is a first step to learn File System fundementals. Currently, I successfully intercepts read requests and XOR them. All reads by Notepad.exe, CorelDraw, Open Office or Ms Office applications are successfully decrypted before passing data to application, cache or virual memory manager. Writes are problematic in MS and Open office.

    For a specific file extentions, I want to follow a generic way to intercept all write requests. Because FltObjects->FileObject->Filename is empty in pre-write path I cannot distinguish the requests.

    The question is How can I identify the target file's extention in pre-write operation callback routine regardless of where it came from such as Virtual manager, Cache manager or user space application.

    I read many pages on OSR which is a valuable and may be the only resouce for driver developers. However, the problem about OSR is some answers in forum pages raise more question. Could you please direct me to the right way without raising more question ? Just write steps in one sentence each, I'll try to implement them.

    My best regards
  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    I realized that if file opened by another process (still opened), next create/open request fails and FltSetStreamContext returns STATUS_NOT_SUPPORTED.
  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    I just add stream context to the files which is opened/created successfuly. After that, I do not get STATUS_NOT_SUPPORTED error code anymore. (Thanks for advice Rod.)

    I cannot intercept read/writes to the some of the temp files even though they have context. That is, some temp files are intercepted in read/write path, others seems just opened and closed without any data flow. It can be usual behaver for some operations such as maybe file is opened for getting some metadata information then closed immediately. Problem is that without my driver, these files get more write requests and this makes them bigger in size. Then they are renamed.

    I need suggestions
  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,021
    I don't have an exact, easy answer to your question.

    You need to understand how files are read/written in Windows file systems.
    It's not sufficient to just attach a stream context and assume that you'll
    see a write from application X. The file might be memory mapped, extended,
    and then modified. At some point you'll eventually see an IRP_MJ_WRITE
    (possibly from the System process).

    You need to learn by doing...

    Build your own version of the FASTFAT source and play with it.

    Run Process Monitor and watch what happens for common user actions (e.g.
    copy/paste).

    Run Process Monitor and watch what happens when you trigger operations with
    FileTest (http://zezula.net/en/fstools/filetest.html)

    Read the Windows NT File System Internals book (twice!) -
    https://store.osr.com/product/osrs-classic-reprints-windows-nt-file-system-internals/

    Take a class - https://www.osr.com/seminars/minifilters/

    Feel free to ask *specific* questions along the way (e.g. "I wrote this test
    app and I don't see the writes in my minifilter, why?")

    Once you understand the Windows file system behaviors you should be able to
    answer your own question. Yes, I do realize that this is like saying, "eat
    this entire elephant and you won't ever need to eat again!" but it's the
    truth.

    -scott
    OSR
    @OSRDrivers

    -scott
    OSR

  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    Thank a lot Scott.

    > I don't have an exact, easy answer to your question. You need to understand how files are
    > read/written in Windows file systems. It's not sufficient to just attach a stream context and
    > assume that you'll see a write from application X. The file might be memory mapped, extended,
    > and then modified. At some point you'll eventually see an IRP_MJ_WRITE (possibly from the
    > System process). You need to learn by doing...

    As you said All IO requests eventually come to my pre-write callback but I missed it since I cannot identify them. I cannot intercept and XOR all files. I tried alternatives to Stream Contexts such as File Id or Object Id etc. But failed. The problem is I do not get any error while attaching context in post-create callback, but when it comes to pre-write callback I cannot get context variable even though FsContext is not null. Why ? Mostly and interestingly this happens *.tmp or *.xml files which accessed by CorelDraw application and a few tmp file in Windows temp folder.

    > Build your own version of the FASTFAT source and play with it.
    I do not have enough time at the moment but I'll do.

    > Run Process Monitor and watch what happens for common user actions (e.g. copy/paste). Run
    > Process Monitor and watch what happens when you trigger operations with FileTest
    > (http://zezula.net/en/fstools/filetest.html)

    I am using these applications (Filetest, Process monitor etc). Maybe I missed something.

    > Read the Windows NT File System Internals book (twice!) - https://store.osr.com/product/osrs-
    > classic-reprints-windows-nt-file-system-internals/

    I already did (at least twice). Maybe more for some parts. And I have still googled on the web.

    > Take a class - https://www.osr.com/seminars/minifilters/ Feel free to ask *specific* questions
    > along the way (e.g. "I wrote this test app and I don't see the writes in my minifilter, why?")

    Why not.

    > Once you understand the Windows file system behaviors you should be able to answer your
    > own question. Yes, I do realize that this is like saying, "eat this entire elephant and you won't
    > ever need to eat again!" but it's the truth.

    Appreciate your help Scott.

    Last two questions:

    1. Is it possible for you to write a simple code like that
    a. Allocate and Attach a stream context to all File Object in post-create .
    b. Then try to get it in pre-write callback.

    2. MS web site says;
    "The NTFS and FAT file systems do not support file, stream, or file object contexts on paging files."
    As far as I know there is only one "paging file" in windows which is "pagefile.sys". Does it mean "mapping file" ?
  • Mike_BoucherMike_Boucher Member - All Emails Posts: 13
    >As far as I know there is only one "paging file" in windows which is
    "pagefile.sys". Does it mean "mapping file" ?

    You can have one page file per volume, but if you create page files on
    volumes that all reside on the same physical device then that's probably
    going to make your life worse.

    Microsoft recommends multiple page files on multiple physical devices if
    you need to make paging faster.

    On Wed, Sep 5, 2018 at 3:41 PM, xxxxx@gmail.com
    wrote:

    > Thank a lot Scott.
    >
    > > I don't have an exact, easy answer to your question. You need to
    > understand how files are
    > > read/written in Windows file systems. It's not sufficient to just attach
    > a stream context and
    > > assume that you'll see a write from application X. The file might be
    > memory mapped, extended,
    > > and then modified. At some point you'll eventually see an IRP_MJ_WRITE
    > (possibly from the
    > > System process). You need to learn by doing...
    >
    > As you said All IO requests eventually come to my pre-write callback but I
    > missed it since I cannot identify them. I cannot intercept and XOR all
    > files. I tried alternatives to Stream Contexts such as File Id or Object Id
    > etc. But failed. The problem is I do not get any error while attaching
    > context in post-create callback, but when it comes to pre-write callback I
    > cannot get context variable even though FsContext is not null. Why ? Mostly
    > and interestingly this happens *.tmp or *.xml files which accessed by
    > CorelDraw application and a few tmp file in Windows temp folder.
    >
    > > Build your own version of the FASTFAT source and play with it.
    > I do not have enough time at the moment but I'll do.
    >
    > > Run Process Monitor and watch what happens for common user actions (e.g.
    > copy/paste). Run
    > > Process Monitor and watch what happens when you trigger operations with
    > FileTest
    > > (http://zezula.net/en/fstools/filetest.html)
    >
    > I am using these applications (Filetest, Process monitor etc). Maybe I
    > missed something.
    >
    > > Read the Windows NT File System Internals book (twice!) -
    > https://store.osr.com/product/osrs-
    > > classic-reprints-windows-nt-file-system-internals/
    >
    > I already did (at least twice). Maybe more for some parts. And I have
    > still googled on the web.
    >
    > > Take a class - https://www.osr.com/seminars/minifilters/ Feel free to
    > ask *specific* questions
    > > along the way (e.g. "I wrote this test app and I don't see the writes in
    > my minifilter, why?")
    >
    > Why not.
    >
    > > Once you understand the Windows file system behaviors you should be able
    > to answer your
    > > own question. Yes, I do realize that this is like saying, "eat this
    > entire elephant and you won't
    > > ever need to eat again!" but it's the truth.
    >
    > Appreciate your help Scott.
    >
    > Last two questions:
    >
    > 1. Is it possible for you to write a simple code like that
    > a. Allocate and Attach a stream context to all File Object in
    > post-create .
    > b. Then try to get it in pre-write callback.
    >
    > 2. MS web site says;
    > "The NTFS and FAT file systems do not support file, stream, or file
    > object contexts on paging files."
    > As far as I know there is only one "paging file" in windows which is
    > "pagefile.sys". Does it mean "mapping file" ?
    >
    > ---
    > NTFSD is sponsored by OSR
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    Mike

    If that is the case, the problem is not about paging files. So, wdo I miss some write ?

    Please someone post a simple example here;
    1. Allocate and attach stream context to FILE_OBJECT the
    2. Get it in pre-write callback

    Swapbuffer example in github can be used to create such a project. I started from that project https://github.com/Microsoft/Windows-driver-samples/tree/master/filesys/miniFilter/swapBuffers

    Regards
  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    I didn't read Windows NT File System Internals. just read Windows Internals. I will.
  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,021
    <QUOTE>
    The problem is I do not get any error while attaching context in post-create
    callback, but when it comes to pre-write callback I cannot get context
    variable even though FsContext is not null. Why ? Mostly and interestingly
    this happens *.tmp or *.xml files which accessed by CorelDraw application
    and a few tmp file in Windows temp folder.
    </QUOTE>

    Are you attaching your minifilter during boot or on demand after the system
    starts? If you're attaching on demand it's possible that you've missed the
    open of the temporary files.

    <QUOTE>
    1. Is it possible for you to write a simple code like that
    a. Allocate and Attach a stream context to all File Object in
    post-create .
    b. Then try to get it in pre-write callback.
    </QUOTE>

    See the Change sample:

    https://github.com/Microsoft/Windows-driver-samples/tree/master/filesys/miniFilter/change

    With respect to paging files, that's an unrelated topic. They're the special
    "swap files" named pagefile.sys and have special processing in the file
    system. You can detect someone opening a paging file by checking for the
    SL_OPEN_PAGING_FILE in PreCreate

    -scott
    OSR
    @OSRDrivers

    -scott
    OSR

  • Murad_KarakasMurad_Karakas Member - All Emails Posts: 13
    <QUOTE> Are you attaching your minifilter during boot or on demand after the system starts? If you're attaching on demand it's possible that you've missed the open of the temporary files.</QUOTE>
    I test it on boot. Nothing changed.

    I'll look the "change" example.

    Thanks again scott
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!