Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Why CcGetFileObjectFromSectionPtrs needed for file reference counting ?

Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198

Hello!

In NT Insider article "Tracking State and Context - Reference Counting for File System Filter Drivers" there is sentence "For each IRP_MJ_CLOSE the filter driver decrements the reference count on the per-file context structure if the FO_STREAM_FILE bit is not set for the file object and if CcGetFileObjectFromSectionPtrs does not return this file object."

I don't understand why we should call CcGetFileObjectFromSectionPtrs.
If this file is being closed then it can't be used for caching. Not anymore. So if we got irp_mj_close we already know that this file is not used for caching.
If the file still being used for caching then we won't get irp_mj_close.

Comments

  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,112

    Consider

    • You get the file object
    • You get pre-empted
    • The newly reactivated thread purges the cache
    • As a result of the purge the file object is closed
    • You get rescheduled
    • Your file object is invalid

    If you’d grabbed a refcount then the file object would still be valid.

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Thank you Rod !

    Not sure I understand.
    If I understand correctly fileobject always valid while I still in the callback.
    Plus CcGetFileObjectFromSectionPtrs doesn’t reference fo. CcGetFileObjectFromSectionPtrsRef does.

    I think that CcGetFileObjectFromSectionPtrs getting called just to check if current fo used for caching. I don’t understand why.

    Or maybe I just misunderstood your explanation?
  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,112
    edited March 26

    Ok Sergey,
    I misread your initial question - I thought you were asking why use CcGetFileObjectFromSectionPtrsRef rather than CcGetFileObjectFromSectionPtrs.

    If you are asking "why would you want to call either of them?" the answer is - you almost certainly don't unless you code also contains code to CcInitializeCacheMap and you are in a crazy complicated situation pobably involing the network". Or even "You don't want to ever call these functions, until you do and then you'll know why".

    I have found myself in such a crazy situation and when I do it I want to look at the particular file object for reasons which are too banal to go into.

    If all you want to know is whether the file is cached or not then CcIsFileCached is your friend.

    Does that answer your question - but to be honest I haven't read that article since it was written aeons ago.

    Post edited by rod_widdowson on
  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Thank you Rod !

    I guess I can use CcIsFileCached. I just don’t understand why would I :)

    Thank you very much for your excellent reply(as always).

    Unfortunately I think that doesn’t answer my question :(
    I really don’t understand the logic in article. From my point of view CcGetFileObjectFromSectionPtrs should always return pointer to another fo on mj_close
  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,112

    From my point of view CcGetFileObjectFromSectionPtrs should always return pointer to another fo on mj_close

    That or null. I'd agree.

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,230

    I will preface by saying that this article is VERY old and none of this matters since Win2K SP4/XP...

    I also agree that this call was unnecessary. For additional confirmation, I just searched our old source code database for file system filters from around this time and we never called CcGetFileObjectFromSectionPtrs.

    -scott
    OSR

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Scott, thank you !

    The article indeed very old, but I think maybe it should be corrected. Years ago while I was junior developer I was given a link to this article and my tutor just said don’t bother about CcGetFileObjectFromSectionPtrs. I think that this article still used as reference till now. And since this article still in use( supposedly) imho it is better to not confuse people.

    Maybe you don’t need this now if you just need context, but in case you do your own reference counting(for example to know when file actually closed) this info is very useful.
  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,230

    You shouldn't need any of the guidance in this article anymore. Filters are no longer responsible for reference counting and knowing when a file/stream goes away. The file systems do it by supporting stream and file contexts (they're the only ones that really know definitively so it makes sense to be there).

    I'd like all the information to be as accurate as possible, but OSR Online is read only at this point. If that ever changes I'm sure we could stay very busy fixing things :)

    -scott
    OSR

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198

    Let's say on file closing I want to take action - delete or don't delete file. How should I do that without counting references myself ? Of course i can do that in context cleanup. But at this time file can be gone or changed days ago. So while I agree that fsrtl and fltmgr contexts are outstanding feature, sometimes you still need to count yourself.

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,230

    Triggering processing based on the last IRP_MJ_CLOSE is generally not a great idea. You're not guaranteed to ever see the last close, even if you shutdown the system.

    This falls into the common problem of people wanting to know when Windows is "done" with a file. Mostly it's sufficient to use IRP_MJ_CLEANUP as that means the application is done with the file, though that doesn't work in the case of memory mapped files. Memory mapped files for write are even worse because when you see the writes is entirely non-deterministic (you can't even necessarily know which File Object is going to be used for the write). In these cases I suppose you could try to trigger your processing based on IRP_MJ_CLOSE, but again you're not guaranteed to ever see it so it depends on precisely what you're trying to accomplish.

    -scott
    OSR

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Thank you!
    For now my plan is to use cleanup and ignore memory mapped files. I will increment counter on each create and decrement on each cleanup for non-stream file object
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 20 Apr 2020 LIVE ONLINE
Writing WDF Drivers 11 May 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA