Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTFSD

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

Transactional NTFS future - the real story

David_GilbertDavid_Gilbert Member Posts: 28
Hello,

I have developed an NTFS minifilter driver that I am considering enhancing to take account of Transactional NTFS (TxF) functionality.

It seems to me, however, that every piece of Microsoft documentation for the TxF APIs include a disclaimer similar to the following:

"Microsoft strongly recommends developers utilize alternative means to achieve your application?s needs. Many scenarios that TxF was developed for can be achieved through simpler and more readily available techniques. Furthermore, TxF may not be available in future versions of Microsoft Windows. For more information, and alternatives to TxF, please see Alternatives to using Transactional NTFS."

These disclaimers appear even more emphatic in the Windows 8 preliminary documentation.

Does anyone have any insight into what the real story is concerning TxF futures?
Do any Microsoft or major 3rd party applications actually use it?

Any information or insights are extremely welcome.

Thanks in advance,
David Gilbert

Comments

  • Bill_WandelBill_Wandel Member - All Emails Posts: 237
    I have seen it used by Microsoft. I don't remember the particulars. I saw it
    when I had a crash in my transaction handling. It was probably in W2K8 R2.

    Bill Wandel

    -----Original Message-----
    From: [email protected]
    [mailto:[email protected]] On Behalf Of
    [email protected]
    Sent: Thursday, August 09, 2012 1:23 PM
    To: Windows File Systems Devs Interest List
    Subject: [ntfsd] Transactional NTFS future - the real story

    Hello,

    I have developed an NTFS minifilter driver that I am considering enhancing
    to take account of Transactional NTFS (TxF) functionality.

    It seems to me, however, that every piece of Microsoft documentation for the
    TxF APIs include a disclaimer similar to the following:

    "Microsoft strongly recommends developers utilize alternative means to
    achieve your application?s needs. Many scenarios that TxF was developed for
    can be achieved through simpler and more readily available techniques.
    Furthermore, TxF may not be available in future versions of Microsoft
    Windows. For more information, and alternatives to TxF, please see
    Alternatives to using Transactional NTFS."

    These disclaimers appear even more emphatic in the Windows 8 preliminary
    documentation.

    Does anyone have any insight into what the real story is concerning TxF
    futures?
    Do any Microsoft or major 3rd party applications actually use it?

    Any information or insights are extremely welcome.

    Thanks in advance,
    David Gilbert

    ---
    NTFSD is sponsored by OSR

    For our schedule of debugging and file system seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at
    http://www.osronline.com/page.cfm?name=ListServer
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >I have seen it used by Microsoft. I don't remember the particulars.

    Windows Update uses it.

    --
    Maxim S. Shatskih
    Windows DDK MVP
    [email protected]
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Transactions can greatly complicate filters. For example, in isolation filters, you must add special handling for them, because you have to distinguish opens inside from outside a transaction. When the transaction commits, you need to close any shadow file objects you maintain because otherwise they can create subsequent problems.

    For example, if someone opens up a directory inside a transaction, you need to distinguish that from an open of the directory outside the transaction. Then, if they do a relative open (using the directory - or even a file in the case of streams) relative to that initial transaction you need to ensure they are within the same transaction (although it does seem to work if you open the directory OUTSIDE the transaction but then do the relative open INSIDE a transaction.)

    But the usual tricks of trying to keep file objects around becomes problematic - because if you have the directory open inside a transaction and someone tries to open it from a second transaction, you will observe a transactional conflict.

    Ideally, the best policy is really to try and avoid transactions if at all possible. But if you MUST deal with them, be prepared to do quite a bit of work.

    Only NTFS currently supports transactions. It is POSSIBLE to build other transactional file systems, but as far as I know, no one has done that, nor has anyone suggested doing so to me - perhaps others in this space have had different experiences.

    Tony
    OSR
  • David_GilbertDavid_Gilbert Member Posts: 28
    I understand the nature and operation of transactions in TxF and their relationship to the Kernel Transaction Manager.

    What I am looking for is some insight into the future of Transactional NTFS. There is a Microsoft disclaimer on every MSDN page that describes a TxF API that discourages developers from using the API and warns about the disappearance of TxF in future versions of Windows.

    The wording of this disclaimer and its ubiquitous presence leads me to believe that TxF was never used by any major 3rd party products and that its demise is imminent. If anyone close to the gang in Redmond has any information that can be shared I would be very grateful.

    Thanks and regards,
    David Gilbert
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Well, you won't get a commitment from Microsoft one way or the other on TxF is what I suspect, because the general rules of engagement with the outside development community are quite clear on not discussing future product plans with same.

    I've been watching the TxF evolution for 10+ years now and I must admit, I've always tried to understand who was truly going to use it. The only real-world use that I've seen for it is in implementing TxR and installers.

    The most telling thing to me is that ReFS does not support transactions. That suggests to me this is a feature that does not have sufficient demand to justify expending the resources to build TxF support into ReFS. In some ways, ReFS allows Microsoft to focus on those file system features that are useful and to drop support for those features that aren't (and yet cost tremendous resources to maintain).

    I have not noticed the warning message you've described on the MSDN documentation pages I've viewed regarding transactions. For example: http://msdn.microsoft.com/en-us/library/windows/hardware/ff566422(v=vs.85).aspx describes ZwCreateEnlistment, and I don't see any caveats to this saying that this API will not be supported in future OS releases.

    Tony
    OSR
  • David_GilbertDavid_Gilbert Member Posts: 28
    Tony,

    Thanks for the response. I have been looking at other blogs and have come to the conclusion that TxF was never really used because it was too restrictive (no SMB support) and too complicated.

    Concerning the MSDN caveat that I mentioned, it is on all the user-level TxF API pages (CreateFileTransacted, DeleteFileTransacted,...). Looking at ZwCreateTransaction, ZwCreateEnlistment, and other kernel level APIs, it appears that they refer to KTM transactions of which TxF transactions are a subset. Am I near the truth or way off the reservation?

    David Gilbert
  • Christian_AllredChristian_Allred Member Posts: 104
    KTM and TxF are two parts of an entire mechanism. KTM provides the "transaction manager" part and TxF provides the "resource manager" part.

    When you call CreateTransaction() in Win32, this ends up being a call to NtCreateTransaction (== ZwCreateTransaction). Then you take the handle to the transaction you just created and feed it to CreateFileTransacted(), DeleteFileTransacted(), etc. This causes NTFS (of which TxF is part) to enlist on the transaction. Once that enlistment is done, what happens to the file is bound to the lifetime of the transaction, as managed by KTM. So if you call CommitTransaction() that tells KTM to commit the transaction. He in turn sends messages to everything that has enlisted on that transaction (such as NTFS/TxF, TxR, filters that have called FltEnlistInTransaction, etc.) to commit their work. Similarly, calling RollbackTransaction() or closing your handle to the transaction tells KTM to abort the transaction, so he tells all enlistments to roll back their work.
    --
    Christian [MSFT]
    This posting is provided "AS IS" with no warranties, and confers no rights.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Internals & Software Drivers 15 November 2021 Live, Online
Writing WDF Drivers TBD Live, Online
Developing Minifilters 7 February 2022 Live, Online
Kernel Debugging 21 March 2022 Live, Online