Transactional NTFS future - the real story

Hello,

I have developed an NTFS minifilter driver that I am considering enhancing to take account of Transactional NTFS (TxF) functionality.

It seems to me, however, that every piece of Microsoft documentation for the TxF APIs include a disclaimer similar to the following:

“Microsoft strongly recommends developers utilize alternative means to achieve your application?s needs. Many scenarios that TxF was developed for can be achieved through simpler and more readily available techniques. Furthermore, TxF may not be available in future versions of Microsoft Windows. For more information, and alternatives to TxF, please see Alternatives to using Transactional NTFS.”

These disclaimers appear even more emphatic in the Windows 8 preliminary documentation.

Does anyone have any insight into what the real story is concerning TxF futures?
Do any Microsoft or major 3rd party applications actually use it?

Any information or insights are extremely welcome.

Thanks in advance,
David Gilbert

I have seen it used by Microsoft. I don’t remember the particulars. I saw it
when I had a crash in my transaction handling. It was probably in W2K8 R2.

Bill Wandel

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Thursday, August 09, 2012 1:23 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Transactional NTFS future - the real story

Hello,

I have developed an NTFS minifilter driver that I am considering enhancing
to take account of Transactional NTFS (TxF) functionality.

It seems to me, however, that every piece of Microsoft documentation for the
TxF APIs include a disclaimer similar to the following:

“Microsoft strongly recommends developers utilize alternative means to
achieve your application?s needs. Many scenarios that TxF was developed for
can be achieved through simpler and more readily available techniques.
Furthermore, TxF may not be available in future versions of Microsoft
Windows. For more information, and alternatives to TxF, please see
Alternatives to using Transactional NTFS.”

These disclaimers appear even more emphatic in the Windows 8 preliminary
documentation.

Does anyone have any insight into what the real story is concerning TxF
futures?
Do any Microsoft or major 3rd party applications actually use it?

Any information or insights are extremely welcome.

Thanks in advance,
David Gilbert


NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

>I have seen it used by Microsoft. I don’t remember the particulars.

Windows Update uses it.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Transactions can greatly complicate filters. For example, in isolation filters, you must add special handling for them, because you have to distinguish opens inside from outside a transaction. When the transaction commits, you need to close any shadow file objects you maintain because otherwise they can create subsequent problems.

For example, if someone opens up a directory inside a transaction, you need to distinguish that from an open of the directory outside the transaction. Then, if they do a relative open (using the directory - or even a file in the case of streams) relative to that initial transaction you need to ensure they are within the same transaction (although it does seem to work if you open the directory OUTSIDE the transaction but then do the relative open INSIDE a transaction.)

But the usual tricks of trying to keep file objects around becomes problematic - because if you have the directory open inside a transaction and someone tries to open it from a second transaction, you will observe a transactional conflict.

Ideally, the best policy is really to try and avoid transactions if at all possible. But if you MUST deal with them, be prepared to do quite a bit of work.

Only NTFS currently supports transactions. It is POSSIBLE to build other transactional file systems, but as far as I know, no one has done that, nor has anyone suggested doing so to me - perhaps others in this space have had different experiences.

Tony
OSR

I understand the nature and operation of transactions in TxF and their relationship to the Kernel Transaction Manager.

What I am looking for is some insight into the future of Transactional NTFS. There is a Microsoft disclaimer on every MSDN page that describes a TxF API that discourages developers from using the API and warns about the disappearance of TxF in future versions of Windows.

The wording of this disclaimer and its ubiquitous presence leads me to believe that TxF was never used by any major 3rd party products and that its demise is imminent. If anyone close to the gang in Redmond has any information that can be shared I would be very grateful.

Thanks and regards,
David Gilbert

Well, you won’t get a commitment from Microsoft one way or the other on TxF is what I suspect, because the general rules of engagement with the outside development community are quite clear on not discussing future product plans with same.

I’ve been watching the TxF evolution for 10+ years now and I must admit, I’ve always tried to understand who was truly going to use it. The only real-world use that I’ve seen for it is in implementing TxR and installers.

The most telling thing to me is that ReFS does not support transactions. That suggests to me this is a feature that does not have sufficient demand to justify expending the resources to build TxF support into ReFS. In some ways, ReFS allows Microsoft to focus on those file system features that are useful and to drop support for those features that aren’t (and yet cost tremendous resources to maintain).

I have not noticed the warning message you’ve described on the MSDN documentation pages I’ve viewed regarding transactions. For example: http://msdn.microsoft.com/en-us/library/windows/hardware/ff566422(v=vs.85).aspx describes ZwCreateEnlistment, and I don’t see any caveats to this saying that this API will not be supported in future OS releases.

Tony
OSR

Tony,

Thanks for the response. I have been looking at other blogs and have come to the conclusion that TxF was never really used because it was too restrictive (no SMB support) and too complicated.

Concerning the MSDN caveat that I mentioned, it is on all the user-level TxF API pages (CreateFileTransacted, DeleteFileTransacted,…). Looking at ZwCreateTransaction, ZwCreateEnlistment, and other kernel level APIs, it appears that they refer to KTM transactions of which TxF transactions are a subset. Am I near the truth or way off the reservation?

David Gilbert

KTM and TxF are two parts of an entire mechanism. KTM provides the “transaction manager” part and TxF provides the “resource manager” part.

When you call CreateTransaction() in Win32, this ends up being a call to NtCreateTransaction (== ZwCreateTransaction). Then you take the handle to the transaction you just created and feed it to CreateFileTransacted(), DeleteFileTransacted(), etc. This causes NTFS (of which TxF is part) to enlist on the transaction. Once that enlistment is done, what happens to the file is bound to the lifetime of the transaction, as managed by KTM. So if you call CommitTransaction() that tells KTM to commit the transaction. He in turn sends messages to everything that has enlisted on that transaction (such as NTFS/TxF, TxR, filters that have called FltEnlistInTransaction, etc.) to commit their work. Similarly, calling RollbackTransaction() or closing your handle to the transaction tells KTM to abort the transaction, so he tells all enlistments to roll back their work.

Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.