Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting... Please check out the Community Guidelines in the
Announcements and Administration Category.

ReFS and 128 bit file ID

I have requirement to support 128 bit file ID's for ReFS volumes. I am calling FltQueryInformationFile once with FILE_INTERNAL_INFORMATION and again with FILE_ID_INFORMATION to retrieve both the 64 and 128 bit file ID's. I thought the 64 bit file ID would be useless but to my surprise I was able to open the file using FileTest. The 128 bit one works too.

I guess the first question is, how come the 64 bit file ID works? The second question, is the 128 bit one still needed?

Comments

  • I am assuming the 64 bit one works for backwards compatibility. ReFs in
    many ways is still similar to NTFS. Imagine how many apps and drivers out
    there may do things based on the 64 bit file id, but, and this is a crucial
    but without checking the file system on the volume.
    I may of course be wrong and there would be a deeper reason than this, for
    example some part of the ReFs implementation which is NTFS with some new
    additions depended on this 64 bit file id and they could not remove it yet.

    Now again, looking at how you approach this, you seem to be tending to use
    the 64 bit file even if the newer 128 one is there. It is up to you at the
    end of the day. I personally use the 128 on volumes where that is supported
    and 64 bit one in the rest of the cases.

    Cheers,
    Gabriel
    www.kasardia.com

    On Sat, Jan 27, 2018 at 12:34 AM, xxxxx@hotmail.com
    wrote:

    > I have requirement to support 128 bit file ID's for ReFS volumes. I am
    > calling FltQueryInformationFile once with FILE_INTERNAL_INFORMATION and
    > again with FILE_ID_INFORMATION to retrieve both the 64 and 128 bit file
    > ID's. I thought the 64 bit file ID would be useless but to my surprise I
    > was able to open the file using FileTest. The 128 bit one works too.
    >
    > I guess the first question is, how come the 64 bit file ID works? The
    > second question, is the 128 bit one still needed?
    >
    >
    > ---
    > NTFSD is sponsored by OSR
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >



    --
    Bercea. G.

    Cheers,
    Gabriel

  • The 64 bit one is indeed for compatibility.

    NTFS has a single, 48 bit volume wide file index, plus a 16 bit sequence
    number for that index, in a 64 bit value. This means that it is willing
    to reuse file identifiers, and have collisions.

    ReFS has a directory identifier and a file identifier in its file ID,
    and these are monotonically increasing and will never be reused for the
    lifetime of the volume. Unfortunately 64 bits volume wide isn't really
    enough to ensure that a production volume will never "run out" of
    identifiers, either in terms of directories-per-volume or
    files-per-directory. 64 bit support is there to provide limited
    compatibility until software can be reworked to use larger identifiers,
    but if the goal is to run software on a real ReFS volume, using 128 bit
    identifiers is a very good idea.

    As has been noted, 128 bit identifiers also now work on NTFS.
    Unfortunately most real world software still needs 64 bit identifier
    support because it needs to run on older version of Windows where this
    support is not present. IIRC NTFS has 128 bit ID support in 2012
    R2/8.1, so at some point 64 bit ID support can be retired altogether.

    - M

    On 01/26/2018 11:57 PM, Gabriel Bercea wrote:
    > I am assuming the 64 bit one works for backwards compatibility. ReFs in
    > many ways is still similar to NTFS. Imagine how many apps and drivers
    > out there may do things based on the 64 bit file id, but, and this is a
    > crucial but without checking the file system on the volume.
    > I may of course be wrong and there would be a deeper reason than this,
    > for example some part of the ReFs implementation which is NTFS with some
    > new additions depended on this 64 bit file id and they could not remove
    > it yet.
    >
    > Now again, looking at how you approach this, you seem to be tending to
    > use the 64 bit file even if the newer 128 one is there. It is up to you
    > at the end of the day. I personally use the 128 on volumes where that is
    > supported and 64 bit one in the rest of the cases.
    >
    > Cheers,
    > Gabriel
    > www.kasardia.com <http://www.kasardia.com>;
    >
    > On Sat, Jan 27, 2018 at 12:34 AM, xxxxx@hotmail.com
    > <mailto:xxxxx@hotmail.com> <mailto:xxxxx@lists.osr.com>> wrote:
    >
    > I have requirement to support 128 bit file ID's for ReFS volumes.  I
    > am calling FltQueryInformationFile once with
    > FILE_INTERNAL_INFORMATION and again with FILE_ID_INFORMATION to
    > retrieve both the 64 and 128 bit file ID's.  I thought the 64 bit
    > file ID would be useless but to my surprise I was able to open the
    > file using FileTest.  The 128 bit one works too.
    >
    > I guess the first question is, how come the 64 bit file ID works?
    > The second question, is the 128 bit one still needed?
    >
    >
    > ---
    > NTFSD is sponsored by OSR
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at <http://www.osr.com/seminars>;
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > <http://www.osronline.com/page.cfm?name=ListServer>>;
    >
    >
    >
    >
    > --
    > Bercea. G.


    --
    http://www.malsmith.net
  • If you reach the limit of 64 bits on an ReFS volume will the OS start to fail FltQueryInformationFile or will it start recycling the old values? The latter would be very dangerous since it would lead to opening the wrong file.
  • I believe you will not be allowed to create any more files/directories.
    How can you recycle values if all the values are occupied ?

    On Mon, Jan 29, 2018 at 9:47 PM, xxxxx@hotmail.com
    wrote:

    > If you reach the limit of 64 bits on an ReFS volume will the OS start to
    > fail FltQueryInformationFile or will it start recycling the old values?
    > The latter would be very dangerous since it would lead to opening the wrong
    > file.
    >
    > ---
    > NTFSD is sponsored by OSR
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >



    --
    Bercea. G.

    Cheers,
    Gabriel

  • On 01/29/2018 12:47 PM, xxxxx@hotmail.com wrote:
    > If you reach the limit of 64 bits on an ReFS volume will the OS start to fail FltQueryInformationFile or will it start recycling the old values? The latter would be very dangerous since it would lead to opening the wrong file.
    >

    IIRC it will start returning zero for everything, since zero is an
    invalid ID on ReFS. That means there's no concern about opening the
    wrong file, but any code depending on 64 bit file IDs will still fail
    catastrophically.

    The reason the 128 bit APIs are there is to enable it to continue
    creating new files, and have a path for software to work. Failing new
    file creation when the 64 bit limit is reached would prematurely brick
    volumes - if new file creation were failed at the end of the 64 bit
    limit, having a 128 bit API would be pointless.

    - M

    --
    http://www.malsmith.net
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!