I’ve been searching for an excellent post on how to uniquely identify a file on an NTFS volume that I saw awhile back but have been unable to find it. IIRC, it was posted by Tony Mason regarding why neither filenames nor file objects were good identifiers, but there was some sort of index number on the volume itself. I found some terse documentation in the WDK about NTFS index numbers, but the NTFSD posting had more information about the how and why of file tracking. Does this ring a bell for anybody?
Thanks,
Tony Bock
You are looking for NTFS file IDs.
Look at: http://msdn.microsoft.com/en-us/library/ms791535.aspx
FileInternalInformation and FILE_INTERNAL_INFORMATION
typedef struct _FILE_INTERNAL_INFORMATION {
LARGE_INTEGER IndexNumber;
} FILE_INTERNAL_INFORMATION, *PFILE_INTERNAL_INFORMATION;
There are also some other info classes that return this same information.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
Bock, Tony wrote:
I’ve been searching for an excellent post on how to uniquely identify a
file on an NTFS volume that I saw awhile back but have been unable to
find it. IIRC, it was posted by Tony Mason regarding why neither
filenames nor file objects were good identifiers, but there was some
sort of index number on the volume itself. I found some terse
documentation in the WDK about NTFS index numbers, but the NTFSD posting
had more information about the how and why of file tracking. Does this
ring a bell for anybody?
Thanks,
Tony Bock
Tony
Like Saroshh said, you want the ntfs internal file identifier,
FileInternalInformation. Do think about named data streams. The ntfs file
identifiers are unique per file/directory and not per stream. I know of no
unique identifiers for streams; you might not care, or you might be able to
use some combination of the ntfs file identifier and the stream name.
Cheers, Lyndon
“Bock, Tony” wrote in message news:xxxxx@ntfsd…
I’ve been searching for an excellent post on how to uniquely identify a file
on an NTFS volume that I saw awhile back but have been unable to find it.
IIRC, it was posted by Tony Mason regarding why neither filenames nor file
objects were good identifiers, but there was some sort of index number on
the volume itself. I found some terse documentation in the WDK about NTFS
index numbers, but the NTFSD posting had more information about the how and
why of file tracking. Does this ring a bell for anybody?
Thanks,
Tony Bock
Thanks for the replies everybody. Do you know if these identifiers are persistent across moves, defrags, etc.? At least on NTFS, as we aren’t targeting FAT.
Thanks,
Tony Bock
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Wednesday, January 14, 2009 2:39 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Unique ID for files on NTFS?
Tony
Like Saroshh said, you want the ntfs internal file identifier,
FileInternalInformation. Do think about named data streams. The ntfs file
identifiers are unique per file/directory and not per stream. I know of no
unique identifiers for streams; you might not care, or you might be able to
use some combination of the ntfs file identifier and the stream name.
Cheers, Lyndon
“Bock, Tony” wrote in message news:xxxxx@ntfsd…
I’ve been searching for an excellent post on how to uniquely identify a file
on an NTFS volume that I saw awhile back but have been unable to find it.
IIRC, it was posted by Tony Mason regarding why neither filenames nor file
objects were good identifiers, but there was some sort of index number on
the volume itself. I found some terse documentation in the WDK about NTFS
index numbers, but the NTFSD posting had more information about the how and
why of file tracking. Does this ring a bell for anybody?
Thanks,
Tony Bock
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: xxxxx@intel.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
The file ID is a property of the instance of file on a specific volume.
File IDs do not change unless the file is deleted. So they are unchanged
on move within the same volume and defrag.
If the file is moved to a different volume the file ID will different at
the destination. If the file is deleted and restored from backup the
file ID will change.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
Bock, Tony wrote:
Thanks for the replies everybody. Do you know if these identifiers are persistent across moves, defrags, etc.? At least on NTFS, as we aren’t targeting FAT.
Thanks,
Tony Bock
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Wednesday, January 14, 2009 2:39 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Unique ID for files on NTFS?
Tony
Like Saroshh said, you want the ntfs internal file identifier,
FileInternalInformation. Do think about named data streams. The ntfs file
identifiers are unique per file/directory and not per stream. I know of no
unique identifiers for streams; you might not care, or you might be able to
use some combination of the ntfs file identifier and the stream name.
Cheers, Lyndon
“Bock, Tony” wrote in message news:xxxxx@ntfsd…
> I’ve been searching for an excellent post on how to uniquely identify a file
> on an NTFS volume that I saw awhile back but have been unable to find it.
> IIRC, it was posted by Tony Mason regarding why neither filenames nor file
> objects were good identifiers, but there was some sort of index number on
> the volume itself. I found some terse documentation in the WDK about NTFS
> index numbers, but the NTFSD posting had more information about the how and
> why of file tracking. Does this ring a bell for anybody?
>
> Thanks,
>
> Tony Bock
>
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as: xxxxx@intel.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
Thank you very much. I think that should work for our puroses.
Regards,
Tony
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Sarosh Havewala
Sent: Wednesday, January 14, 2009 3:00 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Unique ID for files on NTFS?
The file ID is a property of the instance of file on a specific volume.
File IDs do not change unless the file is deleted. So they are unchanged
on move within the same volume and defrag.
If the file is moved to a different volume the file ID will different at
the destination. If the file is deleted and restored from backup the
file ID will change.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
Bock, Tony wrote:
Thanks for the replies everybody. Do you know if these identifiers are persistent across moves, defrags, etc.? At least on NTFS, as we aren’t targeting FAT.
Thanks,
Tony Bock
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Wednesday, January 14, 2009 2:39 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Unique ID for files on NTFS?
Tony
Like Saroshh said, you want the ntfs internal file identifier,
FileInternalInformation. Do think about named data streams. The ntfs file
identifiers are unique per file/directory and not per stream. I know of no
unique identifiers for streams; you might not care, or you might be able to
use some combination of the ntfs file identifier and the stream name.
Cheers, Lyndon
“Bock, Tony” wrote in message news:xxxxx@ntfsd…
> I’ve been searching for an excellent post on how to uniquely identify a file
> on an NTFS volume that I saw awhile back but have been unable to find it.
> IIRC, it was posted by Tony Mason regarding why neither filenames nor file
> objects were good identifiers, but there was some sort of index number on
> the volume itself. I found some terse documentation in the WDK about NTFS
> index numbers, but the NTFSD posting had more information about the how and
> why of file tracking. Does this ring a bell for anybody?
>
> Thanks,
>
> Tony Bock
>
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as: xxxxx@intel.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: xxxxx@intel.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
> File IDs do not change unless the file is deleted. So they are unchanged
on move within the same volume and defrag.
Hey Sarosh,
Is there a repository of these little nuggets of NTFS information (which is
explicitly contradictory to the link you posted!). One could (and indeed I
have) made that assumption based on a best guess on how one would implement
them if one was writing NTFS, but having that information written down (even
in a note like yours above) makes it easier to sleep at night…
As an aside - If a file is deleted because the transaction it was created in
was rolled back is the FID less or more likely to crop up again than if the
file has just been deleted in the old fashioned way? (Again, assuming an
implementaion: does transactional rollback increment the sequence count part
of the FID or not). I realise that if one was working in a world where TxFs
was built in from day one you could (and should) just forget that you ever
knew that FID as soon as you get the rollback, but sometimes you don’t have
that liberty when TxFs-ifying an existing filter…
/Rod
Rod,
Just to clarify, the little nuggets of information are true only on NTFS
whereas the doc is talking about file IDs in general over all the file
systems. Hence the apparent contradiction.
I agree that it would be useful to have a repository of such
information. Maybe I should start blogging in my copious free time. 
The NTFS file ID should remain unchanged if the Tx Delete is rolled back.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
Rod Widdowson wrote:
> File IDs do not change unless the file is deleted. So they are unchanged
> on move within the same volume and defrag.
Hey Sarosh,
Is there a repository of these little nuggets of NTFS information (which is
explicitly contradictory to the link you posted!). One could (and indeed I
have) made that assumption based on a best guess on how one would implement
them if one was writing NTFS, but having that information written down (even
in a note like yours above) makes it easier to sleep at night…
As an aside - If a file is deleted because the transaction it was created in
was rolled back is the FID less or more likely to crop up again than if the
file has just been deleted in the old fashioned way? (Again, assuming an
implementaion: does transactional rollback increment the sequence count part
of the FID or not). I realise that if one was working in a world where TxFs
was built in from day one you could (and should) just forget that you ever
knew that FID as soon as you get the rollback, but sometimes you don’t have
that liberty when TxFs-ifying an existing filter…
/Rod
Sarosh,
Are you sure it is ONLY true on NTFS? I would assume this is actually
true for any file system that supports NFS (since NFS *uses* those
handles at a later point in time, it’s important that they remain
valid.)
I do know that FAT’s implementation changes them (during rename, as I
recall.) But I know they can’t change on CDFS (which supported SFM, the
original motivation for the “open by file ID” case) and I’d be terribly
surprised if the UDFS implementation changes them. EXFAT probably went
either way, but I’d guess it uses a FAT-like algorithm.
Thus, my question to you: are you saying “this only works for NTFS”
because you KNOW it does not work on these others, or because you KNOW
it works on NTFS, does not work on FAT and assume that it doesn’t work
for the other file systems as well.
Tony
OSR
Good point. I guess I need to qualify my response …
My response was specific to ntfs since that was the file system that
Tony Bock (originator of the thread) was interested in.
I know it does not work for FAT and used that as a data point for what
the MSDN doc was referring to.
I just did not bother to look up or comment on any of the other file
systems. Not that they are any less important, just that, did not think
it was pertinent to the thread.
Regards,
Sarosh.
File System Filter Lead
Microsoft Corp
This posting is provided “AS IS” with no warranties, and confers no Rights
Tony Mason wrote:
Sarosh,
Are you sure it is ONLY true on NTFS? I would assume this is actually
true for any file system that supports NFS (since NFS *uses* those
handles at a later point in time, it’s important that they remain
valid.)
I do know that FAT’s implementation changes them (during rename, as I
recall.) But I know they can’t change on CDFS (which supported SFM, the
original motivation for the “open by file ID” case) and I’d be terribly
surprised if the UDFS implementation changes them. EXFAT probably went
either way, but I’d guess it uses a FAT-like algorithm.
Thus, my question to you: are you saying “this only works for NTFS”
because you KNOW it does not work on these others, or because you KNOW
it works on NTFS, does not work on FAT and assume that it doesn’t work
for the other file systems as well.
Tony
OSR
Sarosh,
In fact, like you, I noted the original query was against NTFS and
refrained from following up initially; but subsequent discussion was
more general which is why I thought it worthy of clarification. So,
consider that this is “for the archive” so that some poor soul that
looks this up in 2 days or a month or a year or a decade understands the
context of the conversation.
So, to summarize: “the scope and lifetime of the File ID is determined
by the file system. A filter driver should not assume a specific model
of behavior for an arbitrary file system.”
Further, I would note that the file ID returned by RDR may not have any
relationship to the file ID of the file on the file server (again, I
won’t commit to specific behavior here, but I do know that at least some
versions of the CIFS/SMB/RDR implementation “make up” the file ID.)
The original purpose of the file ID was to (effectively) implement the
“open by inode number” functionality that had become common on UNIX
systems to support stateless file systems (e.g., SFM and NFS.) File
systems that supported (or do support) this functionality are more
likely to provide persistent file IDs, while file systems that do not
support this functionality are less likely to provide persistent file
IDs.
Tony
OSR
My understanding (and I would hope to be corrected) that there wasnt a unique id for the lifetime of a created file on NTFS is that true?
Much appreciated,
Garyc
— On Fri, 1/16/09, Sarosh Havewala wrote:
> From: Sarosh Havewala
> Subject: Re:[ntfsd] Unique ID for files on NTFS?
> To: “Windows File Systems Devs Interest List”
> Date: Friday, January 16, 2009, 10:58 PM
> Good point. I guess I need to qualify my response …
>
> My response was specific to ntfs since that was the file
> system that Tony Bock (originator of the thread) was
> interested in.
>
> I know it does not work for FAT and used that as a data
> point for what the MSDN doc was referring to.
>
> I just did not bother to look up or comment on any of the
> other file systems. Not that they are any less important,
> just that, did not think it was pertinent to the thread.
>
> Regards,
> Sarosh.
> File System Filter Lead
> Microsoft Corp
>
> This posting is provided “AS IS” with no
> warranties, and confers no Rights
>
>
> Tony Mason wrote:
> > Sarosh,
> >
> > Are you sure it is ONLY true on NTFS? I would assume
> this is actually
> > true for any file system that supports NFS (since NFS
> uses those
> > handles at a later point in time, it’s important
> that they remain
> > valid.)
> >
> > I do know that FAT’s implementation changes them
> (during rename, as I
> > recall.) But I know they can’t change on CDFS
> (which supported SFM, the
> > original motivation for the “open by file
> ID” case) and I’d be terribly
> > surprised if the UDFS implementation changes them.
> EXFAT probably went
> > either way, but I’d guess it uses a FAT-like
> algorithm.
> >
> > Thus, my question to you: are you saying “this
> only works for NTFS”
> > because you KNOW it does not work on these others, or
> because you KNOW
> > it works on NTFS, does not work on FAT and assume that
> it doesn’t work
> > for the other file systems as well.
> >
> > Tony
> > OSR
> >
> >
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as:
> xxxxx@yahoo.com
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
As Sarosh pointed out:
The file ID is a property of the instance of file on a specific volume.
File IDs do not change unless the file is deleted. So they are unchanged on
move within the same volume and defrag.
If the file is moved to a different volume the file ID will different at
the destination. If the file is deleted and restored from backup the file ID
will change.
Can’t give you a more clearer answer.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of gary clark
Sent: Sunday, January 18, 2009 5:44 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Unique ID for files on NTFS?
My understanding (and I would hope to be corrected) that there wasnt a
unique id for the lifetime of a created file on NTFS is that true?
Much appreciated,
Garyc
— On Fri, 1/16/09, Sarosh Havewala wrote:
> From: Sarosh Havewala
> Subject: Re:[ntfsd] Unique ID for files on NTFS?
> To: “Windows File Systems Devs Interest List”
> Date: Friday, January 16, 2009, 10:58 PM
> Good point. I guess I need to qualify my response …
>
> My response was specific to ntfs since that was the file
> system that Tony Bock (originator of the thread) was
> interested in.
>
> I know it does not work for FAT and used that as a data
> point for what the MSDN doc was referring to.
>
> I just did not bother to look up or comment on any of the
> other file systems. Not that they are any less important,
> just that, did not think it was pertinent to the thread.
>
> Regards,
> Sarosh.
> File System Filter Lead
> Microsoft Corp
>
> This posting is provided “AS IS” with no
> warranties, and confers no Rights
>
>
> Tony Mason wrote:
> > Sarosh,
> >
> > Are you sure it is ONLY true on NTFS? I would assume
> this is actually
> > true for any file system that supports NFS (since NFS
> uses those
> > handles at a later point in time, it’s important
> that they remain
> > valid.)
> >
> > I do know that FAT’s implementation changes them
> (during rename, as I
> > recall.) But I know they can’t change on CDFS
> (which supported SFM, the
> > original motivation for the “open by file
> ID” case) and I’d be terribly
> > surprised if the UDFS implementation changes them.
> EXFAT probably went
> > either way, but I’d guess it uses a FAT-like
> algorithm.
> >
> > Thus, my question to you: are you saying “this
> only works for NTFS”
> > because you KNOW it does not work on these others, or
> because you KNOW
> > it works on NTFS, does not work on FAT and assume that
> it doesn’t work
> > for the other file systems as well.
> >
> > Tony
> > OSR
> >
> >
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as:
> xxxxx@yahoo.com
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: xxxxx@gmail.com
To unsubscribe send a blank email to xxxxx@lists.osr.com