understanding alternate data streams

NTFSD
1

Hi all,

I am trying to understand what I need to do in my file system in order to
support alternate data streams. I have searched through the archive of this
list and found some good references, the best being:

http://www.osronline.com/showThread.cfm?link=62574
http://www.osronline.com/showThread.cfm?link=24948

I have found other information here and there in other threads and in
articles on osronline. I still have some questions though.

Is it valid to send a QUERY_INFO IRP to an ADS FO? If so, should the FSD
return attributes and dates and number-of-links and so on that correspond to
the file, or should those be returned as 0/-1/invalid? The file size and
allocation size should be that of the stream I guess?

I know it is valid to send a SET_INFO to an ADS FO, because that is how to
rename a stream. If the SET_INFO is used to set other information, like
dates or file attributes, is that an error or is the file itself updated? I
suppose its valid to set the file size/allocation size/position of the
stream?

Is it valid to send SET_SECURITY/QUERY_SECURITY IRPs to an ADS FO?

Is it valid to send oplock fsctl IRPs to an ADS FO? Wrt renaming
directories, I suppose the rules for breaking batch oplocks and filter
oplocks are the same if ADS FOs are open beneath this directory?

[Tony: you said a little over a month ago that you would write a general
article on alternate data streams since this seems to be an obscure topic.
Maybe you can bounce some of your ideas here before publishing :slight_smile: ]

Thanks.

=================================================
Roger Tawa
http://tawacentral.net/
[One thing about paradigms: shift happens.]
[When you stop, you’re done.]

2

You will have to reverse engineer the behvaiour of ntfs I think. Some points
of note inline below. Its a hard old world.

“Roger Tawa” wrote in message news:xxxxx@ntfsd…
> Hi all,
>
> I am trying to understand what I need to do in my file system in order to
> support alternate data streams. I have searched through the archive of
> this
> list and found some good references, the best being:
>
> http://www.osronline.com/showThread.cfm?link=62574
> http://www.osronline.com/showThread.cfm?link=24948
>
> I have found other information here and there in other threads and in
> articles on osronline. I still have some questions though.
>
> Is it valid to send a QUERY_INFO IRP to an ADS FO?

Yes of course this is valid.

> If so, should the FSD
> return attributes and dates and number-of-links and so on that correspond
> to
> the file, or should those be returned as 0/-1/invalid? The file size and
> allocation size should be that of the stream I guess?

Its a mess. You return the file times of the base object (file or
directory). The attributes you return are a merge of those of the base
object and the precious few which belong to the stream iteself (compression,
sparse). The file size and allocation size are obvious.

>
> I know it is valid to send a SET_INFO to an ADS FO, because that is how to
> rename a stream. If the SET_INFO is used to set other information, like
> dates or file attributes, is that an error or is the file itself updated?
> I
> suppose its valid to set the file size/allocation size/position of the
> stream?

I think the precise behaviour just depends on the particular info. You will
have to reverse ntfs here.

>
> Is it valid to send SET_SECURITY/QUERY_SECURITY IRPs to an ADS FO?

It seems not.

>
> Is it valid to send oplock fsctl IRPs to an ADS FO? Wrt renaming
> directories, I suppose the rules for breaking batch oplocks and filter
> oplocks are the same if ADS FOs are open beneath this directory?

Not a clue.

>
> [Tony: you said a little over a month ago that you would write a general
> article on alternate data streams since this seems to be an obscure topic.
> Maybe you can bounce some of your ideas here before publishing :-)]
>
> Thanks.
>
> =================================================
> Roger Tawa
> http://tawacentral.net/
> [One thing about paradigms: shift happens.]
> [When you stop, you’re done.]
>
>
>
>

3

IRP_MJ_{QUERY,SET}_INFORMATION calls are done against the file normally
(at least with NTFS) and anything that is specific to the stream. This
means that if you open an ADS and then issue a call, you get the info
for the file in most cases, combined (or overridden) with any specific
information for the stream.

In addition, we have found some rather strange semantics around streams
(which are a feature provided by our DMK for file systems that do not
support streams, such as FAT so we have a set of tests for validating we
mimic the odd behavior exhibited by NTFS.) For example, if you open an
ADS and then try to open the default stream for FILE_SUPERSEDE the call
fails with STATUS_SHARING_VIOLATION, but if you open an ADS and then try
to open the default data stream for FILE_OVERWRITE the calls succeeds.
When you close the ADS, NTFS then deletes the stream for you.

This is interesting when you implement things differently than NTFS.
For example, our DMK allows you to selectively encrypt streams (and
using different keys.) Thus, you might indicate
FILE_ATTRIBUTE_ENCRYPTED for those streams that are encrypted,
FILE_ATTRIBUTE_SPARSE for the streams that are sparse,
FILE_ATTRIBUTE_COMPRESSED for a compressed stream, and neither one for
the streams that have neither of these present. But if the *file* is
marked FILE_ATTRIBUTE_SYSTEM then we’d present this attribute regardless
of the stream being queried.

Timestamps (for example) are (in both NTFS and our DMK implementation)
per file attributes, so these are returned regardless of which stream is
used.

Testing this is quite a lot of fun - the IFS Kit tests are a tad light
on stream tests which means you’ll need to either construct your own or
work with someone who already has such tests.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

4

Thanks Lyndon, Tony.

One more question: in one of the threads (62574), several people said
that before implementing support for ADS, an FSD must first support
“open by file id”. I have not implemented “open by file id” because
64-bits is not enough to uniquely identity a file in my file system
(at least not generally, I can probably support the most important
subset of files with a 64-bit id by making assumptions for the other
bits).

Assuming I support “open by file id” for a subset of files, does this
mean ADS will only be supported by this subset? Thanks.

=================================================
Roger Tawa
http://tawacentral.net/
[One thing about paradigms: shift happens.]
[When you stop, you’re done.]

5

Roger,

I know of no reason why you must support open by ID to support ADS. If
you choose to support file IDs, keep in mind they are only unique per
volume - now maybe you just present one very large volume.

Not supporting file IDs means that you will observe problems (now) when
someone runs Services for Unix (SFU) on a machine with your file system.
Since SFU functionality is expected to ship in Vista, you’ll have to
deal with this issue at some point (the file ID is used as part of the
NFS file handle).

Note that object IDs can also be supported - these are 128 bits (UUIDs).

Supporting file IDs has as many odd semantics as supporting ADS (the
interactions between hard links, file IDs, rename, and deletion can
drive one to distraction!)

Best of luck.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Roger Tawa
Sent: Saturday, April 29, 2006 10:49 AM
To: ntfsd redirect
Subject: Re: [ntfsd] understanding alternate data streams

Thanks Lyndon, Tony.

One more question: in one of the threads (62574), several people said
that before implementing support for ADS, an FSD must first support
“open by file id”. I have not implemented “open by file id” because
64-bits is not enough to uniquely identity a file in my file system
(at least not generally, I can probably support the most important
subset of files with a 64-bit id by making assumptions for the other
bits).

Assuming I support “open by file id” for a subset of files, does this
mean ADS will only be supported by this subset? Thanks.

=================================================
Roger Tawa
http://tawacentral.net/
[One thing about paradigms: shift happens.]
[When you stop, you’re done.]

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com