streams - why should I do this?

NTFSD
21

Sorry, this is not FSCTL. This is
MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in CDFS. This
thing enumerates all attribute streams in the file.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Greg Pearce” wrote in message
news:…
> Hey Max! You’ve just hit on some information that I would like to find out
> more about; i.e. what FSCTLs are required for ADS? Where can I find this
> kind of stuff documented?
>
> Thanks,
>
> Greg

22

Thanks Max… that clears that up.

Tony said that I must be missing something still though.

In this attempt to support ADS in my FSD, when my FSD gets an
IRP_MJ_QUERY_VOLUME_INFORMATION, I turn on FILE_NAMED_STREAMS in the
filesystem attributes field of the FILE_FS_ATTRIBUTE_INFORMATION structure,
and set the FileSystemName field to “NTFS”, and explorer stops whining about
losing the ADS information. I never get warned that I would lose ADS info
when doing a copy from the command prompt. I still need to put some meat in
there to parse the name correctly and some other things, but it looks like
it would work if all that code were in place.

What am I missing, besides MJ_QUERY_INFORMATION/FileStreamInformation
support?

Getting back to a previous question that I had, is it a problem legally to
tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t support
all the stuff NTFS does? It seems like Explorer should have a better way to
determine if a file system supports ADS instead of relying on the file
system’s internal “name”. Anyone care to comment? Anyone from Microsoft?
It is greatly appreciated!

Thanks again and again

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> Sorry, this is not FSCTL. This is
> MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in CDFS.
This
> thing enumerates all attribute streams in the file.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Greg Pearce” wrote in message
> news:…
> > Hey Max! You’ve just hit on some information that I would like to find
out
> > more about; i.e. what FSCTLs are required for ADS? Where can I find
this
> > kind of stuff documented?
> >
> > Thanks,
> >
> > Greg
>
>

23

> Getting back to a previous question that I had, is it a problem legally to

tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t support
all the stuff NTFS does?

In this case, the apps will send some NTFS-specific undocumented stuff to your
FSD. For instance, the NTFS defragmenter will possibly be invoked for your
volume :slight_smile:

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

24

Cool! :wink:

I’m going to try that just to see what happens when it tries to re-org a
tape or optical disk (which is where my FSD stores things)… heh heh!
That’ll be interesting…

Thanks for your help!

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> > Getting back to a previous question that I had, is it a problem legally
to
> > tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t
support
> > all the stuff NTFS does?
>
> In this case, the apps will send some NTFS-specific undocumented stuff to
your
> FSD. For instance, the NTFS defragmenter will possibly be invoked for your
> volume :slight_smile:
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>

25

Greg,

I am seeing if I can track someone down from the shell team who can let
me know what is going on. What OS versions have you tried this on?

Sometimes it is a mystery to the file system group as to why the shell
does what it does.

It is probably a bad idea to have your file system identify itself as
NTFS.

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Greg Pearce
Sent: Monday, July 12, 2004 11:18 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:streams - why should I do this?

Thanks Max… that clears that up.

Tony said that I must be missing something still though.

In this attempt to support ADS in my FSD, when my FSD gets an
IRP_MJ_QUERY_VOLUME_INFORMATION, I turn on FILE_NAMED_STREAMS in the
filesystem attributes field of the FILE_FS_ATTRIBUTE_INFORMATION
structure,
and set the FileSystemName field to “NTFS”, and explorer stops whining
about
losing the ADS information. I never get warned that I would lose ADS
info
when doing a copy from the command prompt. I still need to put some
meat in
there to parse the name correctly and some other things, but it looks
like
it would work if all that code were in place.

What am I missing, besides MJ_QUERY_INFORMATION/FileStreamInformation
support?

Getting back to a previous question that I had, is it a problem legally
to
tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t
support
all the stuff NTFS does? It seems like Explorer should have a better
way to
determine if a file system supports ADS instead of relying on the file
system’s internal “name”. Anyone care to comment? Anyone from
Microsoft?
It is greatly appreciated!

Thanks again and again

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> Sorry, this is not FSCTL. This is
> MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in
CDFS.
This
> thing enumerates all attribute streams in the file.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Greg Pearce” wrote in message
> news:…
> > Hey Max! You’ve just hit on some information that I would like to
find
out
> > more about; i.e. what FSCTLs are required for ADS? Where can I find
this
> > kind of stuff documented?
> >
> > Thanks,
> >
> > Greg
>
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

26

> > MSS> ADS are NTFS-only.

>
> Is this supposed to mean that only NTFS has this feature or
this means
> that

Only NTFS has the ADS in its on-disk format.
Neither FAT12/16/32 nor any UNIX FS have them.

IBM’s Unix (AIX) has alternate data streams. It has been a few years
since I’ve been in that code, I believe they are called ‘Extensions’.

As about the network redirs - MRXSMB will support them, if
the server is sharing the NTFS volume.

27

> > MSS> Only NTFS has the ADS in its on-disk format.

> MSS> Neither FAT12/16/32 nor any UNIX FS have them.
>
> There are way too many file systems and some of them do
have streams.

Mainstream ones (ext2, BSD FFS) have no, and POSIX syscall
API has also no support for them.

The feature is nearly useless. It is used in Windows nearly
only for Macintosh fileserver. Even the things like "Save/Web
page (complete) in IE does not use this feature, they use a
directory named “bla-bla-bla.files” instead.

Things like this are never ‘useless’; one just needs to find the proper
application :slight_smile:

For a Unix system that did support alternate data streams, we maintained
extended security attributes for the file sysytem object (not to be
confused with the objects on a windows-base system). These attributes
were protected from unauthorized modification, of course.

[rest of original msg snipped]

28

Thanks Neal - I agree - identifying my FSD as NTFS can’t be good. I’m
currently only running on Win2000 pro and server. Thanks for your help.

Greg

“Neal Christiansen” wrote in message
news:xxxxx@ntfsd…
Greg,

I am seeing if I can track someone down from the shell team who can let
me know what is going on. What OS versions have you tried this on?

Sometimes it is a mystery to the file system group as to why the shell
does what it does.

It is probably a bad idea to have your file system identify itself as
NTFS.

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Greg Pearce
Sent: Monday, July 12, 2004 11:18 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:streams - why should I do this?

Thanks Max… that clears that up.

Tony said that I must be missing something still though.

In this attempt to support ADS in my FSD, when my FSD gets an
IRP_MJ_QUERY_VOLUME_INFORMATION, I turn on FILE_NAMED_STREAMS in the
filesystem attributes field of the FILE_FS_ATTRIBUTE_INFORMATION
structure,
and set the FileSystemName field to “NTFS”, and explorer stops whining
about
losing the ADS information. I never get warned that I would lose ADS
info
when doing a copy from the command prompt. I still need to put some
meat in
there to parse the name correctly and some other things, but it looks
like
it would work if all that code were in place.

What am I missing, besides MJ_QUERY_INFORMATION/FileStreamInformation
support?

Getting back to a previous question that I had, is it a problem legally
to
tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t
support
all the stuff NTFS does? It seems like Explorer should have a better
way to
determine if a file system supports ADS instead of relying on the file
system’s internal “name”. Anyone care to comment? Anyone from
Microsoft?
It is greatly appreciated!

Thanks again and again

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> Sorry, this is not FSCTL. This is
> MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in
CDFS.
This
> thing enumerates all attribute streams in the file.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Greg Pearce” wrote in message
> news:…
> > Hey Max! You’ve just hit on some information that I would like to
find
out
> > more about; i.e. what FSCTLs are required for ADS? Where can I find
this
> > kind of stuff documented?
> >
> > Thanks,
> >
> > Greg
>
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

29

This is actually a good point. When we constructed Episode many years
ago (part of the Open Group’s Distributed Computing Environment) for
UNIX (including AIX, which is what reminded me in the first place) we
explicitly included support for alternative data streams.

This is because HFS supports alternative data streams. Given that OS X
is a UNIX variant, I’d argue that this is another instance of
alternative data streams on UNIX. Also, given that we’ve seen HFS ports
to Windows, it constitutes another implementation on Windows as well.

They are useful for a variety of tasks. For example, you can construct
some very clever data management file system filters over NTFS by using
data streams.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vossen, Joseph (ISS
Atlanta)
Sent: Tuesday, July 13, 2004 8:11 AM
To: ntfsd redirect
Subject: RE: Re:[ntfsd] streams - why should I do this?

> MSS> ADS are NTFS-only.
>
> Is this supposed to mean that only NTFS has this feature or
this means
> that

Only NTFS has the ADS in its on-disk format.
Neither FAT12/16/32 nor any UNIX FS have them.

IBM’s Unix (AIX) has alternate data streams. It has been a few years
since I’ve been in that code, I believe they are called ‘Extensions’.

As about the network redirs - MRXSMB will support them, if
the server is sharing the NTFS volume.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

30

[snip]

This is because HFS supports alternative data streams. Given
that OS X is a UNIX variant, I’d argue that this is another
instance of alternative data streams on UNIX. Also, given
that we’ve seen HFS ports to Windows, it constitutes another
implementation on Windows as well.

IIRC, OS X is based on NeXTStep, which is/was based on a (highly
modified) version of the Mach kernel; lots of cool stuff there.

They are useful for a variety of tasks. For example, you can
construct some very clever data management file system
filters over NTFS by using data streams.

Oh yeah… :slight_smile:

31

Be that as it may, Mach in turn was nothing more than UNIX on top of
Accent (the original Rashid CMU micro-kernel). So, I’ll stand by “UNIX
variant”. I’d consider AIX to be a UNIX variant as well, although it
bears little resemblance to either BSD 4.3/4.4 or SVR4 either.

Ah, the Golden Age of operating systems…

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vossen, Joseph (ISS
Atlanta)
Sent: Tuesday, July 13, 2004 9:12 AM
To: ntfsd redirect
Subject: RE: Re:[ntfsd] streams - why should I do this?

[snip]

This is because HFS supports alternative data streams. Given
that OS X is a UNIX variant, I’d argue that this is another
instance of alternative data streams on UNIX. Also, given
that we’ve seen HFS ports to Windows, it constitutes another
implementation on Windows as well.

IIRC, OS X is based on NeXTStep, which is/was based on a (highly
modified) version of the Mach kernel; lots of cool stuff there.

They are useful for a variety of tasks. For example, you can
construct some very clever data management file system
filters over NTFS by using data streams.

Oh yeah… :slight_smile:

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

32

> Be that as it may, Mach in turn was nothing more than UNIX on

top of Accent (the original Rashid CMU micro-kernel). So,
I’ll stand by “UNIX variant”. I’d consider AIX to be a UNIX
variant as well, although it bears little resemblance to
either BSD 4.3/4.4 or SVR4 either.

True, for the most part, the only thing that AIX has in common with most
Unixs is the API; the kernel is, how should I say this, different? :))

Ah, the Golden Age of operating systems…

Exec-8 (os100), MVS, VM, assembly language, card readers, paper
tape…sniff…

33

Greg,

I talked with the shell team and they are currently checking for NTFS to
determine if the file system supports multiple streams or not.

They have already fixed this for longhorn so that they check the volume
properties.

They are willing to fix this for XP SP3 and for SRV03 SP1. Would that
be sufficient to meet your needs?

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Monday, July 12, 2004 8:24 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Re:Re:streams - why should I do this?

Greg,

I am seeing if I can track someone down from the shell team who can let
me know what is going on. What OS versions have you tried this on?

Sometimes it is a mystery to the file system group as to why the shell
does what it does.

It is probably a bad idea to have your file system identify itself as
NTFS.

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Greg Pearce
Sent: Monday, July 12, 2004 11:18 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:streams - why should I do this?

Thanks Max… that clears that up.

Tony said that I must be missing something still though.

In this attempt to support ADS in my FSD, when my FSD gets an
IRP_MJ_QUERY_VOLUME_INFORMATION, I turn on FILE_NAMED_STREAMS in the
filesystem attributes field of the FILE_FS_ATTRIBUTE_INFORMATION
structure,
and set the FileSystemName field to “NTFS”, and explorer stops whining
about
losing the ADS information. I never get warned that I would lose ADS
info
when doing a copy from the command prompt. I still need to put some
meat in
there to parse the name correctly and some other things, but it looks
like
it would work if all that code were in place.

What am I missing, besides MJ_QUERY_INFORMATION/FileStreamInformation
support?

Getting back to a previous question that I had, is it a problem legally
to
tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t
support
all the stuff NTFS does? It seems like Explorer should have a better
way to
determine if a file system supports ADS instead of relying on the file
system’s internal “name”. Anyone care to comment? Anyone from
Microsoft?
It is greatly appreciated!

Thanks again and again

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> Sorry, this is not FSCTL. This is
> MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in
CDFS.
This
> thing enumerates all attribute streams in the file.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Greg Pearce” wrote in message
> news:…
> > Hey Max! You’ve just hit on some information that I would like to
find
out
> > more about; i.e. what FSCTLs are required for ADS? Where can I find
this
> > kind of stuff documented?
> >
> > Thanks,
> >
> > Greg
>
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

34

Neal,

Yes, that would be fine… thanks for following up on it and getting back to
me (us).

Regards,

Greg

“Neal Christiansen” wrote in message
news:xxxxx@ntfsd…
Greg,

I talked with the shell team and they are currently checking for NTFS to
determine if the file system supports multiple streams or not.

They have already fixed this for longhorn so that they check the volume
properties.

They are willing to fix this for XP SP3 and for SRV03 SP1. Would that
be sufficient to meet your needs?

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Monday, July 12, 2004 8:24 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Re:Re:streams - why should I do this?

Greg,

I am seeing if I can track someone down from the shell team who can let
me know what is going on. What OS versions have you tried this on?

Sometimes it is a mystery to the file system group as to why the shell
does what it does.

It is probably a bad idea to have your file system identify itself as
NTFS.

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Greg Pearce
Sent: Monday, July 12, 2004 11:18 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:streams - why should I do this?

Thanks Max… that clears that up.

Tony said that I must be missing something still though.

In this attempt to support ADS in my FSD, when my FSD gets an
IRP_MJ_QUERY_VOLUME_INFORMATION, I turn on FILE_NAMED_STREAMS in the
filesystem attributes field of the FILE_FS_ATTRIBUTE_INFORMATION
structure,
and set the FileSystemName field to “NTFS”, and explorer stops whining
about
losing the ADS information. I never get warned that I would lose ADS
info
when doing a copy from the command prompt. I still need to put some
meat in
there to parse the name correctly and some other things, but it looks
like
it would work if all that code were in place.

What am I missing, besides MJ_QUERY_INFORMATION/FileStreamInformation
support?

Getting back to a previous question that I had, is it a problem legally
to
tell callers of QUERY VOLUME that I’m “NTFS”, even though I don’t
support
all the stuff NTFS does? It seems like Explorer should have a better
way to
determine if a file system supports ADS instead of relying on the file
system’s internal “name”. Anyone care to comment? Anyone from
Microsoft?
It is greatly appreciated!

Thanks again and again

Greg

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntfsd…
> Sorry, this is not FSCTL. This is
> MJ_QUERY_INFORMATION/FileStreamInformation, which is also absent in
CDFS.
This
> thing enumerates all attribute streams in the file.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Greg Pearce” wrote in message
> news:…
> > Hey Max! You’ve just hit on some information that I would like to
find
out
> > more about; i.e. what FSCTLs are required for ADS? Where can I find
this
> > kind of stuff documented?
> >
> > Thanks,
> >
> > Greg
>
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com