There seems to be a dearth of information about the capabilities of IFS (on
the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
You mean replace the existing NTFS with another one ?
I doubt it, unless you have the complete sources of it.
You can easily replace e.g. Fastfat.sys by the version
compiled from IFS kit. But NTFS sources are not included
in IFS.
L.
For all practical uses, the only realistic technique is to use a file system
filter driver to layer your own extensions on top of NTFS (or whatever base
filesystem you are using).
Failing that, you need to write a replacement file system from scratch -
generally a lot more difficult.
If you can restrict your platforms to service packs that have not yet been
released, then I would strongly recommend using the new Filter Manager as
the basis for your extensions.
I hope this helps
Brian
“Ryan Ross” wrote in message news:xxxxx@ntfsd…
> There seems to be a dearth of information about the capabilities of IFS
(on
> the internet), so please do not flame me if my question seems odd.
>
> Using IFS, is it possible to swap out NTFS in Windows for an extended
> version (NTFS + custom code)?
>
> Sorry if this confuses you.
>
> Thanks,
> Ryan
>
>
>
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Ryan,
Is there somethig in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Hello!
You (“Tony Mason” ) wrote to “Windows File Systems Devs
Interest List” on Wed, 2 Jun 2004 06:16:02 -0400:
TM> At one point the EULA for the IFS Kit explicitly prohibited buliding
TM> replacement file systems for those shipped in the base OS kit
TM> (http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx)
TM> but it no longer appears to do so, although I seriously doubt that
TM> Microsoft would be willing to sign such a driver (and that is
TM> required by the license agreement).
Can you please explain the above in more details? Does it mean that using
IFS kit one can’t build a file system that can be used to format a partition
on HDD? Or one can’t create a file system driver which stores data locally?
Or … ?
Thanks in advance
Sincerely yours,
Eugene Mayevski
One cannot replace NTFS or FAT.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Eugene Mayevski
Sent: Thursday, June 03, 2004 4:46 PM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
Hello!
You (“Tony Mason” ) wrote to “Windows File Systems Devs
Interest List” on Wed, 2 Jun 2004 06:16:02 -0400:
TM> At one point the EULA for the IFS Kit explicitly prohibited buliding
TM> replacement file systems for those shipped in the base OS kit
TM> (http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx)
TM> but it no longer appears to do so, although I seriously doubt that
TM> Microsoft would be willing to sign such a driver (and that is
TM> required by the license agreement).
Can you please explain the above in more details? Does it mean that
using IFS kit one can’t build a file system that can be used to format a
partition on HDD? Or one can’t create a file system driver which stores
data locally?
Or … ?
Thanks in advance
Sincerely yours,
Eugene Mayevski
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along with
associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/featus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m well
aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Ryan,
Again, this is a traditional use of the file system filter driver. You
can track even small changes (user level modifications) within a file
system filter driver. There are a number of people that participate on
this list who have done so.
In your filter you can monitor each write I/O operation; most people do
it at the non-cached I/O level, which tends to block the changes into
PAGE_SIZE chunks (up to 64KB) but I have also seen it done at the user
I/O (cached) level, although this approach requires that you be able to
detect and handle when an application memory maps the file.
Once you have the data, you can then deal with it appropriately.
Again, given there are existing solutions in this space, you might also
want to consider licensing someone else’s solution.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 11:22 PM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along
with associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/fe
atus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m
well aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Aright, so a system filter driver has these capabilities…
So, just to make sure I have this correct, you are saying that using a
filter driver, I can extend the FS such that 1.) changes are stored as
records, 2.) the original data is not touched, and 3.) Windows will play
ball with this driver without any extreme modifications?
The general idea is to allow the records to be merged with the data at
runtime or on command (commit). And for as long as these records persist,
one can strip them out, if the original data’s integrity is affected by
those records.
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 4:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Again, this is a traditional use of the file system filter driver. You
can track even small changes (user level modifications) within a file
system filter driver. There are a number of people that participate on
this list who have done so.
In your filter you can monitor each write I/O operation; most people do
it at the non-cached I/O level, which tends to block the changes into
PAGE_SIZE chunks (up to 64KB) but I have also seen it done at the user
I/O (cached) level, although this approach requires that you be able to
detect and handle when an application memory maps the file.
Once you have the data, you can then deal with it appropriately.
Again, given there are existing solutions in this space, you might also
want to consider licensing someone else’s solution.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 11:22 PM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along
with associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/fe
atus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m
well aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Ryan,
So, what you propose doing is to build a “change engine” to sit over
NTFS (or FAT or whatever). Changes are recorded elsewhere by your
filter, reads are satisfied first from your filter (if you have a
change) and then from the underlying file system.
I would ask you why you would implement this “per file” rather than “per
volume” (e.g., file system filter driver versus a volume filter driver).
The latter is easier to build. The former makes sense if you are trying
to do this on a file-by-file basis (e.g., you want to protect only
certain files.)
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Friday, June 04, 2004 10:03 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Aright, so a system filter driver has these capabilities…
So, just to make sure I have this correct, you are saying that using a
filter driver, I can extend the FS such that 1.) changes are stored as
records, 2.) the original data is not touched, and 3.) Windows will play
ball with this driver without any extreme modifications?
The general idea is to allow the records to be merged with the data at
runtime or on command (commit). And for as long as these records
persist,
one can strip them out, if the original data’s integrity is affected by
those records.
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 4:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Again, this is a traditional use of the file system filter driver. You
can track even small changes (user level modifications) within a file
system filter driver. There are a number of people that participate on
this list who have done so.
In your filter you can monitor each write I/O operation; most people do
it at the non-cached I/O level, which tends to block the changes into
PAGE_SIZE chunks (up to 64KB) but I have also seen it done at the user
I/O (cached) level, although this approach requires that you be able to
detect and handle when an application memory maps the file.
Once you have the data, you can then deal with it appropriately.
Again, given there are existing solutions in this space, you might also
want to consider licensing someone else’s solution.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 11:22 PM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along
with associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/fe
atus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m
well aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Hmm. That is a matter of debate. I am still learning information in this
area.
The idea was to slam it into the NTFS driver, and then lock down the system
so that read/writes only action through the driver (third party disk
software lucks out). The goal for this system is to keep accidental/viral
problems at bay. If you drop a table (SQL) or the latest worm affects the
system, stripping out the changes should be trivial (remove the bad records
via the Commit console).
I would like to use this on entire volumes, so in this case, do you believe
a volume filter driver is better?
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 12:09 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
So, what you propose doing is to build a “change engine” to sit over
NTFS (or FAT or whatever). Changes are recorded elsewhere by your
filter, reads are satisfied first from your filter (if you have a
change) and then from the underlying file system.
I would ask you why you would implement this “per file” rather than “per
volume” (e.g., file system filter driver versus a volume filter driver).
The latter is easier to build. The former makes sense if you are trying
to do this on a file-by-file basis (e.g., you want to protect only
certain files.)
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Friday, June 04, 2004 10:03 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Aright, so a system filter driver has these capabilities…
So, just to make sure I have this correct, you are saying that using a
filter driver, I can extend the FS such that 1.) changes are stored as
records, 2.) the original data is not touched, and 3.) Windows will play
ball with this driver without any extreme modifications?
The general idea is to allow the records to be merged with the data at
runtime or on command (commit). And for as long as these records
persist,
one can strip them out, if the original data’s integrity is affected by
those records.
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 4:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Again, this is a traditional use of the file system filter driver. You
can track even small changes (user level modifications) within a file
system filter driver. There are a number of people that participate on
this list who have done so.
In your filter you can monitor each write I/O operation; most people do
it at the non-cached I/O level, which tends to block the changes into
PAGE_SIZE chunks (up to 64KB) but I have also seen it done at the user
I/O (cached) level, although this approach requires that you be able to
detect and handle when an application memory maps the file.
Once you have the data, you can then deal with it appropriately.
Again, given there are existing solutions in this space, you might also
want to consider licensing someone else’s solution.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 11:22 PM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along
with associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/fe
atus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m
well aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Check out www.shadowstor.com
Jamey
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Sunday, June 06, 2004 8:26 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Hmm. That is a matter of debate. I am still learning information in this
area.
The idea was to slam it into the NTFS driver, and then lock down the system
so that read/writes only action through the driver (third party disk
software lucks out). The goal for this system is to keep accidental/viral
problems at bay. If you drop a table (SQL) or the latest worm affects the
system, stripping out the changes should be trivial (remove the bad records
via the Commit console).
I would like to use this on entire volumes, so in this case, do you believe
a volume filter driver is better?
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 12:09 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
So, what you propose doing is to build a “change engine” to sit over
NTFS (or FAT or whatever). Changes are recorded elsewhere by your
filter, reads are satisfied first from your filter (if you have a
change) and then from the underlying file system.
I would ask you why you would implement this “per file” rather than “per
volume” (e.g., file system filter driver versus a volume filter driver).
The latter is easier to build. The former makes sense if you are trying
to do this on a file-by-file basis (e.g., you want to protect only
certain files.)
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Friday, June 04, 2004 10:03 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Aright, so a system filter driver has these capabilities…
So, just to make sure I have this correct, you are saying that using a
filter driver, I can extend the FS such that 1.) changes are stored as
records, 2.) the original data is not touched, and 3.) Windows will play
ball with this driver without any extreme modifications?
The general idea is to allow the records to be merged with the data at
runtime or on command (commit). And for as long as these records
persist,
one can strip them out, if the original data’s integrity is affected by
those records.
Thanks,
Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, June 04, 2004 4:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Again, this is a traditional use of the file system filter driver. You
can track even small changes (user level modifications) within a file
system filter driver. There are a number of people that participate on
this list who have done so.
In your filter you can monitor each write I/O operation; most people do
it at the non-cached I/O level, which tends to block the changes into
PAGE_SIZE chunks (up to 64KB) but I have also seen it done at the user
I/O (cached) level, although this approach requires that you be able to
detect and handle when an application memory maps the file.
Once you have the data, you can then deal with it appropriately.
Again, given there are existing solutions in this space, you might also
want to consider licensing someone else’s solution.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 11:22 PM
To: ntfsd redirect
Subject: RE: [ntfsd] Question about IFS
Yes, the USN is insufficient.
Correct if I’m wrong (I’m a novice in regards to low-level file system
drivers), but the USN only records that a change has occurred (along
with associated file information, etc.).
http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/fe
atus
ability/msjntfs5.mspx
I’d like to overload the driver, so that the USN records not only that a
change has occurred, but (using a diff engine) the change itself. I’m
well aware of the overhead generated by this modification.
Think of it as NTFS/Shadow Files on steroids.
-Ryan
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, June 03, 2004 4:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Question about IFS
Ryan,
Is there something in the existing USN journal mechanism that is
insufficient for your purposes?
As an aside, there are numerous products that use file system filter
drivers to monitor changes to the underlying file system data - this is
a traditional use of the file system filter driver.
But Microsoft added the USN journal in NTFS precisely to address many of
the needs/requirements here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Thursday, June 03, 2004 3:50 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Question about IFS
Right. I’d like to add delta records to the FS. Would a filter driver
provide these capabilities?
Cheers,
Ryan
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Generally, one would not “swap out” NTFS in the fashion you described,
but rather include the added functionality in a file system filter
driver.
At one point the EULA for the IFS Kit explicitly prohibited buliding
replacement file systems for those shipped in the base OS kit
(http://www.microsoft.com/whdc/devtools/ifskit/ServerIFSEULA.mspx) but
it no longer appears to do so, although I seriously doubt that Microsoft
would be willing to sign such a driver (and that is required by the
license agreement).
So, the answer would be: “technically, yes you can replace anything in
the OS that you want”. Logistically, that probably isn’t worth the
tremendous effort required. Contractually, it is at best dubious.
I’d suggest that unless you have some very compelling reason (and a
bundle of cash you want to spend on development) you look to file system
filter drivers and not NTFS replacement.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ryan Ross
Sent: Wednesday, June 02, 2004 3:33 AM
To: ntfsd redirect
Subject: [ntfsd] Question about IFS
There seems to be a dearth of information about the capabilities of IFS
(on the internet), so please do not flame me if my question seems odd.
Using IFS, is it possible to swap out NTFS in Windows for an extended
version (NTFS + custom code)?
Sorry if this confuses you.
Thanks,
Ryan
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@nni.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com