question on IFS

NTFSD
1

Hi gurus,

I need your advice on the idea that I have. Now a days the disk access has
become a bottleneck. To help improve performance Is it possible to manage
some of the file activity like copy of larger files without moving the data
accross the PCI bus, where the larger video files get stored on the disk (
because if stored on the outer edge will have faster access), run
defgramentation in the background, do backup in the background. May be by
writting a file system filter driver that would export some entry points for
the applications to perform these functions.

Thank You,
Shakeel.

2

For SCSI devices, there are the copy commands that allow the SCSI bus to
move data between devices without the computers intervention.

One device will act like an initiator and the other device will act like a
target.

Doing this in Windows is another story :slight_smile:

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Monday, August 30, 2004 10:26 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] question on IFS

Hi gurus,

I need your advice on the idea that I have. Now a days the disk access has
become a bottleneck. To help improve performance Is it possible to manage
some of the file activity like copy of larger files without moving the data
accross the PCI bus, where the larger video files get stored on the disk (
because if stored on the outer edge will have faster access), run
defgramentation in the background, do backup in the background. May be by
writting a file system filter driver that would export some entry points for
the applications to perform these functions.

Thank You,
Shakeel.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

__________ NOD32 1.853 (20040830) Information __________

This message was checked by NOD32 antivirus system.
http://www.nod32.com

3

Thanks Jamey,

This is exactly what I am thinking of doing is to move data from one drive
to another drive or may be to the same drive depends on the user action.
Instead of the way the normal file copy does is first reads data into host
memory and then writes read data from host memory to the disk. This cosumes
CPU cycles also data has to move accross the PCI bus.

Since the the HBA is RAID controller and only way to communicate is by LBA
address of source and destination location to move the data.

Now, my question is how to determine the physical locations of the source
file, destination file on the disk. Are there APIs/IOCTLs to
read/create/modify the MFT table entries? or the APIs/IOCTLs that
encapsulates both FAT32 and NTFS.

Thanks,
Shakeel.

“Jamey Kirby” wrote in message news:xxxxx@ntfsd…
> For SCSI devices, there are the copy commands that allow the SCSI bus to
> move data between devices without the computers intervention.
>
> One device will act like an initiator and the other device will act like a
> target.
>
> Doing this in Windows is another story :slight_smile:
>
> Jamey
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
> Sent: Monday, August 30, 2004 10:26 AM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] question on IFS
>
> Hi gurus,
>
> I need your advice on the idea that I have. Now a days the disk access has
> become a bottleneck. To help improve performance Is it possible to manage
> some of the file activity like copy of larger files without moving the
data
> accross the PCI bus, where the larger video files get stored on the disk (
> because if stored on the outer edge will have faster access), run
> defgramentation in the background, do backup in the background. May be by
> writting a file system filter driver that would export some entry points
for
> the applications to perform these functions.
>
> Thank You,
> Shakeel.
>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> NOD32 1.853 (20040830) Information
>
> This message was checked by NOD32 antivirus system.
> http://www.nod32.com
>
>
>

4

FSCTL_GET_RETRIEVAL_POINTERS
FSCTL_MOVE_FILE

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“shakeel” wrote in message
news:…
> Thanks Jamey,
>
> This is exactly what I am thinking of doing is to move data from one drive
> to another drive or may be to the same drive depends on the user action.
> Instead of the way the normal file copy does is first reads data into host
> memory and then writes read data from host memory to the disk. This cosumes
> CPU cycles also data has to move accross the PCI bus.
>
> Since the the HBA is RAID controller and only way to communicate is by LBA
> address of source and destination location to move the data.
>
> Now, my question is how to determine the physical locations of the source
> file, destination file on the disk. Are there APIs/IOCTLs to
> read/create/modify the MFT table entries? or the APIs/IOCTLs that
> encapsulates both FAT32 and NTFS.
>
> Thanks,
> Shakeel.
>
> “Jamey Kirby” wrote in message news:xxxxx@ntfsd…
> > For SCSI devices, there are the copy commands that allow the SCSI bus to
> > move data between devices without the computers intervention.
> >
> > One device will act like an initiator and the other device will act like a
> > target.
> >
> > Doing this in Windows is another story :slight_smile:
> >
> > Jamey
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
> > Sent: Monday, August 30, 2004 10:26 AM
> > To: Windows File Systems Devs Interest List
> > Subject: [ntfsd] question on IFS
> >
> > Hi gurus,
> >
> > I need your advice on the idea that I have. Now a days the disk access has
> > become a bottleneck. To help improve performance Is it possible to manage
> > some of the file activity like copy of larger files without moving the
> data
> > accross the PCI bus, where the larger video files get stored on the disk (
> > because if stored on the outer edge will have faster access), run
> > defgramentation in the background, do backup in the background. May be by
> > writting a file system filter driver that would export some entry points
> for
> > the applications to perform these functions.
> >
> > Thank You,
> > Shakeel.
> >
> >
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> > NOD32 1.853 (20040830) Information
> >
> > This message was checked by NOD32 antivirus system.
> > http://www.nod32.com
> >
> >
> >
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

5

On Tue, 31 Aug 2004 11:38:11 +0400, you wrote:

FSCTL_GET_RETRIEVAL_POINTERS
FSCTL_MOVE_FILE

The first one will *read* the file’s on-disk location, but
unfortunately not *write* it as required in this case.

Working on a device level might be easier: here, you’ll see commands
to read from one location and write the results to another - it may be
possible to convert those into device commands, bypassing the PCI bus.
Trying to do this on the FS level doesn’t seem possible: for obvious
reasons the NTFS and FAT32 drivers won’t expect blocks of data to move
“independently”!

(Alternatively, remapping the clusters on disk so files are really
physically but not logically contiguous should work - although the
extra overhead may negate much of the benefit…)

James.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“shakeel” wrote in message
>news:…
>> Thanks Jamey,
>>
>> This is exactly what I am thinking of doing is to move data from one drive
>> to another drive or may be to the same drive depends on the user action.
>> Instead of the way the normal file copy does is first reads data into host
>> memory and then writes read data from host memory to the disk. This cosumes
>> CPU cycles also data has to move accross the PCI bus.
>>
>> Since the the HBA is RAID controller and only way to communicate is by LBA
>> address of source and destination location to move the data.
>>
>> Now, my question is how to determine the physical locations of the source
>> file, destination file on the disk. Are there APIs/IOCTLs to
>> read/create/modify the MFT table entries? or the APIs/IOCTLs that
>> encapsulates both FAT32 and NTFS.
>>
>> Thanks,
>> Shakeel.
>>
>> “Jamey Kirby” wrote in message news:xxxxx@ntfsd…
>> > For SCSI devices, there are the copy commands that allow the SCSI bus to
>> > move data between devices without the computers intervention.
>> >
>> > One device will act like an initiator and the other device will act like a
>> > target.
>> >
>> > Doing this in Windows is another story :slight_smile:
>> >
>> > Jamey
>> >
>> > -----Original Message-----
>> > From: xxxxx@lists.osr.com
>> > [mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
>> > Sent: Monday, August 30, 2004 10:26 AM
>> > To: Windows File Systems Devs Interest List
>> > Subject: [ntfsd] question on IFS
>> >
>> > Hi gurus,
>> >
>> > I need your advice on the idea that I have. Now a days the disk access has
>> > become a bottleneck. To help improve performance Is it possible to manage
>> > some of the file activity like copy of larger files without moving the
>> data
>> > accross the PCI bus, where the larger video files get stored on the disk (
>> > because if stored on the outer edge will have faster access), run
>> > defgramentation in the background, do backup in the background. May be by
>> > writting a file system filter driver that would export some entry points
>> for
>> > the applications to perform these functions.
>> >
>> > Thank You,
>> > Shakeel.
>> >
>> >
>> >
>> > —
>> > Questions? First check the IFS FAQ at
>> > https://www.osronline.com/article.cfm?id=17
>> >
>> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
>> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>> >
>> > NOD32 1.853 (20040830) Information
>> >
>> > This message was checked by NOD32 antivirus system.
>> > http://www.nod32.com
>> >
>> >
>> >
>>
>>
>>
>> —
>> Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>>
>> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@spamcop.net
>To unsubscribe send a blank email to xxxxx@lists.osr.com

6

> > FSCTL_GET_RETRIEVAL_POINTERS

> FSCTL_MOVE_FILE

The first one will *read* the file’s on-disk location, but
unfortunately not *write* it as required in this case.

The second will write :slight_smile:

Working on a device level might be easier: here, you’ll see commands
to read from one location and write the results to another

SCSI only. No such commands in IDE.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

7

On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:

> > FSCTL_GET_RETRIEVAL_POINTERS
> > FSCTL_MOVE_FILE
>
> The first one will *read* the file’s on-disk location, but
> unfortunately not *write* it as required in this case.

The second will write :slight_smile:

It will write - but it doesn’t do what the previous poster wanted, of
updating the on-disk location to reflect a move already carried out -
it causes NTFS to perform the move itself.

James.

8

Thanks James and Maxim for your help,

I am novice in this subject, please help me understand. suppose if I write
file system filter driver that intercept the file copy operation and calls
FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP to determine file’s on
disk locaitons and unused cluster locations and sends the vender specific
command to RAID HBA to move the data. will this work? How would windows know
or update its own internal structures about the new file?

Thanks,

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:

>> > FSCTL_GET_RETRIEVAL_POINTERS
>> > FSCTL_MOVE_FILE
>>
>> The first one will read the file’s on-disk location, but
>> unfortunately not write it as required in this case.
>
>The second will write :slight_smile:

It will write - but it doesn’t do what the previous poster wanted, of
updating the on-disk location to reflect a move already carried out -
it causes NTFS to perform the move itself.

James.

9

That’s their point - Windows will not know to update its internal
structure.

You could probably achieve this IF you dismounted the volume and changed
the data layout “underneath” the Windows file system (we once did
something similar under NTFS with a filter that allowed us to dismount
whenever we wanted and then reassociated afterwards. At the time, NTFS
was not quite up to the task of being remounted every six seconds,
however, and eventually would leak all of non-paged pool.) But to do
this you must understand the on disk format and modify it directly
within your software. Certainly not documented, and quite likely to
break each time the on disk format changes.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Tuesday, August 31, 2004 4:03 PM
To: ntfsd redirect
Subject: Re:[ntfsd] question on IFS

Thanks James and Maxim for your help,

I am novice in this subject, please help me understand. suppose if I
write
file system filter driver that intercept the file copy operation and
calls
FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP to determine
file’s on
disk locaitons and unused cluster locations and sends the vender
specific
command to RAID HBA to move the data. will this work? How would windows
know
or update its own internal structures about the new file?

Thanks,

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:

>> > FSCTL_GET_RETRIEVAL_POINTERS
>> > FSCTL_MOVE_FILE
>>
>> The first one will read the file’s on-disk location, but
>> unfortunately not write it as required in this case.
>
>The second will write :slight_smile:

It will write - but it doesn’t do what the previous poster wanted, of
updating the on-disk location to reflect a move already carried out -
it causes NTFS to perform the move itself.

James.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

10

On Tue, 31 Aug 2004 13:02:56 -0700, you wrote:

Thanks James and Maxim for your help,

I am novice in this subject, please help me understand. suppose if I write
file system filter driver that intercept the file copy operation and calls
FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP to determine file’s on
disk locaitons and unused cluster locations and sends the vender specific
command to RAID HBA to move the data. will this work? How would windows know
or update its own internal structures about the new file?

It won’t - that’s the problem. There isn’t a call to tell the FS “the
data you thought was at location X is now in location Y instead.”
Either you need to modify the on-disk structures directly yourself
(which is … messy) or operate on a lower level, either converting
“normal” read/write accesses into device-specific move calls or
remapping clusters to make non-contiguous files really occupy
contiguous areas on disk.

Whichever you choose, it won’t be pretty or easy.

James.

Thanks,

“James” wrote in message news:xxxxx@ntfsd…
>On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:
>
>>> > FSCTL_GET_RETRIEVAL_POINTERS
>>> > FSCTL_MOVE_FILE
>>>
>>> The first one will read the file’s on-disk location, but
>>> unfortunately not write it as required in this case.
>>
>>The second will write :slight_smile:
>
>It will write - but it doesn’t do what the previous poster wanted, of
>updating the on-disk location to reflect a move already carried out -
>it causes NTFS to perform the move itself.
>
>
>James.
>
>
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@spamcop.net
>To unsubscribe send a blank email to xxxxx@lists.osr.com

11

Thanks Tony, James and Maxim for enlightening on this subject.

I am so glad to here that Tony you have tried doing something similar in the
past. To directly modify the on-disk data structure, how to determine what
data fields of on the disk data structure to modify and the physical
location on the disk.
Are these data structures documented? Please point me to the source where I
learn about it.

Thanks,
Shakeel.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
That’s their point - Windows will not know to update its internal
structure.

You could probably achieve this IF you dismounted the volume and changed
the data layout “underneath” the Windows file system (we once did
something similar under NTFS with a filter that allowed us to dismount
whenever we wanted and then reassociated afterwards. At the time, NTFS
was not quite up to the task of being remounted every six seconds,
however, and eventually would leak all of non-paged pool.) But to do
this you must understand the on disk format and modify it directly
within your software. Certainly not documented, and quite likely to
break each time the on disk format changes.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Tuesday, August 31, 2004 4:03 PM
To: ntfsd redirect
Subject: Re:[ntfsd] question on IFS

Thanks James and Maxim for your help,

I am novice in this subject, please help me understand. suppose if I
write
file system filter driver that intercept the file copy operation and
calls
FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP to determine
file’s on
disk locaitons and unused cluster locations and sends the vender
specific
command to RAID HBA to move the data. will this work? How would windows
know
or update its own internal structures about the new file?

Thanks,

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:

>> > FSCTL_GET_RETRIEVAL_POINTERS
>> > FSCTL_MOVE_FILE
>>
>> The first one will read the file’s on-disk location, but
>> unfortunately not write it as required in this case.
>
>The second will write :slight_smile:

It will write - but it doesn’t do what the previous poster wanted, of
updating the on-disk location to reflect a move already carried out -
it causes NTFS to perform the move itself.

James.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

12

Notice that I did say “similar”. In the case I was considering, the on
disk structure was modified by a different NTFS implementation
(basically, it was an early form of file system sharing, where we
allowed only one copy of NTFS to modify the on disk structure).

The Linux community has an NTFS implementation and that is likely to be
the closest you will come to finding the on disk format information,
short of licensing the information directly from Microsoft.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Tuesday, August 31, 2004 6:35 PM
To: ntfsd redirect
Subject: Re:[ntfsd] question on IFS

Thanks Tony, James and Maxim for enlightening on this subject.

I am so glad to here that Tony you have tried doing something similar in
the
past. To directly modify the on-disk data structure, how to determine
what
data fields of on the disk data structure to modify and the physical
location on the disk.
Are these data structures documented? Please point me to the source
where I
learn about it.

Thanks,
Shakeel.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
That’s their point - Windows will not know to update its internal
structure.

You could probably achieve this IF you dismounted the volume and changed
the data layout “underneath” the Windows file system (we once did
something similar under NTFS with a filter that allowed us to dismount
whenever we wanted and then reassociated afterwards. At the time, NTFS
was not quite up to the task of being remounted every six seconds,
however, and eventually would leak all of non-paged pool.) But to do
this you must understand the on disk format and modify it directly
within your software. Certainly not documented, and quite likely to
break each time the on disk format changes.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October
18, 2004 in Silicon Valley!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Tuesday, August 31, 2004 4:03 PM
To: ntfsd redirect
Subject: Re:[ntfsd] question on IFS

Thanks James and Maxim for your help,

I am novice in this subject, please help me understand. suppose if I
write
file system filter driver that intercept the file copy operation and
calls
FSCTL_GET_RETRIEVAL_POINTERS, FSCTL_GET_VOLUME_BITMAP to determine
file’s on
disk locaitons and unused cluster locations and sends the vender
specific
command to RAID HBA to move the data. will this work? How would windows
know
or update its own internal structures about the new file?

Thanks,

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 13:57:31 +0400, you wrote:

>> > FSCTL_GET_RETRIEVAL_POINTERS
>> > FSCTL_MOVE_FILE
>>
>> The first one will read the file’s on-disk location, but
>> unfortunately not write it as required in this case.
>
>The second will write :slight_smile:

It will write - but it doesn’t do what the previous poster wanted, of
updating the on-disk location to reflect a move already carried out -
it causes NTFS to perform the move itself.

James.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

13

On Tue, 31 Aug 2004 15:34:37 -0700, you wrote:

Thanks Tony, James and Maxim for enlightening on this subject.

I am so glad to here that Tony you have tried doing something similar in the
past. To directly modify the on-disk data structure, how to determine what
data fields of on the disk data structure to modify and the physical
location on the disk.
Are these data structures documented? Please point me to the source where I
learn about it.

There’s public documentation - but not official Microsoft
documentation - here: http://linux-ntfs.sourceforge.net/ntfs/

Alternatively, you could do what Executive Software did to create
their NTFS defragmenter for Windows NT 3.5 (which lacked any defrag
API): license the source code from Microsoft, develop your own
modified NTFS and FAT drivers for NT with the necessary modifications
then use that. Of course, this did turn out to be a support and
compatibility nightmare when they tried it…

James.

14

Tony, James thank you so much for clearing the clouds. I will start with the
linux documentation.
I am wondering how the companies like anti-virus software, disk
magangement/utility software are manage to get that information. Are they
getting information directly from Microsoft? will Microsoft share the NTFS
information with us?

Thanks,
Shakeel.

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 15:34:37 -0700, you wrote:

>Thanks Tony, James and Maxim for enlightening on this subject.
>
>I am so glad to here that Tony you have tried doing something similar in
the
>past. To directly modify the on-disk data structure, how to determine what
>data fields of on the disk data structure to modify and the physical
>location on the disk.
>Are these data structures documented? Please point me to the source where I
>learn about it.

There’s public documentation - but not official Microsoft
documentation - here: http://linux-ntfs.sourceforge.net/ntfs/

Alternatively, you could do what Executive Software did to create
their NTFS defragmenter for Windows NT 3.5 (which lacked any defrag
API): license the source code from Microsoft, develop your own
modified NTFS and FAT drivers for NT with the necessary modifications
then use that. Of course, this did turn out to be a support and
compatibility nightmare when they tried it…

James.

15

We call it hard work!

Virus scanners do not need to know any specifics about the on-disk
structure. Defragmenters use the file system APIs to defragment.

Partition tools like Partition Magic take their lives and your data in their
hands every time you run that crap.

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of shakeel
Sent: Wednesday, September 01, 2004 10:29 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] question on IFS

Tony, James thank you so much for clearing the clouds. I will start with the
linux documentation.
I am wondering how the companies like anti-virus software, disk
magangement/utility software are manage to get that information. Are they
getting information directly from Microsoft? will Microsoft share the NTFS
information with us?

Thanks,
Shakeel.

“James” wrote in message news:xxxxx@ntfsd…
On Tue, 31 Aug 2004 15:34:37 -0700, you wrote:

>Thanks Tony, James and Maxim for enlightening on this subject.
>
>I am so glad to here that Tony you have tried doing something similar in
the
>past. To directly modify the on-disk data structure, how to determine what
>data fields of on the disk data structure to modify and the physical
>location on the disk.
>Are these data structures documented? Please point me to the source where I
>learn about it.

There’s public documentation - but not official Microsoft
documentation - here: http://linux-ntfs.sourceforge.net/ntfs/

Alternatively, you could do what Executive Software did to create
their NTFS defragmenter for Windows NT 3.5 (which lacked any defrag
API): license the source code from Microsoft, develop your own
modified NTFS and FAT drivers for NT with the necessary modifications
then use that. Of course, this did turn out to be a support and
compatibility nightmare when they tried it…

James.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

NOD32 1.855 (20040901) Information

This message was checked by NOD32 antivirus system.
http://www.nod32.com

16

On Wed, 1 Sep 2004 10:29:24 -0700, you wrote:

Tony, James thank you so much for clearing the clouds. I will start with the
linux documentation.
I am wondering how the companies like anti-virus software, disk
magangement/utility software are manage to get that information. Are they
getting information directly from Microsoft? will Microsoft share the NTFS
information with us?

Most of those programs don’t need any of this information: anti-virus
and defrag tools need only use the documented interfaces within
Windows. Very few tools have ever needed to bypass that and
“understand” NTFS for themselves: Diskeeper (before NT4), Partition
Magic, NTFSDOS (although the later version of that runs the NT NTFS
drivers under DOS instead!) and the Linux and FreeBSD NTFS drivers.

Microsoft does supply the information (and NT source code) to some
people, but it’s not easy to get hold of - IIRC the first two items on
my list used it, I’m not sure about NTFSDOS, and the other NTFS
implementations don’t.

James.

17

Antivirus are not needed to do this, as about disk management tools - at
least some of them use the reverse-engineered info.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “shakeel”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, September 01, 2004 9:29 PM
Subject: Re:[ntfsd] question on IFS

> Tony, James thank you so much for clearing the clouds. I will start with the
> linux documentation.
> I am wondering how the companies like anti-virus software, disk
> magangement/utility software are manage to get that information. Are they
> getting information directly from Microsoft? will Microsoft share the NTFS
> information with us?
>
> Thanks,
> Shakeel.
>
> “James” wrote in message news:xxxxx@ntfsd…
> On Tue, 31 Aug 2004 15:34:37 -0700, you wrote:
>
> >Thanks Tony, James and Maxim for enlightening on this subject.
> >
> >I am so glad to here that Tony you have tried doing something similar in
> the
> >past. To directly modify the on-disk data structure, how to determine what
> >data fields of on the disk data structure to modify and the physical
> >location on the disk.
> >Are these data structures documented? Please point me to the source where I
> >learn about it.
>
> There’s public documentation - but not official Microsoft
> documentation - here: http://linux-ntfs.sourceforge.net/ntfs/
>
> Alternatively, you could do what Executive Software did to create
> their NTFS defragmenter for Windows NT 3.5 (which lacked any defrag
> API): license the source code from Microsoft, develop your own
> modified NTFS and FAT drivers for NT with the necessary modifications
> then use that. Of course, this did turn out to be a support and
> compatibility nightmare when they tried it…
>
>
> James.
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com