Map Large Files To Small Files

It’s possible that what I desire already exists. If so, please let me
know. But if not, I’m asking for the simplest method to create the
following:

A method (I assume using a file system filter driver) that will let me
create a virtual volume that maps to multiple physical volumes, with the
following criteria:

  • Work under Windows XP
  • On write, split large virtual files (say 10’s of GB) into smaller
    physical files (say 2.35 GB).
  • On read, treat multiple small physical files as one large virtual file
    to the application.
  • Provide no appreciable performance penalty (i.e. extra data copies).
  • Support drives with completely different geometry (size, etc.).
  • Allow the virtual volume to be accessed by any application.
  • Configure the translation to add/remove destination volumes.
  • Configure the maximum size of the destination files (e.g. 2.35 GB).
  • No reconfiguration of the physical volumes to support virtual mapping
    (i.e. they are already configured).
  • Ability to access physical volumes to access individual physical files.
  • No requirement of physical file sizes on read (i.e. they can all be
    different sizes).
  • No requirement of additional support files (i.e. index or list files;
    the filenames should be sufficient)

Some optional features:

  • No special limitations (e.g. physical file size, write and read
    mutually exclusive, pre-determination of virtual file size, etc.)
  • The ability to add/remove volumes dynamically (without a reboot)
  • On write, spill files onto other volumes when one fills.
  • On read, deal with files on multiple volumes.
  • Work under other versions of Windows.

Rick Tillery

Any chance someone could let me know if a file system filter driver is
the appropriate approach for this? Is there a different level at which
I should interface?

Rick

Rick Tillery wrote:

It’s possible that what I desire already exists. If so, please let me
know. But if not, I’m asking for the simplest method to create the
following:

A method (I assume using a file system filter driver) that will let me
create a virtual volume that maps to multiple physical volumes, with
the following criteria:

  • Work under Windows XP
  • On write, split large virtual files (say 10’s of GB) into smaller
    physical files (say 2.35 GB).
  • On read, treat multiple small physical files as one large virtual
    file to the application.
  • Provide no appreciable performance penalty (i.e. extra data copies).
  • Support drives with completely different geometry (size, etc.).
  • Allow the virtual volume to be accessed by any application.
  • Configure the translation to add/remove destination volumes.
  • Configure the maximum size of the destination files (e.g. 2.35 GB).
  • No reconfiguration of the physical volumes to support virtual
    mapping (i.e. they are already configured).
  • Ability to access physical volumes to access individual physical files.
  • No requirement of physical file sizes on read (i.e. they can all be
    different sizes).
  • No requirement of additional support files (i.e. index or list
    files; the filenames should be sufficient)

Some optional features:

  • No special limitations (e.g. physical file size, write and read
    mutually exclusive, pre-determination of virtual file size, etc.)
  • The ability to add/remove volumes dynamically (without a reboot)
  • On write, spill files onto other volumes when one fills.
  • On read, deal with files on multiple volumes.
  • Work under other versions of Windows.

Rick Tillery


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@hogfan.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

No. You posted in HTML with fonts too small for easy reading. If a
newsgroup message has HTML or fonts or bugs, I won’t touch it.

“Rick Tillery” wrote in message news:xxxxx@ntfsd…
> Any chance someone could let me know if a file system filter driver is
> the appropriate approach for this? Is there a different level at which
> I should interface?
>
> Rick
>
> Rick Tillery wrote:
>
>> It’s possible that what I desire already exists. If so, please let me
>> know. But if not, I’m asking for the simplest method to create the
>> following:
>>
>> A method (I assume using a file system filter driver) that will let me
>> create a virtual volume that maps to multiple physical volumes, with
>> the following criteria:
>>
>> + Work under Windows XP
>> + On write, split large virtual files (say 10’s of GB) into smaller
>> physical files (say 2.35 GB).
>> + On read, treat multiple small physical files as one large virtual
>> file to the application.
>> + Provide no appreciable performance penalty (i.e. extra data copies).
>> + Support drives with completely different geometry (size, etc.).
>> + Allow the virtual volume to be accessed by any application.
>> + Configure the translation to add/remove destination volumes.
>> + Configure the maximum size of the destination files (e.g. 2.35 GB).
>> + No reconfiguration of the physical volumes to support virtual
>> mapping (i.e. they are already configured).
>> + Ability to access physical volumes to access individual physical files.
>> + No requirement of physical file sizes on read (i.e. they can all be
>> different sizes).
>> + No requirement of additional support files (i.e. index or list
>> files; the filenames should be sufficient)
>>
>> Some optional features:
>> + No special limitations (e.g. physical file size, write and read
>> mutually exclusive, pre-determination of virtual file size, etc.)
>> + The ability to add/remove volumes dynamically (without a reboot)
>> + On write, spill files onto other volumes when one fills.
>> + On read, deal with files on multiple volumes.
>> + Work under other versions of Windows.
>>
>> Rick Tillery
>>
>>
>> —
>> Questions? First check the IFS FAQ at
>> https://www.osronline.com/article.cfm?id=17
>>
>> You are currently subscribed to ntfsd as: xxxxx@hogfan.com
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>
>

Actually, I posted in HTML and text. I didn’t realize there would be a
problem until you responded…thank you.

David J. Craig wrote:
>> No. You posted in HTML with fonts too small for easy reading. If a
>> newsgroup message has HTML or fonts or bugs, I won’t touch it.

So let’s try again:

It’s possible that what I desire already exists. If so, please let me
know. But if not, I’m asking for the simplest method to create the
following:

A method (I assume using a file system filter driver) that will let me
create a virtual volume that maps to multiple physical volumes, with the
following criteria:

  • Work under Windows XP
  • On write, split large virtual files (say 10’s of GB) into smaller
    physical files (say 2.35 GB).
  • On read, treat multiple small physical files as one large virtual file
    to the application.
  • Provide no appreciable performance penalty (i.e. extra data copies).
  • Support drives with completely different geometry (size, etc.).
  • Allow the virtual volume to be accessed by any application.
  • Configure the translation to add/remove destination volumes.
  • Configure the maximum size of the destination files (e.g. 2.35 GB).
  • No reconfiguration of the physical volumes to support virtual mapping
    (i.e. they are already configured).
  • Ability to access physical volumes to access individual physical files.
  • No requirement of physical file sizes on read (i.e. they can all be
    different sizes).
  • No requirement of additional support files (i.e. index or list files;
    the filenames should be sufficient)

Some optional features:

  • No special limitations (e.g. physical file size, write and read
    mutually exclusive, pre-determination of virtual file size, etc.)
  • The ability to add/remove volumes dynamically (without a reboot)
  • On write, spill files onto other volumes when one fills.
  • On read, deal with files on multiple volumes.
  • Work under other versions of Windows.

Rick Tillery

You could do this with a file system. Not a good idea, but this is not the
correct newsgroup for the better solutions. Some more information about the
physical file containing those multiple physical volumes is a good idea or
at least express any format restrictions that may exist.

If you mean just take a hard drive with a MBR and one or more partitions on
it and copy all its sectors to a file. If you then want to break that
single data file into multiple files, then it is easy. It is not usually
done at the file system level.

“Rick Tillery” wrote in message news:xxxxx@ntfsd…
> Actually, I posted in HTML and text. I didn’t realize there would be a
> problem until you responded…thank you.
>
> David J. Craig wrote:
> >> No. You posted in HTML with fonts too small for easy reading. If a
> >> newsgroup message has HTML or fonts or bugs, I won’t touch it.
>
>
> So let’s try again:
>
> It’s possible that what I desire already exists. If so, please let me
> know. But if not, I’m asking for the simplest method to create the
> following:
>
> A method (I assume using a file system filter driver) that will let me
> create a virtual volume that maps to multiple physical volumes, with the
> following criteria:
>
> + Work under Windows XP
> + On write, split large virtual files (say 10’s of GB) into smaller
> physical files (say 2.35 GB).
> + On read, treat multiple small physical files as one large virtual file
> to the application.
> + Provide no appreciable performance penalty (i.e. extra data copies).
> + Support drives with completely different geometry (size, etc.).
> + Allow the virtual volume to be accessed by any application.
> + Configure the translation to add/remove destination volumes.
> + Configure the maximum size of the destination files (e.g. 2.35 GB).
> + No reconfiguration of the physical volumes to support virtual mapping
> (i.e. they are already configured).
> + Ability to access physical volumes to access individual physical files.
> + No requirement of physical file sizes on read (i.e. they can all be
> different sizes).
> + No requirement of additional support files (i.e. index or list files;
> the filenames should be sufficient)
>
> Some optional features:
> + No special limitations (e.g. physical file size, write and read
> mutually exclusive, pre-determination of virtual file size, etc.)
> + The ability to add/remove volumes dynamically (without a reboot)
> + On write, spill files onto other volumes when one fills.
> + On read, deal with files on multiple volumes.
> + Work under other versions of Windows.
>
> Rick Tillery
>
>
>
>

I don’t want to operate on the sector level. In fact, this should be a
virtual file that maps to multiple physical files. The physical files
can be handled entirely through user-level APIs. But the creation of
the virtual volume for handling the virtual files is where I thought the
file system filter driver was appropriate. If there is a better
approach, please let me know and I’ll investigate elsewhere.

Rick

David J. Craig wrote:

You could do this with a file system. Not a good idea, but this is not the
correct newsgroup for the better solutions. Some more information about the
physical file containing those multiple physical volumes is a good idea or
at least express any format restrictions that may exist.

If you mean just take a hard drive with a MBR and one or more partitions on
it and copy all its sectors to a file. If you then want to break that
single data file into multiple files, then it is easy. It is not usually
done at the file system level.

“Rick Tillery” wrote in message news:xxxxx@ntfsd…
>
>
>>Actually, I posted in HTML and text. I didn’t realize there would be a
>>problem until you responded…thank you.
>>
>>David J. Craig wrote:
>>
>>
>>>>No. You posted in HTML with fonts too small for easy reading. If a
>>>>newsgroup message has HTML or fonts or bugs, I won’t touch it.
>>>>
>>>>
>>So let’s try again:
>>
>>It’s possible that what I desire already exists. If so, please let me
>>know. But if not, I’m asking for the simplest method to create the
>>following:
>>
>>A method (I assume using a file system filter driver) that will let me
>>create a virtual volume that maps to multiple physical volumes, with the
>>following criteria:
>>
>>+ Work under Windows XP
>>+ On write, split large virtual files (say 10’s of GB) into smaller
>>physical files (say 2.35 GB).
>>+ On read, treat multiple small physical files as one large virtual file
>>to the application.
>>+ Provide no appreciable performance penalty (i.e. extra data copies).
>>+ Support drives with completely different geometry (size, etc.).
>>+ Allow the virtual volume to be accessed by any application.
>>+ Configure the translation to add/remove destination volumes.
>>+ Configure the maximum size of the destination files (e.g. 2.35 GB).
>>+ No reconfiguration of the physical volumes to support virtual mapping
>>(i.e. they are already configured).
>>+ Ability to access physical volumes to access individual physical files.
>>+ No requirement of physical file sizes on read (i.e. they can all be
>>different sizes).
>>+ No requirement of additional support files (i.e. index or list files;
>>the filenames should be sufficient)
>>
>>Some optional features:
>>+ No special limitations (e.g. physical file size, write and read
>>mutually exclusive, pre-determination of virtual file size, etc.)
>>+ The ability to add/remove volumes dynamically (without a reboot)
>>+ On write, spill files onto other volumes when one fills.
>>+ On read, deal with files on multiple volumes.
>>+ Work under other versions of Windows.
>>
>>Rick Tillery
>>
>>
>>
>>
>>
>>
>
>
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@hogfan.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>