Direct Write To a volume

Hi All,

I did develop a driver which can mount a file as a volume or a folder(mounted folder)
I am required to support big virtual files(500GB and more) and it works.
However, when new big file is created it can take a very long time to create the file only(not sparse file). If file is create as a sparse file then during format it will need to go and allocate all clusters anyway which takes time.

To improve that, I did change the way I am writing to a “host” file. Before I used ZwWrite/Read and now in the driver I am getting file extents and do my own mapping of file pointer to extents,
and I am writing directly to a volume.

Everything works just fine.
I am able to create 500GB file, mount it, format it in 10sec not a problem, but on 2k/zp only.

As we all now on vista and up microsoft blocks direct writes to a volume . I thought that limtations was for user mode apps only, but its not. Is there is a way to write to a volume form the driver?
What if I convert logical extents to physical and try to write directly to physical disk instead of logical? would this help?
Or maybe i need to create IRP and send MJ_WRITE to the volume device directly?

Would appreciate any suggestions.

Best Regards,
Igor

>As we all now on vista and up microsoft blocks direct writes to a volume . I thought that limtations >was for user mode apps only, but its not. Is there is a way to write to a volume form the driver?
IoMeter works on Vista and up which could do direct I/O on both logical volume and physical disk. It is an open source project.

Igor Sharovar

As we all now on vista and up microsoft blocks direct writes to a volume . I
thought that limtations was for user mode apps only, but its not. Is there
is a way to write to a volume form the driver?

No are not blocked, restricted to some areas but that wouldn’t bother your
case.

What if I convert logical extents to physical and try to write directly to

physical disk instead of logical? would this help?
Or maybe i need to create IRP and send MJ_WRITE to the volume device
directly?

Yes you can do this also. It will work.

-Deepak

> As we all now on vista and up microsoft blocks direct writes to a volume .

It does not. The restore path of the image backup app uses exactly this, and these apps often work without changes in Vista, without putting any efforts on making them Vista-aware.

Also, FORMAT and CHKDSK /F does the same.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Interesting,

Thank Maxim, I guess I have to spend more effort to find out why I can read and why I get access_denied on write. Same code works just fine on XP x32 and x64.

Igor

Maxim , is there is anything special about how I get a handle to a volume?

Thats how I open it:
status = ZwCreateFile(&pDevExt->hVolumeOfHostFile, FILE_SHARE_READ | FILE_SHARE_WRITE, &objAttrVolume, &Irp->IoStatus, NULL, FILE_ATTRIBUTE_NORMAL, FILE_SHARE_VALID_FLAGS, FILE_OPEN,
FILE_NON_DIRECTORY_FILE | FILE_SEQUENTIAL_ONLY | FILE_NO_INTERMEDIATE_BUFFERING | FILE_SYNCHRONOUS_IO_NONALERT, NULL, 0);

It works, just on writes I get access denied

Paste the code where you do 'InitializeObjectAttributes"

-Deepak

On Fri, Apr 15, 2011 at 1:59 PM, wrote:

> Maxim , is there is anything special about how I get a handle to a volume?
>
>
> Thats how I open it:
> status = ZwCreateFile(&pDevExt->hVolumeOfHostFile, FILE_SHARE_READ |
> FILE_SHARE_WRITE, &objAttrVolume, &Irp->IoStatus, NULL,
> FILE_ATTRIBUTE_NORMAL, FILE_SHARE_VALID_FLAGS, FILE_OPEN,
> FILE_NON_DIRECTORY_FILE | FILE_SEQUENTIAL_ONLY |
> FILE_NO_INTERMEDIATE_BUFFERING | FILE_SYNCHRONOUS_IO_NONALERT, NULL, 0);
>
> It works, just on writes I get access denied
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi,

UNICODE_STRING usVolume
RtlInitUnicodeString(&usVolume, L"\??\c:");

InitializeObjectAttributes(&objAttrVolume, &usVolume, OBJ_CASE_INSENSITIVE | OBJ_KERNEL_HANDLE, NULL, NULL);

status = ZwCreateFile(&pDevExt->hVolumeOfHostFile, FILE_SHARE_READ | FILE_SHARE_WRITE, &objAttrVolume, &Irp->IoStatus, NULL,
FILE_ATTRIBUTE_NORMAL, FILE_SHARE_VALID_FLAGS, FILE_OPEN,
FILE_NON_DIRECTORY_FILE | FILE_SEQUENTIAL_ONLY |
FILE_NO_INTERMEDIATE_BUFFERING | FILE_SYNCHRONOUS_IO_NONALERT, NULL, 0);

Thanks,
Igor

Hmm I didn’t notice it in previous post.

Why is your DesiredAccess ‘FILE_SHARE_READ | FILE_SHARE_WRITE’. It should be
GENERIC_WRITE.

All your parameters to ZwCreateFile look messed up (what does
FILE_SHARE_VALID_FLAGS means)

Read the documentation properly -
http://msdn.microsoft.com/en-us/library/ff566424.aspx and fill in correct
parameters to the API.

-Deepak

On Fri, Apr 15, 2011 at 3:50 PM, wrote:

> Hi,
>
> UNICODE_STRING usVolume
> RtlInitUnicodeString(&usVolume, L"\??\c:");
>
> InitializeObjectAttributes(&objAttrVolume, &usVolume, OBJ_CASE_INSENSITIVE
> | OBJ_KERNEL_HANDLE, NULL, NULL);
>
> status = ZwCreateFile(&pDevExt->hVolumeOfHostFile, FILE_SHARE_READ |
> FILE_SHARE_WRITE, &objAttrVolume, &Irp->IoStatus, NULL,
> FILE_ATTRIBUTE_NORMAL, FILE_SHARE_VALID_FLAGS, FILE_OPEN,
> FILE_NON_DIRECTORY_FILE | FILE_SEQUENTIAL_ONLY |
> FILE_NO_INTERMEDIATE_BUFFERING | FILE_SYNCHRONOUS_IO_NONALERT, NULL, 0);
>
>
> Thanks,
> Igor
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

h_MyDrive_Dest=CreateFile( L"\\.\O:",GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE, NULL,OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL|FILE_FLAG_NO_BUFFERING, NULL);

while ()
{
ReadFile()

WriteFile()

}

On Fri, Apr 15, 2011 at 5:19 PM, Deepak Gupta wrote:

> Hmm I didn’t notice it in previous post.
>
> Why is your DesiredAccess ‘FILE_SHARE_READ | FILE_SHARE_WRITE’. It should
> be GENERIC_WRITE.
>
> All your parameters to ZwCreateFile look messed up (what does
> FILE_SHARE_VALID_FLAGS means)
>
> Read the documentation properly -
> http://msdn.microsoft.com/en-us/library/ff566424.aspx and fill in correct
> parameters to the API.
>
> -Deepak
>
> On Fri, Apr 15, 2011 at 3:50 PM, wrote:
>
>> Hi,
>>
>> UNICODE_STRING usVolume
>> RtlInitUnicodeString(&usVolume, L"\??\c:");
>>
>> InitializeObjectAttributes(&objAttrVolume, &usVolume, OBJ_CASE_INSENSITIVE
>> | OBJ_KERNEL_HANDLE, NULL, NULL);
>>
>> status = ZwCreateFile(&pDevExt->hVolumeOfHostFile, FILE_SHARE_READ
>> | FILE_SHARE_WRITE, &objAttrVolume, &Irp->IoStatus, NULL,
>> FILE_ATTRIBUTE_NORMAL, FILE_SHARE_VALID_FLAGS, FILE_OPEN,
>> FILE_NON_DIRECTORY_FILE | FILE_SEQUENTIAL_ONLY |
>> FILE_NO_INTERMEDIATE_BUFFERING | FILE_SYNCHRONOUS_IO_NONALERT, NULL, 0);
>>
>>
>> Thanks,
>> Igor
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

A new check was added to the FSDs in Vista to restrict some direct disk
access scenarios. You can see the check that was added in the FAT source in
the WDK:

//
// If this is a write on a disk-based volume that is not locked, we need to
limit
// the sectors we allow to be written within the volume. Specifically, we
only
// allow writes to the reserved area. Note that extended DASD can still be
used
// to write past the end of the volume. We also allow kernel mode callers
to force
// access via a flag in the IRP. A handle that issued a dismount can write
anywhere
// as well.
//

if ((Vcb->TargetDeviceObject->DeviceType == FILE_DEVICE_DISK) &&
!FlagOn( Vcb->VcbState, VCB_STATE_FLAG_LOCKED ) &&
!FlagOn( IrpSp->Flags, SL_FORCE_DIRECT_WRITE ) &&
!FlagOn( Ccb->Flags, CCB_FLAG_COMPLETE_DISMOUNT )) {

//
// First check for a write beyond the end of the volume.
//

if (!WriteToEof && (StartingLbo < VolumeSize)) {

//
// This write is within the volume. Make sure it is not beyond the
reserved section.
//

if ((StartingLbo >= FatReservedBytes( &(Vcb->Bpb) )) ||
(ByteCount > (FatReservedBytes( &(Vcb->Bpb) ) - StartingLbo))) {

FatCompleteRequest( IrpContext, Irp, STATUS_ACCESS_DENIED );
return STATUS_ACCESS_DENIED;
}
}
}

The same type of check exists in NTFS in Vista and later.

-scott


Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

wrote in message news:xxxxx@ntdev…

Interesting,

Thank Maxim, I guess I have to spend more effort to find out why I can read
and why I get access_denied on write. Same code works just fine on XP x32
and x64.

Igor

Yeah I guess this was added after demonstration of pagefile attack by
Joanna.

-Deepak

On Fri, Apr 15, 2011 at 6:36 PM, Scott Noone wrote:

> A new check was added to the FSDs in Vista to restrict some direct disk
> access scenarios. You can see the check that was added in the FAT source in
> the WDK:
>
> //
> // If this is a write on a disk-based volume that is not locked, we need
> to limit
> // the sectors we allow to be written within the volume. Specifically, we
> only
> // allow writes to the reserved area. Note that extended DASD can still
> be used
> // to write past the end of the volume. We also allow kernel mode callers
> to force
> // access via a flag in the IRP. A handle that issued a dismount can
> write anywhere
> // as well.
> //
>
> if ((Vcb->TargetDeviceObject->DeviceType == FILE_DEVICE_DISK) &&
> !FlagOn( Vcb->VcbState, VCB_STATE_FLAG_LOCKED ) &&
> !FlagOn( IrpSp->Flags, SL_FORCE_DIRECT_WRITE ) &&
> !FlagOn( Ccb->Flags, CCB_FLAG_COMPLETE_DISMOUNT )) {
>
> //
> // First check for a write beyond the end of the volume.
> //
>
> if (!WriteToEof && (StartingLbo < VolumeSize)) {
>
> //
> // This write is within the volume. Make sure it is not beyond the
> reserved section.
> //
>
> if ((StartingLbo >= FatReservedBytes( &(Vcb->Bpb) )) ||
> (ByteCount > (FatReservedBytes( &(Vcb->Bpb) ) - StartingLbo))) {
>
> FatCompleteRequest( IrpContext, Irp, STATUS_ACCESS_DENIED );
> return STATUS_ACCESS_DENIED;
> }
> }
> }
>
> The same type of check exists in NTFS in Vista and later.
>
> -scott
>
> –
> Scott Noone
> Consulting Associate and Chief System Problem Analyst
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
> wrote in message news:xxxxx@ntdev…
>
>
> Interesting,
>
> Thank Maxim, I guess I have to spend more effort to find out why I can read
> and why I get access_denied on write. Same code works just fine on XP x32
> and x64.
>
> Igor
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi ,

FILE_SHARE_VALID_FLAGS includes file_share_read/write/delete

I do have generic_read,write in the code.
It was late and I was experimenting with parameters, put them there by mistake

Why is your DesiredAccess ‘FILE_SHARE_READ | FILE_SHARE_WRITE’. It should be
GENERIC_WRITE.

Hi All,

here is something I found which looks like a good explanation
http://support.microsoft.com/kb/942448

Cheers,
Igor