ReFS Disk Mounting

NTFSD
1

Hi experts,

we have a product which lets us mount volume images. It is a very old
product, and uses a reflector architecture, whre a kernel mode driver
exposes the PDO/FDO and the image is mounted in user land by a
service. The mounted image would then be exposed as a drive on the OS.

it works fine for FAT, FAT32. and NTFS.

It used to work fine for ReFS also, till build 8440.

With build 8441, the ReFs mount broke.

Now, there are several storage, disk and volume ioctls which we had
not supported so far, and we could successfully mount ntfs and fat
images. But with build 8441 something else seems to be needed. Below
is a dump of all the IOCTLs we *don’t* support as of now, perhaps one
of you could help us isolate the one needed in this case…

Also, some of the IOCTLs are not found in the various DDK headers, I
tried the latest win 8 WDK also, no luck, but the same IOCTLS also
come for NTFS mount, and we dont support them, but mount happens.

So does ReFs need anything extra?

IOCTLS in NTFS mount which we *dont* support (still mount succeeds)

IOCTL 000700F8 IOCTL_DISK_IS_CLUSTERED
IOCTL 00074804 IOCTL_DISK_MEDIA_REMOVAL

IOCTL 002D0C14 IOCTL_STORAGE_GET_HOTPLUG_INFO
IOCTL 002D118C
IOCTL 002D1190
IOCTL 002D1400 IOCTL_STORAGE_QUERY_PROPERTY
IOCTL 002D148C

IOCTL 00560000 IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
IOCTL 0056C008 IOCTL_VOLUME_ONLINE
IOCTL 00560030 IOCTL_VOLUME_IS_CLUSTERED
IOCTL 00560038 IOCTL_VOLUME_GET_GPT_ATTRIBUTES
IOCTL 00560048 IOCTL_VOLUME_IS_DYNAMIC
IOCTL 00564052 IOCTL_VOLUME_QUERY_ALLOCATION_HINT
IOCTL 0056C064

IOCTL 004D000C IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME
IOCTL 004D0010
IOCTL 004D0018 IOCTL_MOUNTDEV_QUERY_STABLE_GUID

IOCTL 0066001B FT_BALANCED_READ_MODE

IOCTLS in ReFS mount which we *dont* support (mount fails)

IOCTL 000700F8 IOCTL_DISK_IS_CLUSTERED
IOCTL 00074804 IOCTL_DISK_MEDIA_REMOVAL

IOCTL 002D0C14 IOCTL_STORAGE_GET_HOTPLUG_INFO
IOCTL 002D118C
IOCTL 002D1190
IOCTL 002D1400 IOCTL_STORAGE_QUERY_PROPERTY
IOCTL 002D148C

IOCTL 00560000 IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
IOCTL 0056C008 IOCTL_VOLUME_ONLINE
IOCTL 00560030 IOCTL_VOLUME_IS_CLUSTERED
IOCTL 00560038 IOCTL_VOLUME_GET_GPT_ATTRIBUTES
IOCTL 00560048 IOCTL_VOLUME_IS_DYNAMIC
IOCTL 0056C064

IOCTL 004D0018 IOCTL_MOUNTDEV_QUERY_STABLE_GUID
IOCTL 004D000C IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME
IOCTL 004DC010 IOCTL_MOUNTDEV_LINK_CREATED

2

So which IOCTLs did you get for ReFS before build 8440, when mounting ReFS used to work ?

Thanks,
Alex.

3

hi alex,

we were getting almost the same ioctls

IOCTL 00074804 IOCTL_DISK_MEDIA_REMOVAL

IOCTL 002D0C14 IOCTL_STORAGE_GET_HOTPLUG_INFO
IOCTL 002D118C
IOCTL 002D1190
IOCTL 002D1400 IOCTL_STORAGE_QUERY_PROPERTY
IOCTL 002D148C

IOCTL 004D000C IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME
IOCTL 004D0010 *
IOCTL 004D0018 IOCTL_MOUNTDEV_QUERY_STABLE_GUID
IOCTL 004DC010 IOCTL_MOUNTDEV_LINK_CREATED

IOCTL 00560000 IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
IOCTL 00560030 IOCTL_VOLUME_IS_CLUSTERED
IOCTL 00560038 IOCTL_VOLUME_GET_GPT_ATTRIBUTES
IOCTL 00560048 IOCTL_VOLUME_IS_DYNAMIC
IOCTL 0056C008 IOCTL_VOLUME_ONLINE
IOCTL 0056C064

IOCTL 0066001B FT_BALANCED_READ_MODE

Note that IOCTL_DISK_IS_CLUSTERED is not in the above list. I tried
adding this support in the FDO and returned 0 which means that the
device doesnt support clustering, but the error persists.

a new property id has been added n windows 8 for
IOCTL_STORAGE_QUERY_PROPERTY it is called
StorageDeviceResiliencyProperty I am not sure if this is required or
not. I dont have the latest WDK installed so couldnt check off hand.

rest all looks the same to me.

Any help for any of you would be appreciated.

best

Ami

On Tue, Jul 24, 2012 at 11:33 PM, wrote:
> So which IOCTLs did you get for ReFS before build 8440, when mounting ReFS used to work ?
>
> Thanks,
> Alex.
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

4

  1. What is the last IOCTL you see on your device before the mount fails?

  2. Is this a volume that is formatted on an earlier build or are you formatting the volume on the same build?

Regards,
Sarosh.
Development Lead, File System Core
Microsoft Corp

This posting is provided “AS IS” with no warranties, and confers no Rights

5

Hi Sarosh,

Thanks for trying to help.

I dont have the last IOCTL off hand, I will try to figure it out and
post it here tomorrow.

No, the volume was formatted with the same build,i.e 8441.

Surprisingly, when we try mounting it with our driver running on 8440
it mounts perfectly. Which makes us think the behaviour of some of the
ioctls has changed between 8440 and 8441. Perhaps the NTFS driver was
more forgiving, than the REFS one when we didnt handle a particular
IOCTL call.

Long back in win 2K days i recall, NTFS couldn’t be mounted in read
only mode, is there such a restriction for ReFS? I ask so, because our
image mounting software only allows read only mounts.

thanks
Ami

On Wed, Jul 25, 2012 at 11:46 PM, wrote:
> 1. What is the last IOCTL you see on your device before the mount fails?
>
> 2. Is this a volume that is formatted on an earlier build or are you formatting the volume on the same build?
>
> Regards,
> Sarosh.
> Development Lead, File System Core
> Microsoft Corp
>
> This posting is provided “AS IS” with no warranties, and confers no Rights
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

Hi Sarosh, Alex, all,

I debugged this further, and found that read only mounts go well. It
is only when we do writable mounts we have a problem.

I see the following log line:

SynchronousMinstoreIo (flush) to device FFFFFA80055B0BE0 returned
status c0000010

the device in question is our FDO.

This seems something new in Wndows 8 build 8441? Teh stack shows ReFS
trying to send IOs.

1: kd> kv
DBGHELP: Wdf01000 - private symbols
e:\websyms\Wdf01000.pdb\3AC6E535793745F99D15F1A578791BFC2\Wdf01000.pdb
Child-SP RetAddr : Args to Child
: Call Site
fffff880067c7ce8 fffff8800108038e : fffffa800552ed20 fffffa800552ed20 fffff880067c7d30 8000001200000001 :
MyDriver!FdoDeviceEvtIoWrite [f:\work\Mydriver\MyDriverfdo.cpp @ 659]
fffff880067c7cf0 fffff8800107ff0f : fffffa80055ad000 fffff880067c7db0 fffffa80055ad020 0000000000000001 :
Wdf01000!FxIoQueue::DispatchRequestToDriver+0x3ce
fffff880067c7d70 fffff8800108a877 : fffffa8005e7e010 fffffa800552ed00 0000000000000000 fffffa800552ed20 :
Wdf01000!FxIoQueue::DispatchEvents+0x66f
fffff880067c7df0 fffff8800108593a : fffffa8005e7e000 fffffa800552ed20 fffffa80057eaca0 fffff880067c7ed0 :
Wdf01000!FxIoQueue::QueueRequest+0x2ab
fffff880067c7e60 fffff8800108159e : fffffa800552ed20 fffffa80057eaca0 0000000000000000 fffffa80057eaca0 :
Wdf01000!FxPkgIo::Dispatch+0x4da
fffff880067c7ed0 fffff880010813ae : fffffa80057eaca0 0000000000000000 fffffa80055b4690 fffff880067c8000 :
Wdf01000!FxDevice::Dispatch+0x19a
DBGHELP: ReFS - public symbols
e:\websyms\refs.pdb\27923D3EC86D42E3B434582D048D89BC1\refs.pdb
fffff880067c7f10 fffff88003a09e52 : fffff88004176840 0000000000000000 fffffa80057abb00 0000000000000001 :
Wdf01000!FxDevice::DispatchWithLock+0xa6
fffff880067c7f50 fffff80051ae56a7 : 0000000002d2f638 0000000000000001 0000000000000000 0000000000000000 :
ReFS!RefsStorageDriverCallout+0x16
fffff880067c7f80 fffff80051ae566d : fffffa8005411d00 0000000000000000 0000000000000002 fffff80051b29606 :
nt!KySwitchKernelStackCallout+0x27 (TrapFrame @ fffff880067c7e40) fffff880041766e0 fffff80051b29606 : 0000000000000006
fffffa8005411d00 0000000000000006 0000000000000000 : nt!KiSwitchKernelStackContinue fffff88004176700 0000000000000000 : 0000000000000000
0000000000000000 0000000000000000 00000000`00000000 :
nt!KeExpandKernelStackAndCalloutInternal+0x206

Can any of you throw some more light on this log message, and where to
look for the bug? I have traced, our write handler returns proper
status sucecess, but still ReFS Seems unhappy.

On Thu, Jul 26, 2012 at 1:11 AM, Ami Awbadhho wrote:
> Hi Sarosh,
>
> Thanks for trying to help.
>
> I dont have the last IOCTL off hand, I will try to figure it out and
> post it here tomorrow.
>
> No, the volume was formatted with the same build,i.e 8441.
>
> Surprisingly, when we try mounting it with our driver running on 8440
> it mounts perfectly. Which makes us think the behaviour of some of the
> ioctls has changed between 8440 and 8441. Perhaps the NTFS driver was
> more forgiving, than the REFS one when we didnt handle a particular
> IOCTL call.
>
> Long back in win 2K days i recall, NTFS couldn’t be mounted in read
> only mode, is there such a restriction for ReFS? I ask so, because our
> image mounting software only allows read only mounts.
>
> thanks
> Ami
>
>
> On Wed, Jul 25, 2012 at 11:46 PM, wrote:
>> 1. What is the last IOCTL you see on your device before the mount fails?
>>
>> 2. Is this a volume that is formatted on an earlier build or are you formatting the volume on the same build?
>>
>> Regards,
>> Sarosh.
>> Development Lead, File System Core
>> Microsoft Corp
>>
>> This posting is provided “AS IS” with no warranties, and confers no Rights
>>
>>
>> —
>> NTFSD is sponsored by OSR
>>
>> For our schedule of debugging and file system seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

Can you please provide more of the stack? The portion you show cuts off just before the useful frames.

Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.

8

On 07/27/2012 05:40 AM, Ami Awbadhho wrote:

Hi Sarosh, Alex, all,

I debugged this further, and found that read only mounts go well. It
is only when we do writable mounts we have a problem.

I see the following log line:

SynchronousMinstoreIo (flush) to device FFFFFA80055B0BE0 returned
status c0000010

the device in question is our FDO.

This seems something new in Wndows 8 build 8441? Teh stack shows ReFS
trying to send IOs.

Yes, ReFS depends on flush for correctness. If you have a read-write
device and fail IRP_MJ_FLUSH_BUFFERS, ReFS will treat this as a fatal
condition. The enforcement of this arrived in 8441.

Can I ask why you wouldn’t support flush on a read-write device? This
will cause problems for NTFS too, but rather than fail explicitly it
defeats recovery and can cause corrupt volumes.

  • M
9

Hi Malcom,

thanks for clarifying this!

Can I ask why you wouldn’t support flush on a read-write device? This will
cause problems for NTFS too, but rather than fail explicitly it defeats
recovery and can cause corrupt volumes.

If my understanding is correct this ioctl is supposed to be handled by
FS drivers and FSF drivers. Ours is not the case. We are exposing a
virtual disk device with a volume on top of it.

So are we supposed to handle this IRP?

Best

ami

On Sat, Jul 28, 2012 at 12:06 PM, Malcolm Smith [MSFT]
wrote:
> On 07/27/2012 05:40 AM, Ami Awbadhho wrote:
>>
>> Hi Sarosh, Alex, all,
>>
>> I debugged this further, and found that read only mounts go well. It
>> is only when we do writable mounts we have a problem.
>>
>> I see the following log line:
>>
>> SynchronousMinstoreIo (flush) to device FFFFFA80055B0BE0 returned
>> status c0000010
>>
>> the device in question is our FDO.
>>
>> This seems something new in Wndows 8 build 8441? Teh stack shows ReFS
>> trying to send IOs.
>
>
> Yes, ReFS depends on flush for correctness. If you have a read-write device
> and fail IRP_MJ_FLUSH_BUFFERS, ReFS will treat this as a fatal condition.
> The enforcement of this arrived in 8441.
>
> Can I ask why you wouldn’t support flush on a read-write device? This will
> cause problems for NTFS too, but rather than fail explicitly it defeats
> recovery and can cause corrupt volumes.
>
> - M
>
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

10

> virtual disk device with a volume on top of it.

Disk devices can have internal caches and thus require flush.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

11

IRP_MJ_FLUSH_BUFFERS is a major IRP function, not an IOCTL value.

If you look at the SCSI disk class driver sample in the WDK, you will see that it does handle IRP_MJ_FLUSH_BUFFERS. From this I would expect other disk devices to handle it as well. I’d expect to find the volume managers also handle this, since it just means “make sure your cache is clean”.

Tony
OSR

12

Everyone,

thanks for the help, after implementing IRP_MJ_FLUSH_BUFFERS we are
able to mount ReFS properly.

Malcolm,

Can I ask why you wouldn’t support flush on a read-write device? This will cause problems for NTFS too, but rather than fail explicitly it defeats recovery and can cause corrupt volumes

The reason for this was that even though we mount it as RW, we would
be discarding all writes to the ramdisk baking up the volume image,
this is related to some feature in the product, which requires
temporary RW support. The expected behaviour is that the user will
lose all modifications to the mounted volume.

Thanks

ami

On Mon, Jul 30, 2012 at 7:46 PM, Tony Mason wrote:
> IRP_MJ_FLUSH_BUFFERS is a major IRP function, not an IOCTL value.
>
> If you look at the SCSI disk class driver sample in the WDK, you will see that it does handle IRP_MJ_FLUSH_BUFFERS. From this I would expect other disk devices to handle it as well. I’d expect to find the volume managers also handle this, since it just means “make sure your cache is clean”.
>
> Tony
> OSR
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer