Clarification of DeviceExtension use with IoCreateDeviceSecure

Hello list!

My apologies for the crappy looking question, hopefully someone will have
patience to read it and reply…

I was hoping someone could provide some insight into using DeviceExtension
with IoCreateDeviceSecure.

In this case I want to simulate (as much as I can) the way mounting works
on Unix, so there are not many examples to look at out there. I have been
looking at win-btrfs sources, and when I can, fastfat.

So, I call:

status = IoCreateDeviceSecure(WIN_DriverObject,
sizeof(mount_t),
&diskDeviceName,
FILE_DEVICE_DISK,
deviceCharacteristics,
FALSE,
&SDDL_DEVOBJ_SYS_ALL_ADM_RWX_WORLD_RW_RES_R,
NULL,
&diskDeviceObject);

mount_t *zmo_dcb = diskDeviceObject->DeviceExtension;

and fill in my struct ‘zmo_dcb’ with all the goodies I need later.

Some extra bits that follows that, calling IoCreateSymbolicLink(),
unsetting DO_DEVICE_INITIALIZING and finally calling IoVerifyVolume().

Then we wait for the mount request to come in:

case IRP_MJ_FILE_SYSTEM_CONTROL:
switch (IrpSp->MinorFunction) {
case IRP_MN_MOUNT_VOLUME:

DeviceToMount = IrpSp->Parameters.MountVolume.DeviceObject;
mount_t *dcb = DeviceToMount->DeviceExtension; [**1]

which then goes on to call IoCreateDevice(FILE_DEVICE_DISK) and carry on
with other code.

and this works great! No really, it works all is well!

… for me, and for a few other people who have tested it, and I thought I
was done with the mounting part.

Then another user comes along to test it, (on Win10 Pro, like myself, but
hardware vs VMs), and it does not work!

Debugging starts:

The DeviceExtension in [**1] is not mine suddenly - I have no idea what it
points to, but it isn’t the structure I filled in.

When I verify the dcb I get:

if ((dcb->type != MOUNT_TYPE_DCB) ||
(dcb->size != sizeof(mount_t))) {
dprintf(“%s: Not a ZFS dataset – dcb %p ignoring: type 0x%x != 0x%x,
size %d != %d\n”,
func, dcb,
dcb->type, MOUNT_TYPE_DCB, dcb->size,
sizeof(mount_t));

zfs_vnop_mount: Not a ZFS dataset – ignoring: type 0x9fb41040 !=
0x3a444342, size -7423 != 192

Why would this be? Isn’t this what the DeviceExtension ptr is for? Also,
why does it only happen on his install of Win10 ?

I had noticed that win-btrfs also call:

case IRP_MN_MOUNT_VOLUME:
pdo = DeviceToMount;
while (IoGetLowerDeviceObject(pdo)) {
pdo = IoGetLowerDeviceObject(pdo);
dprintf(“going deeper %p\n”, pdo);
}

Which was not needed on my machines (so far), but I thought I would try it
on the trouble machine, in case the device gets encapsulated for some reason:

*** mount request for FFFFD38FE0472040
… going deeper FFFFD38FE03C3C50
… going deeper FFFFD38FE03C4B40
… going deeper FFFFD38FE061F030
… going deeper FFFFD38FE03A9880
zfs_vnop_mount: Not a ZFS dataset – dcb FFFFD38FE03A99D0 ignoring: type
0xe03a9880 != 0x3a444342, size -11377 != 192

which surprised me, and yet, none of those ‘lower device objects’ has a
DeviceExtension that “is mine”.

What gives? Can I not rely on DeviceExtension to stay untouched? Should I
match the new device some other way?
Perhaps by IOCTL_MOUNTDEV_QUERY_DEVICE_NAME ?

Why can his system call IoGetLowerDeviceObject() and get 3 additional
“devices”? How do I know which is mine?

Can I somehow replicate what happens on his system, so I can more easily
debug it myself?

Be patient with me, I’m just a Unix dev… :slight_smile:

Lund


Jorgen Lundman |
Unix Administrator | +81 (0)90-5578-8500
Shibuya-ku, Tokyo | Japan

Here’s my great wisdom for the day: mounting is different on Windows than it
is on Unix :slight_smile:

That being said…I’ll start from the beginning and possibly tell you things
you already know. But, I think that will ultimately be clearer than just
trying to cherry pick the part I think you need.

Each file system creates a File System Device Object (FsDO) and registers it
with IoRegisterFileSystem. The DeviceType of the registered FsDO matters
greatly as it determines which mount requests you receive:

  • FILE_DEVICE_DISK_FILE_SYSTEM : Notified for local disks
  • FILE_DEVICE_CD_ROM_FILE_SYSTEM : Notified for CD-ROMs
  • FILE_DEVICE_TAPE_FILE_SYSTEM : Notified for tapes (yes, you can have a
    tape file system!)

Now, the system runs and other devices are enumerated. For each disk, tape,
or CD-ROM Device Object that is created, the I/O Manager creates a special
structure called a Volume Parameter Block (VPB) and sticks it in the Device
Object. The VPB is the link between the mountable device and the file system
mounted over it.

The first time someone attempts to open the disk/tape/CD-ROM, the I/O
Manager notices that there is a VPB but the file system has yet to be
mounted. It then runs the appropriate list of registered file systems (disk,
tape, or CD-ROM) and serially sends an IRP_MN_MOUNT_VOLUME request to the
FsDOs .

IrpSp->Parameters.MountVolume.DeviceObject contains the disk/tape/CD-ROM
Device Object that you’re being asked to mount over*. On Windows, the file
system driver then has to determine if their file system formatted on the
media (see FatMountVolume). If it *is* you file system, you create what the
file systems (somewhat confusingly) call a Volume Device Object (VDO) and
wire it into the VPB.

Long story short, you have no idea where
IrpSp->Parameters.MountVolume.DeviceObject came from and it may in fact not
be of interest to you. You need some mechanism to determine if it’s a device
that your file system should mount over. If you’re also creating the media
devices then this might be as simple as an IOCTL that you send.

As to what this worked previously, you were just very unlucky :slight_smile: Probably
has to do with the order of the file systems registered.

*Slightly more complicated than that: it’s the top of the device stack. So,
it might be a Filter Device Object and not the actual media device

-scott
OSR
@OSRDrivers

DeviceExtension pointer of course stay untouched. you simply use wrong, not your, device
when you call IoVerifyVolume(DeviceObject) and it call IopMountVolume(DeviceObject) it do

// https://github.com/Zer0Mem0ry/ntoskrnl/blob/master/Io/iomgr/internal.c#L4801

attachedDevice = DeviceObject;
while (attachedDevice->AttachedDevice) {
attachedDevice = attachedDevice->AttachedDevice;
}

// https://github.com/Zer0Mem0ry/ntoskrnl/blob/master/Io/iomgr/internal.c#L4913

irpSp->Parameters.MountVolume.DeviceObject = attachedDevice;

as result in general case in Parameters.MountVolume.DeviceObject can be not your device. you need

DeviceToMount = IoGetDeviceAttachmentBaseRef(irpSp->Parameters.MountVolume.DeviceObject);

can of course do and IoGetLowerDeviceObject in loop, but not forget that every successful call to IoGetLowerDeviceObject must be matched by a subsequent call ObfDereferenceObject. same and for IoGetDeviceAttachmentBaseRef

you can check that device on your driver by

// PDRIVER_OBJECT g_DiverObject; // saved pointer to your driver object
if (DeviceToMount->DriverObject == g_DiverObject) {
// do something
} else {
// not my disk device, skip it
}

if you have several device types on driver you can design common device extension header and all device extension begin/inherit from this common header, so you can determinate concrete your device type

“none of those ‘lower device objects’ has a DeviceExtension that “is mine”.”

hard to say why. but dump not only device object pointer but driver name on which this device &DeviceObject->DriverObject->DriverName and device name, if exist (use ObQueryNameString) after this will be visible for which device stack you got IRP_MN_MOUNT_VOLUME.

but if you register file system device, you got IRP_MN_MOUNT_VOLUME not only for your created diskDeviceObject but for any another FILE_DEVICE_DISK created by somebody else

xxxxx@gmail.com wrote:

DeviceToMount = IoGetDeviceAttachmentBaseRef(irpSp->Parameters.MountVolume.DeviceObject);

can of course do and IoGetLowerDeviceObject in loop, but not forget that every successful call to IoGetLowerDeviceObject must be matched by a subsequent call ObfDereferenceObject. same and for IoGetDeviceAttachmentBaseRef

Ah right, that makes sense - and I have updated my code to do that.

hard to say why. but dump not only device object pointer but driver name on which this device &DeviceObject->DriverObject->DriverName and device name, if exist (use ObQueryNameString) after this will be visible for which device stack you got IRP_MN_MOUNT_VOLUME.

I added dumping the name as suggested, turns out that his system:

00000034 28.09723663 *** mount request for FFFFC20525342040 : minor
00000035 29.20065498 … going deeper FFFFC20525255A30
00000036 29.20070076 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume3’
00000037 30.48188400 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume4’
00000038 31.59117889 … going deeper FFFFC20525255C40
00000039 31.59121895 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume3’
00000040 32.87241364 … going deeper FFFFC20525248A30
00000041 32.87245560 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume4’
00000042 33.98175430 … going deeper FFFFC205252D6030
00000043 33.98178864 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume3’
00000044 35.26293945 … going deeper FFFFC20525248C40
00000045 35.26297760 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume4’
00000046 36.37228012 … going deeper FFFFC2052525CC70
00000047 36.37232208 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume3’
00000048 37.65350723 … going deeper FFFFC205252D4030
00000049 37.65354919 zfs_vnop_mount: given deviceName ‘\Device\HarddiskVolume4’
00000050 38.76288986 done dumping device names

I think possibly that he just has some devices/partitions not mounted, and
Windows will periodically retry them. So it is correct to ignore those
requests.

It actually came down to:

00000522 71.86611176 **** unknown disk Windows IOCTL: 0x70000

Ie, IOCTL_DISK_GET_DRIVE_GEOMETRY - which I had not implemented, as it was
listed as deprecated, and never observed this call on the VMs. But it was
required on (at least) his hardware, so now mounts work there as well.

Thank you for the assist!

Sincerely,


Jorgen Lundman |
Unix Administrator | +81 (0)90-5578-8500
Shibuya-ku, Tokyo | Japan

if you register file system device, you got IRP_MN_MOUNT_VOLUME not only for your created diskDeviceObject but for any another FILE_DEVICE_DISK created by somebody else. so that none of those ‘lower device objects’ not “is mine” - is not surprised. your file system device must be prepare handle IRP_MN_MOUNT_VOLUME from any disk device. what is unclear - why you combine in the same driver - file system functional and virtual disk creation ? for what you create FILE_DEVICE_DISK at all. and as side note, if create virtual disk, may be easy will be create PDO which return “GenDisk\0” for BusQueryCompatibleIDs. as result buil-it disk.sys create and attach self disk FDO for him

xxxxx@gmail.com wrote:

what is unclear - why you combine in the same driver - file system functional and virtual disk creation ? for what you create FILE_DEVICE_DISK at all. and as side note, if create virtual disk, may be easy will be create PDO which return “GenDisk\0” for BusQueryCompatibleIDs. as result buil-it disk.sys create and attach self disk FDO for him

Ah, well, to be honest - I don’t know. I am fresh to Win dev, coming from
Unix. Since userland needs to be in charge of what is mounted, where and
when, I followed the lead of the only example I could find.

But it also needs to be able to create ZVOL, virtual disk(s), which should
show as a normal disk that you can format NTFS etc. But that part does not
work, as the disks I create do not show in diskpart, Disk Manager etc. The
problem is further down on the list of things to port.

Lund


Jorgen Lundman |
Unix Administrator | +81 (0)90-5578-8500
Shibuya-ku, Tokyo | Japan

If you want your virtual disk to look as much like a real disk as possible, you need to write a StorPort Virtual Miniport.

-scott
OSR
@OSRDrivers

xxxxx@osr.com wrote:

If you want your virtual disk to look as much like a real disk as possible, you need to write a StorPort Virtual Miniport.

I guess the first question would be, do I? If you create a ZVOL, it should
show as a disk, which the user should be able to put NTFS / FAT on if they
want, and be assigned a volume as normal. Partitioning is not required
(some platforms allow it, some do not, so its more a question of how
complicated it would be on Windows).

How far would one have to go to be able to do that?


Jorgen Lundman |
Unix Administrator | +81 (0)90-5578-8500
Shibuya-ku, Tokyo | Japan

If your goal is to make the disk generically available as a disk (e.g. Disk
Management, WMI, Storage Spaces, etc.) then you need a StorPort Virtual
Miniport. Other ways to make the disk appear will work, but at some point
you’ll find SOMEthing that doesn’t work properly.

For example, you CAN just create a device of FILE_DEVICE_DISK, give it a
drive letter with DefineDosDevice, and implement a few disk IOCTLs that are
sent by the file systems and Shell. This gets you something that looks
enough like a disk for some applications, but you won’t find it in Disk
Management.

-scott
OSR
@OSRDrivers