How to make kernel Device type FILE_DEVICE_DISK to have visible Volume entry in mklink (mountvol)?

SoLaR · January 9, 2019, 11:12pm

Have written functional Virtual DISK driver for Windows7 (WDM driver model).
Driver creates DeviceNames: “\Device\VirtualDiskDriver\VirtDiskX”, where X is device number.
Later have created driver application that assigns DefineDosDeviceA(DDD_RAW_TARGET_PATH,DosDeviceName,DeviceName)
and sends commands (mount file, unmount file, status, …, counted 17 control messages so far…)
And all of this works fine with assigned DosDevice name (drive letter).
·
But what I want to implement is linux approach to mounting disks, floppies, cdroms, etc…
Instead of mapping to DosDevice (driver letter), I want to map to folder on physical disk/desktop/etc… (hdd0, hdd1, fd0, cd0, …)
Only thing I know that works like this in Windows is “mklink” command that if you know volume GUID you can map it to folder.
If you type mountvol at the end of command you can see the list of

    \\?\Volume{5c65a0da-2677-11e8-a513-806e6f6e6963}\
        C:\
    \\?\Volume{5c65a0de-2677-11e8-a513-806e6f6e6963}\
        A:\
    \\?\Volume{5c65a0dd-2677-11e8-a513-806e6f6e6963}\
        D:\
D: is CDROM

So you can type

    mklink /j cdrom0 \\?\Volume{5c65a0dd-2677-11e8-a513-806e6f6e6963}\

and mklink creates folder cdrom0 that is junction point and it behaves as normal folder in all applications, while in reality is using your CDROM (driver)…

Tried several things but came out short from wanted functionality.

Question: How can I make my Virtual DISK driver expose volume that can be visible by mklink (and mountvol) and behave in same manner as other disks (for starters), or if anyone has better idea how to accomplish the same task?

anton_bassov · January 10, 2019, 2:13am

In order to be in a position to allow something like that, Windows NT would have to implement VFS the way UNIX-like systems do, and to present the Object Manager to it as a special filesystem that is more or less similar to devfs on a UNIX-like system. If it did so, instead of using drive letters that correspond to different volumes (and, hence, are totally unrelated to one another), it would be able to present everything just a unified filesystem that allowed treating “//Device//XYY…” object directories just as regular ones in a filesystem, allowed creating the symbolic links to them from any location on the filesystem, as well as mounting the target block devices to the directories of your choice.

However, Windows NT does not have this unified approach. This is why it still uses drive letters C’,D et al. In this particular example, mklink works at the level of NTFS while mapping volumes to drive letters works at the level of the Object Manager. Although they both use the symbolic links, these are two totally different types of symbolic links that are totally unrelated to one another. Therefore, you are out of luck.

Anton Bassov

SoLaR · January 10, 2019, 5:16am

I am not sure you understood me, or maybe I am misunderstanding you.
All I want is to use my functional virtual disk driver (WDM) and not create dosdevice but volume (entry in mountvol like \?\Volume{a7358143-1488-11e9-8154-000c29c7bdf3}) to be mountable as JUNCTION POINT in NTFS on physical disk.

I have found sample driver which is KDMF sample driver that actually does this :
Virtual Volume Driver (WDKVirtualVolume)
https://code.msdn.microsoft.com/windowshardware/VirtualVolume-83334efd
Description sais it can create NTFS but i do not see it anywhere in code (MS is known to misled you).
.
anyway it creates volume mount point \?\Volume{a7358143-1488-11e9-8154-000c29c7bdf3}\ (which in output below is mounted on drive letter R:):

    \\?\Volume{5c65a0da-2677-11e8-a513-806e6f6e6963}\
        C:\
    \\?\Volume{a7358143-1488-11e9-8154-000c29c7bdf3}\
        R:\
    \\?\Volume{5c65a0de-2677-11e8-a513-806e6f6e6963}\
        A:\
    \\?\Volume{5c65a0dd-2677-11e8-a513-806e6f6e6963}\
        D:\
        C:\cdrom0\

And you can mount this volume with MKLINK /J VIRTVOL \?\Volume{a7358143-1488-11e9-8154-000c29c7bdf3}
As I’ve mounted CD-ROM volume to C:\cdrom0\ folder (obviously two different FS’s, but it works).
.
Unfortunately it’s FAT FS hardcoded and I am currently trying to strip of it, and make it formatable by Windows,
I’ve managed to force windows to ask me to format drive (if I try format it returns error), also if I try to format by cmd it sais:

C:\>format r:
Error in IOCTL call.

Need to figure out whats missing of IOCTRL codes to be able to format drive to any FS.
added logging of unhandled IOCTRL’s, and I am seeing alot of 002d and 0007 control base codes that aren’t implemented…
Or maybe it’s missing something in AddDevice procedure (I am total newbie regarding KDMF driver model need to get used to it to be able to read this sample properly…)
.
But it can be done, unfortunately different driver model…
If I can make work with image file instead of memory and to be able to format to any FS …
After that only thing left for me is decision to merge KDMF->WDM or vice-verso!?

maik_peterson · January 10, 2019, 10:09am

solar, if kdmf can do wdm can do. look into the kdmf implementation…

anton_bassov · January 10, 2019, 12:14pm

I am not sure you understood me, or maybe I am misunderstanding you.
All I want is to use my functional virtual disk driver (WDM) and not create dosdevice but volume (entry in mountvol like > ?\Volume{a7358143-1488-11e9-8154-000c29c7bdf3}) to be mountable as JUNCTION POINT in NTFS on physical disk.

I just got you wrong then…

Please ignore my previous post

Anton Bassov

SoLaR · January 11, 2019, 3:27am

It took me whole day to make virtvol sample format drive to NTFS.
After fixing whole bunch of MS-bugs, added bunch of IOCTRLS (lost track, need to compare to original code) unfortunately it no longer looks like original sample code
.
For posterity: crucial part was MEDIA_TYPE

// what a hack ?
#define VIRTVOL_MEDIA_TYPE              0xF8    

    //pDevExt->DiskGeometry.MediaType =  VIRTVOL_MEDIA_TYPE;
    pDevExt->DiskGeometry.MediaType =  FixedMedia;

I am not even sure what is type 0xF8, since it doesn’t exists or at least I was unable to find it in headers anywhere.
But the moment I’ve changed this windows offered me to formats drive to NTFS, FAT, and exFAT and driver stated receiving loads of IOCTRL’s finally.
Windows offered to format and I’ve set NTFS and drive formated successfully.

“Recycle.bin” is created on drive
but only 1/3 (16MB ram disk) of drive is empty, guessing old CLFS.sys friend made transaction log!?
Can this two behaviors of NTFS be suppressed somehow, or we must do it manually?
.
Also now I am experiencing same problem as with WDM version of driver:
devcon -remove virtvol.inf virtvol can’t remove virtual drive device that is mapped to (DosDevice) drive letter, requires reboot.
The device that has only mount point entry and not volume drive letter was successfully uninitialized and removed from the system.
Haven’t looked all IOCTRL’s was lock allowed or not or does devcon allows such behaviour, does anyone knows how to properly remove/eject/uninstall NTFS from (virtual) disk device to be able to remove driver without reboot? (think will post as separate topic). Notice: all was working fine while I was using FAT (restarted only 2 times whole day - those were BSOD’s).
.

Tim_Roberts · January 11, 2019, 6:39pm

SoLaR wrote:

It took me whole day to make virtvol sample format drive to NTFS.

Why would you do that? NTFS is not efficient for small volumes.

* “Recycle.bin” is created on drive
* but only 1/3 (16MB ram disk) of drive is empty, guessing old CLFS.sys friend made transaction log!?

And the master file table, and the redundant copies, and a number of
other overhead tables and files.

Can this two behaviors of NTFS be suppressed somehow, or we must do it manually?

Well, you can suppress that behavior by using FAT32 instead. It’s a much
better choice for small volumes.

Seriously, no. All of that overhead supports the features that make
NTFS what it is. The journaling features of NTFS are totally useless
on a RAM disk anyway. If there’s an interruption and reboot, the whole
drive is lost.

Also now I am experiencing same problem as with WDM version of driver:

devcon -remove virtvol.inf virtvol can’t remove virtual drive device that is mapped to (DosDevice) drive letter, requires reboot.

The device that has only mount point entry and not volume drive letter was successfully uninitialized and removed from the system.

Right. That’s how it works. You need to eject or “safely remove”
before you uninstall the driver.

SoLaR · January 12, 2019, 10:35pm

| SoLaR wrote:
| > It took me whole day to make virtvol sample format drive to NTFS.
|
| Why would you do that? NTFS is not efficient for small volumes.

I want to be able to format to anything, ie: tommorow if i decide to learn how to write FS driver for example I would like to know how to mount it on disk, storage, … and for example properly support formatting that fs.

| > * “Recycle.bin” is created on drive
| > * but only 1/3 (16MB ram disk) of drive is empty, guessing old CLFS.sys friend made transaction log!?
|
| And the master file table, and the redundant copies, and a number of
| other overhead tables and files.

Thanks for intell on CLFS.sys guess there is much more to read on subject - finding it is different story, guess need to check FS community group.

| > Can this two behaviors of NTFS be suppressed somehow, or we must do it manually?
|
| Well, you can suppress that behavior by using FAT32 instead. It’s a much
| better choice for small volumes.
|
| Seriously, no. All of that overhead supports the features that make
| NTFS what it is. The journaling features of NTFS are totally useless
| on a RAM disk anyway. If there’s an interruption and reboot, the whole
| drive is lost.

Yes I was thinking on it, also agree for RAM disks, but still want to stick for full support to learn, and I am using this only to test functionality, to learn how it all works in general - as it doesn’t require driver application to send IOCTL’s to mount/unmount image file (less hassle while testing).

And realized I have encountered same problem as in WDM version I previously wrote (when disk mounted and formatted to ntfs only way to replace driver was to reboot). In same time trying to learn KMDF along with solving problem with mountvol/mklink in WDM (which is solved as this driver can expose volume UUID which I can mount - figured out what I need to support/call).
While figuring out how to support mountvol/mklink made comparison of functionalities supported by one (WDM) and the other (KMDF) driver. And it seams most IOCTL’s are there for supporting virtual disk, and now I am rewriting KMDF to support file instead of RAM seams maybe better solution (cleaner since it would be my 3rd time writing the same). Added file structures, IOCTL’s and other stuff for file instead of RAM, I guess i could try linking now file instead of ram in adddevice and read/write procs, and see how it goes from there.
But it’s tedious needing to reboot each time…

|> Also now I am experiencing same problem as with WDM version of driver:
|>
|> devcon -remove virtvol.inf virtvol can’t remove virtual drive device that is mapped to (DosDevice) drive letter, requires reboot.
|>
|> The device that has only mount point entry and not volume drive letter was successfully uninitialized and removed from the system.
|
|Right. That’s how it works. You need to eject or “safely remove”
|before you uninstall the driver.

Yes I’ve spent part of day reading on eject, safely remove… and I am nowhere on that subject (have even more questions than answers).
Figured manually removing all mount points didn’t do the job (still requires reboot), so for the time focused energy and time to re-implementing functionalities of WDM before trying to figure this one out. But at least now I can see the light at the end of the tunnel

anyway thanks again for info

Tim_Roberts · January 14, 2019, 7:13am

On Jan 12, 2019, at 2:35 PM, SoLaR wrote:
>
> I want to be able to format to anything, ie: tommorow if i decide to learn how to write FS driver for example I would like to know how to mount it on disk, storage, … and for example properly support formatting that fs.

Fine. Your RAM disk supports NTFS. It just so happens that NTFS has very high overhead on a very small disk, but it’s working. Right?
—
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

SoLaR · January 15, 2019, 8:20pm

Well managed to be able to format my disk in FAT, FAT32 (not in all cases haven’t figured yet why), exFAT, and NTFS ( guess last two are enough or just might stick for the moment with exFAT if I figure out how to make my driver unload and not to be legacy driver in same time, then I could

start driver
mount volume,
…do what ever…
un-mount volume,
stop driver,
replace driver with new version and repeat .

This would speed up things not need to worry about reboots unless if I make some BSOD.

Tried to make PNP work but lack of proper docs from MS doesn’t help always end up with more and more questions. So spent day to implement it, and it was total disaster: loosing while day, learning almost nothing, and ending with not wanted functionality. It seems like MS documentation is written from perspective of filter driver, which is not what I am making, and from perspective of functional device controlling hardware which I am not as well.
Main questions I’ve posed more then several times, are:

should I pass PNP Irp to lower device, and what is that device since I do not have hardware?
If there is no lower device who completes Irp and now in such case Irp should be completed?
Some examples of PNP if lower device (next device) is not found they are failing PNP Irps (mostly QUERY), should i try to do same or not?

In effort to properly understand how OS sees things, or does things, managed to discover (to my surprise) working Windows clone capable of running Windows WDM drivers on it ie: ReactOS (it’s somewhere between w2k and xp), true manual installation of driver is required. Still in alpha stadium but it works. Have run it for few hours successfully with VMWare Windows Drivers, several browsers like mozilla opera some micro…, total commander, and other stuff, it just works. They have done reverse engineering of mostly everything from base Windows XP functionality, and there have found this piece of code:

https://doxygen.reactos.org/d3/dc5/mountmgr_8c_source.html

Where we can see step by step all IRP’s sent to our devices, and expected behavior on our responses. And it seams to be well documented for source code. Only need to figure our what is missing from Windows 7 perspective but some things are definitely more clearer when I’ve seen this piece of code.
Wonder what are others opinions on this OS and it’s source code, is usable in getting under the hood how things actually work in Windows?

Doron_Holan · January 15, 2019, 8:26pm

If you want functioning PNP for close to free, use KMDF. If you opt into pnp, you need power handling too (nearly free in KMDF as well)

d