Namespace virtualization

Hi all,

I’m in the early stages of writing a minifilter that provide a virtualized namespace of files which are backed by remote files which can be fetched via some network protocol (like HTTP). That aside, I’m still processing what needs to be written since I’m no pro at minifilter development.

Since I’m virtualizing files that don’t exist virtually, I assume I’ll need to virtualize the handles that CreateFile returns. What’s the general approach tot his? One though I had was to create an event and use that handle, saving it in some sort of lookup table I can use in the callbacks. That feels a bit hacky though.

Also, I assume that I’ll need to implement file locking (via LockFile UM call) and sharing (like FILE_SHARE_WRITE in CreateFile) as well. Are there any facilities that can help me do this, or I’ll have to implement all these features myself?

Thanks!

> I’m in the early stages of writing a minifilter that provide a
virtualized namespace of files which are backed by remote files which can
be fetched via some network protocol (like HTTP).
For this goal FSD would be better solution.

Since I’m virtualizing files that don’t exist virtually
I can’t understand what you mean. If you’re giving access to remote files
in real-time then it’s network FSD. If you’re giving access in not
real-time then it’s something like a cloud you synchronize with.

I assume I’ll need to virtualize the handles that CreateFile returns.
Handles are created by I/O manager, not by FSD.

Also, I assume that I’ll need to implement file locking (via LockFile UM
call) and sharing (like FILE_SHARE_WRITE in CreateFile) as well.
It’s FSD responsibilities.

2016-10-29 3:34 GMT+03:00 :

> Hi all,
>
> I’m in the early stages of writing a minifilter that provide a virtualized
> namespace of files which are backed by remote files which can be fetched
> via some network protocol (like HTTP). That aside, I’m still processing
> what needs to be written since I’m no pro at minifilter development.
>
> Since I’m virtualizing files that don’t exist virtually, I assume I’ll
> need to virtualize the handles that CreateFile returns. What’s the general
> approach tot his? One though I had was to create an event and use that
> handle, saving it in some sort of lookup table I can use in the callbacks.
> That feels a bit hacky though.
>
> Also, I assume that I’ll need to implement file locking (via LockFile UM
> call) and sharing (like FILE_SHARE_WRITE in CreateFile) as well. Are there
> any facilities that can help me do this, or I’ll have to implement all
> these features myself?
>
> Thanks!
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

As a first project in the Windows kernel, you’ve picked a good one …
plan on about a year, or more, to get this working and stable. That
said, you’ll probably need to go with a layered file system if these
files are being virtualized from nothing on the local system. What I
mean is that you could go with a stub file approach where you have a
small stub file which resides on a local file system and you are backing
the data with your remote content. This allows you to pass along a lot
of the name space handling to the underlying file system but you would
still be advised to go with a layered FS.

If you are not going in this direction, but instead virtualizing the
entire name space then you’ll need to start with 1) a virtual volume
design to get your layered file system loaded, 2) a layered file system
to virtualize the entire name space and 3) a lot of time and patience to
get this up and running.

There are lots of references to layered file system in this forum as
well as a handful of blogs on the web which describe them. If you have
little or no understanding of file systems in general, start with
Nagar’s book. It’s old but will give you an idea of what a file system
on Windows is all about though there are many aspects not covered in his
book due to it’s age. Then take a look at the FastFat source code in the
latest WDK. If you are new to Windows kernel development then start with
a couple of classes such as those offered by OSR.

Good luck!

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

------ Original Message ------
From: xxxxx@outlook.com
To: “Windows File Systems Devs Interest List”
Sent: 10/28/2016 6:34:42 PM
Subject: [ntfsd] Namespace virtualization

>Hi all,
>
>I’m in the early stages of writing a minifilter that provide a
>virtualized namespace of files which are backed by remote files which
>can be fetched via some network protocol (like HTTP). That aside, I’m
>still processing what needs to be written since I’m no pro at
>minifilter development.
>
>Since I’m virtualizing files that don’t exist virtually, I assume I’ll
>need to virtualize the handles that CreateFile returns. What’s the
>general approach tot his? One though I had was to create an event and
>use that handle, saving it in some sort of lookup table I can use in
>the callbacks. That feels a bit hacky though.
>
>Also, I assume that I’ll need to implement file locking (via LockFile
>UM call) and sharing (like FILE_SHARE_WRITE in CreateFile) as well.
>Are there any facilities that can help me do this, or I’ll have to
>implement all these features myself?
>
>Thanks!
>
>—
>NTFSD is sponsored by OSR
>
>
>MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>software drivers!
>Details at http:
>
>To unsubscribe, visit the List Server section of OSR Online at
>http:</http:></http:>