registry and fs virtualization

all,

i am being asked to design a system virtualizer which can isolate IOs from
specific proceses and system changes related to them so that they can be
removed later on.

yes there are several such products, but still this is the requirement.

can you give me some pointers as to what kind of drivers i would need to do
it, if there are any exmaples/samples, as to where i can start. i know i
need the registry also in this but i dont know of any layering techniques
to filter registry, unless one hooks which is not an advisable idea…

Look at minispy and regfiltr. All I can say is this is going to be a
huge amount of work, especially if you have to ask what driver types you
need.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Bedanto” wrote in message news:xxxxx@ntdev:

> all,
>
> i am being asked to design a system virtualizer which can isolate IOs from
> specific proceses and system changes related to them so that they can be
> removed later on.
>
> yes there are several such products, but still this is the requirement.
>
> can you give me some pointers as to what kind of drivers i would need to do
> it, if there are any exmaples/samples, as to where i can start. i know i
> need the registry also in this but i dont know of any layering techniques
> to filter registry, unless one hooks which is not an advisable idea…

On 26-Apr-2013 22:43, Bedanto wrote:

all,
i am being asked to design a system virtualizer which can isolate IOs
from specific proceses and system changes related to them so that they
can be removed later on.
yes there are several such products, but still this is the requirement.
can you give me some pointers as to what kind of drivers i would need to
do it, if there are any exmaples/samples, as to where i can start. i
know i need the registry also in this but i dont know of any layering
techniques to filter registry, unless one hooks which is not an
advisable idea…

Read the brilliant post of Dr. Newcomer, earlier today…
(thread: How to detect activity from Windows Explorer from net share)

– pa

Wow… That IS a good post. That’s gotta be in the running for Post of the Week!

Peter
OSR

The “only” problem is that above mentioned post is totally unrelated to the topic of a discussion…

The problem that the original poster ob the above thread was trying to solve is unsolvable per se. What the OP here speaks about is, basically, a virtualization scheme that is known as Jaisl and Zones under respectively FreeBSD and Solaris, and this scheme is anything but " p-baked (p < 0.05) Security" hack" - instead, it has proved to work quite well. Although such a solution is rather problematic in the Windows world, it is still not infeasible one. Just to give you a tip, go and check the Bromium site - they are approaching this problem via hardware-assisted virtualization…

Anton Bassov

That the concept has validity is not in question. That the concept is
compatible with the Windows security model would be an incorrect assertion
to put it mildly. Whatever might be possible in other operating systems,
Joe is correct about the applicability to Windows

wrote in message news:xxxxx@ntdev…

The “only” problem is that above mentioned post is totally unrelated to
the topic of a discussion…

The problem that the original poster ob the above thread was trying to solve
is unsolvable per se. What the OP here speaks about is, basically, a
virtualization scheme that is known as Jaisl and Zones under respectively
FreeBSD and Solaris, and this scheme is anything but " p-baked (p < 0.05)
Security" hack" - instead, it has proved to work quite well. Although such
a solution is rather problematic in the Windows world, it is still not
infeasible one. Just to give you a tip, go and check the Bromium site - they
are approaching this problem via hardware-assisted virtualization…

Anton Bassov

On 28-Apr-2013 00:49, xxxxx@hotmail.com wrote:

The problem that the original poster ob the above thread was trying to solve is unsolvable per se.
>
What the OP here speaks about is, basically, a virtualization scheme
that is known as Jaisl and Zones under respectively

FreeBSD and Solaris, and this scheme is anything but " p-baked (p < 0.05) Security" hack" - instead,

it has proved to work quite well. Although such a solution is rather problematic in the Windows world, it is still not infeasible one.

Just to give you a tip, go and check the Bromium site - they are approaching this problem via hardware-assisted virtualization…

The next snake oil, IMHO. Have they released a practically usable
version?
Basic isolation of processes is not so hard.
VMware’s ThinApp (and something similar from MS) basically does this in
software. A hypervisor based solution of course is more effective.
But, as you’ve said, the problem of Windows is that it is not Linux, BSD
or whatever. Namely, process isolation in Windows is much harder because
every interesting app interacts with OS services and other components,
for example, via DCOM.
Not that I know much about virtualization. I am a pure hardware type,
but not long ago had to take a brief detour into that sort of a project.
It was enough to understand what it takes to isolate a non-trivial
Windows process.

On the other hand, it may be feasible to write software in a way
suitable for this isolation model - but this will be a significant
effort - like what MS has done to isolate “administrator” parts of lot
of Windows components, and make the “protected mode” IE.
But this yet has to be done…

– pa

> That the concept is compatible with the Windows security model would be an incorrect assertion

to put it mildly.

Sorry, but the same is true for so-called '“whitelisting” - although access permission bitmap is one of basic cornerstones of the UNIX security model, it is a foreign concept in the Windows world. Ironically, somehow it happens that these third-party “unsupported” whitelisting-based products offer, probably, the most reliable,up to this point, solution to the Windows security…

Certainly, the thing that the OP wants to do requires significantly more tampering with the system than implementation of '“whitelisting” does - this is out of question. However, don’t forget that we are living in the age of hardware-assisted virtualization. When it comes to “gory details”, you can think of the sandbox that the OP plans to implement simply as of a guest kernel that has no access either to the physical resources of the host nor to its registry.This kernel can be patched, modified and messed around with in any way the OP wishes without any chance of damaging the actual machine. I guess hardware-assisted virtualization is the very first direction that the OP should investigate…

Whatever might be possible in other operating systems, Joe is correct about the applicability to Windows

This is not about Joe’s statement - it is about Pavel’s reference to it. Joe’s statement is certainly true in the context Joe made it. However, I am not sure the same holds true in context Pavel referred it to…

Anton Bassov

On 28-Apr-2013 03:25, xxxxx@hotmail.com wrote:

This is not about Joe’s statement - it is about Pavel’s reference to it. Joe’s statement is certainly true in the context Joe made it.

However, I am not sure the same holds true in context Pavel referred it to…

In the very specific context of today’s and yesterday’s Windows and
legacy 3rd party desktop software.

No doubt that MS has plans for hardware-assisted virtualization, more
over - they have power to influence designs of hardware makers.
But it will be tomorrow, and the security crisis is now,
and we all are here bogged with WinXP, Win7 and legacy apps.
Smart folks use this situation to raise some investment money - good
for them… but most of these (if not all) actually are, as Dr. Newcomer
wrote, “p-baked hacks”.

/* the Physics dept. of our uni received a perpetuum mobile project
every couple of months when I’ve studied. Professors took turns to find
the “bug” and politely reply to every inventor… yes, there were times.
So the fashion has changed… */

– pa

>> However, I am not sure the same holds true in context Pavel referred it to…

In the very specific context of today’s and yesterday’s Windows and legacy 3rd party
desktop software. No doubt that MS has plans for hardware-assisted virtualization,
more over - they have power to influence designs of hardware makers. But it will be tomorrow,
and the security crisis is now, and we all are here bogged with WinXP, Win7 and legacy apps.
Smart folks use this situation to raise some investment money - good for them…
but most of these (if not all) actually are, as Dr. Newcomer wrote, “p-baked hacks”.

Sorry, Pavel, but this is simply nonsense…

Look - the ONLY reason why hardware-assisted virtualization is useful here is because you have no access
to the Windows kernel sources, and, hence, are forced to patch the kernel at the runtime. Doing it with the one that controls your physical machine may be dodgy. Therefore, the idea may be patching a kernel of a guest system - even if it goes badly, there will be no damage to the physical machine. The whole thing is entirely consistent with the very idea of a sandbox - normally you want to sandbox only those apps that you don’t really trust, right. Therefore, there is nothing to worry about even if something goes wrong from time to time - although you may rely upon “p-baked hacks” where p<0.5 as far as the guest is concerned,
from the host’s perspective, ’ if( p==1) ’ statement evaluates to TRUE …

Unlike you and me, MSFT has a full access to the kernel sources. Therefore, if it had any intention to implement the concept of Jails/Zones under Windows it would certainly do it in a software, without having to resort to ANY hardware-assisted virtualization, and certainly without having to deal with hardware makers.
Therefore, this is NOT the question of “tomorrow” - this is the question of the third-party solutions addressing the problem that MSFT, for this or that reason, does not want to address. Nothing particularly new here, don’t you think…

Anton Bassov

> [quote]

Read the brilliant post of Dr. Newcomer, earlier today… (thread: How to
detect activity from Windows Explorer from net share)

[/quote]

The “only” problem is that above mentioned post is totally unrelated to
the topic of a discussion…

The problem that the original poster ob the above thread was trying to
solve is unsolvable per se. What the OP here speaks about is, basically, a
virtualization scheme that is known as Jaisl and Zones under respectively
FreeBSD and Solaris, and this scheme is anything but " p-baked (p < 0.05)
Security" hack" - instead, it has proved to work quite well. Although such
a solution is rather problematic in the Windows world, it is still not
infeasible one. Just to give you a tip, go and check the Bromium site -
they are approaching this problem via hardware-assisted
virtualization…

Actually, one of these scenarios turns out to be that some management
dweeb /heard/ about the idea in system and therfore assumes it
is doable in Windows. I don’t know the features of those systems, but I
dealt with Unices for 15 years and not once encountered anything like the
Registry.

And hardware-assisted virtualization is also not relevant to the question,
any more than the successful implementation of in
is relevant. IBM’s AIX on the RISC/6000 series
had a fully transacted file system, and their editor had the property that
if the system crashed, after it rebooted it would put you back in the file
where you were editing and lose at most one keystroke. So if someone comes
along and says “Now that Windows has a transacted file system, I want you
to write VBA extensions to Word that give it that behavior” the existence
proof that AIX implemented it does not mean it is possible to make Word do
it.

Which always returns to te root question: exactly what problem is this
supposed to solve?

Dweeb says, “I hear that modifying the Registry can compromise security,
so I want you to make a virtual Registry that will make it possible to
undo changes”. Now, most of the Registry tricks I know of involve HKLM
being compromised. But this is not a problem, because only privileged
users can do this, and ordinary users are running without the rights to
modify HKLM. So, another rumor is "You can modify HKCU in such a way that
. But HKCU is the root of a number of subtrees. Do you
know which ones can cause problems? I started by protecting all of the
Registry against modification by anyone running as a normal user. Then I
determined which keys were expected to be modified, and opened those up to
the restricted user account. After a week of fiddling, my Registry was
secure. Most management dweebs (and far too many Windows sysadmins and
programmers) do not know that the Registry provides rather fine-grain
access control via ACLs (alas, ACLs are the “assembly code” of security,
and we really should have a 5GL to manage security, but I digress). Now,
what does it mean to “undo changes”? Is it the all-or-nothing approach
like Windows reset points, or is it per-key or per-value? What does the
interface to it look like? Hint: do NOT say “we’ll burn that bridge when
we cross it”; if you don’t know what you are going to present to the user,
how doyou know what information you need to save? (Seriously, I ended up
with a catastrophe because the implementors of an XML-like system didn’t
think about the symbolic I/O reader/writer issues while writing the
middleware part of the design. I was tasked to fix the problem in 3
months. It took me 4 months to completely re-implement it, and 12 months
to recover me).

It isn’t enough to say “OK, you start coding, and I’ll go up and find out
what the requirements are”. If you don’t know what problem you’re trying
to solve, how do you know that a collection of ever-more-complex hacks is
going to be the solution.

I currently run my server with a offsite-backup system that images files
out to a remote server each time they are written. I can exclude files
like .obj, .ilk, .pdb, and similar large-but-otherwise-re-creatable files.
But it is not a mirrored file system; if I need this morning’s, or last
Tuesday’s, version of the file, I can go to the offsite backup server and
retrieve it, and bring it back with or without replacing the current
version of the file. So unless you are actually developing such a
product, you don’t need to create it; it already exists, and it would be
far cheaper to license than re-create it. Now the Registry hives are just
files. How would having them go to an archiving server not give you what
you need? (There are probably several good reasons, but you would need to
show how this form of virtualization fails to accomplish the goal,
whatever it is).

Another problem is “I’m just a lowly programmer trying to do the project
assigned to me. These decisions were made far above my pay grade.” This
is the “I was just following orders” defense, and it doesn’t work too
well. Anyone who creates such designs without talking to the people who
know what’s really going on in the OS is not going to come up with a valid
design. And tasking programmers to implement solutions to unsolvable
problems is actually another example of management failure. I spent a lot
of time finding out the requirements for whatever bizarre task I was
given, and sometimes the answer was “that can’t be done”. But I could
also say /why/ it wasn’t possible. So whatever the resulting code is, it
should only exist if there is a need for it. And I have this nagging
memory that Vista already allowed virtualization of the Registry, because
the original idea that values like HKEY_CLASSES entries were necessarily
global to the entire machine; but in my case, I want to edit a JPG file
using CorelPaint, and you want to use Adobe PhotoShop, and Fred, the third
person who can log onto this machine, wants to use Microsoft Paint. So
you need to understand how this virtualization works, and whether or not
you are trying to duplicate it.

One local school I have dealt with keeps read-only copies of VMs which
students can load (and that’s the only thing their extremely limited login
on the host machine allows). They can save the VM in their own area, but
if it becomes corrupted, they just start another one of the master copies
(they have several different flavors) and they’re back where they started.
In September. If these were to image to an offsite backup, files could
be retrieved, so work would not be lost.

So it is important to understand the requirements (and “virtualize the
Registry and file system” is NOT a requirement; it is an implementation
strategy). Until you understand the requirements, you cannot possibly
know if your work is going to meet them.
joe
>
>
> Anton Bassov
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Joe,

Sorry, but the keyphrase to your entire post (in fact, to quite a few of your posts) is " I’ve got NO FUCKING IDEA about the problem in question - I don’t even fully understand its domain, and simply repeat the same mantra again and again and again in an expectation that it will apply on this particular occasion"…

Dweeb says, “I hear that modifying the Registry can compromise security, so I want you to
make a virtual Registry that will make it possible to undo changes”.

And hardware-assisted virtualization is also not relevant to the question, any more than
the successful implementation of in is relevant.



> Which always returns to te root question: exactly what problem is this supposed to solve?



> So it is important to understand the requirements (and “virtualize the Registry and file system”
> is NOT a requirement; it is an implementation strategy). Until you understand the requirements,
> you cannot possibly know if your work is going to meet them.

Which just proves that my statement above is entirely correct, don’t you think. You just don’t understand the domain of the problem, and, hence, cannot understand how this problem may be addressed

Anton Bassov

Anton,

The problem is that the way the original question was asked by the
OP indicates he does not know Windows. So without know the overall goal
of the project one has to expect that the simple minded question is
missing half of the problems with whatever they are doing on Windows.
Yes, you are correct that one can do useful products by isolating and
recording registry and file system changes, I’ve done the kernel work on
some of these for Windows products. I can think of software that has
been doing this since NT3.5.1.

But I have also been called in to a lot of products, where people
spent a ton of time, grabbing filemon or filespy creating their log,
then running into problems because they did not know Windows. Many of
these failures are by groups that are populated by people who came from
other operating system backgrounds, and expect that the system and the
application work their way. I’ve stayed out of the argument till now,
since both sides have valid points, the OP may have a valid product in
mind, but it is also extremely likely that since he asked what types of
drivers he needed, that his project is making a pig fly as Mr Viscarola
so eloquently described in a pontification.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

xxxxx@hotmail.com” wrote in message
news:xxxxx@ntdev:

> Joe,
>
>
> Sorry, but the keyphrase to your entire post (in fact, to quite a few of your posts) is " I’ve got NO FUCKING IDEA about the problem in question - I don’t even fully understand its domain, and simply repeat the same mantra again and again and again in an expectation that it will apply on this particular occasion"…
>
>
> > Dweeb says, “I hear that modifying the Registry can compromise security, so I want you to
> > make a virtual Registry that will make it possible to undo changes”.
>
> …
>
>
> > And hardware-assisted virtualization is also not relevant to the question, any more than
> > the successful implementation of in is relevant.
>
> …
>
> > Which always returns to te root question: exactly what problem is this supposed to solve?
>
> …
>
>
> > So it is important to understand the requirements (and “virtualize the Registry and file system”
> > is NOT a requirement; it is an implementation strategy). Until you understand the requirements,
> > you cannot possibly know if your work is going to meet them.
>
>
>
>
> Which just proves that my statement above is entirely correct, don’t you think. You just don’t understand the domain of the problem, and, hence, cannot understand how this problem may be addressed
>
>
>
> Anton Bassov

Don,

The problem is that the way the original question was asked by the OP indicates he does not know Windows.

Well, the OP is just a VERY special case of a poster - normally you would not expect a question like that from someone who has been asking questions on NTDEV for the last 8 years (you can check the archives and see his posting history),would you. Therefore, I don’t want to even question his plainly obvious inability to handle a project like of this complexity level.

What I do object to here is dumbly repeating certain mantras, without even understanding the problem per se…

In fact, after having thought about the whole thing a bit, I already see the solution, which can be described
simply as a “generic SSDT virtualization framework” - by combining a KVM-like kernel component with few UM libraries you can simulate SSDT hooking for a target process, effectively making it believe in anything you wish, and do all the above without going anywhere close to the actual SSDT. At this point you can already implement a sandbox, i.e. do it on top of this framework…

Anton Bassov

>implement the concept of Jails/Zones under Windows

Parallels Virtuozzo was exactly the Jails/Zones for Windows. Hack upon a hack, you need to update Virtuozzo ASAP after some Windows updates :slight_smile: or the OS will not work at all, maybe even crash.

For now, Parallels is going into full real virtualization.

Jails/Zones are obsolete. For now, full virtualization is the way to go, given it has neglectable perf penalty.

Also, Jails/Zones for Windows are problematic due to lots of app server software accessed by the UI apps via DCOM or RPC. UNIXen also have similar things like “esd” (Enlightenment Sound Daemon), but this is not so popular in UNIXen.


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

>Jails/Zones are obsolete. For now, full virtualization is the way to go, given it has neglectable perf penalty.

Well, it depends on your objectives - you don’t really need to buy a cow if you just need a glass of milk, even if this cow is very cheap, don’t you think. As far as a typical home user running some stripped-down home edition of Windows that has been pre-installed on his/her computer is concerned, you can think of an extra OS image that full virtualization approach implies, as of this cow…

Also, Jails/Zones for Windows are problematic due to lots of app server software accessed
by the UI apps via DCOM or RPC.

Well, the very idea of Jails/Zones seems to be more appropriate for low-end workstation, rather than server, don’t you think. When it comes to the server space full virtualization, indeed,seems to be the right way to go…

Anton Bassov

> Well, the very idea of Jails/Zones seems to be more appropriate for low-end workstation, rather than

server

No, it’s much more appropriate for server, since it allows creation of several virtual servers with some management stuff (like firewall) in common.


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

> No, it’s much more appropriate for server, since it allows creation of several virtual servers

with some management stuff (like firewall) in common

Well, we seem to be going a bit OT already, but there are different objectives that may be associated with different levels of virtualizatiion. We had a discussion on NTTALK about it already, and it was started off by my proposal to discuss Theo de Raadt’s opinion on the subject ( http://kerneltrap.org/OpenBSD/Virtualization_Security).

I guess we should continue this discussion on NTTALK - otherwise, the whole thing will naturally evolve (or,maybe, ‘degrade’ is more appropriate term for that ) into yet another "long rambling thread"on NTDEV. We don’t really need it, do we…

Anton Bassov

Dr. Newcomer wrote Sunday, April 28, 2013 12:34 PM:

I don’t know the features of those systems, but I
dealt with Unices for 15 years and not once encountered anything like
the Registry.

What about AIX ODM ?

Best regards,
Alex Krol

I never saw that. It may well have existed in AIX, but (a) the project I
was on ported to 30 different flavors of Unix, so there was no motivation
to use platform-specific feature (and consequently, no motivation to find
out about them) and (b) I only used AIX via a graphics shell and only used
it about six months. That was the year IBM lost $5,000,000,000 and one of
the victims of their cost-cutting was our research project. I and about
30 others became unemployed in December.

So a Registry-like feature may have been on the one Unixoid I briefly
used, but that was the only one.
joe

Dr. Newcomer wrote Sunday, April 28, 2013 12:34 PM:

> I don’t know the features of those systems, but I
> dealt with Unices for 15 years and not once encountered anything like
> the Registry.

What about AIX ODM ?

Best regards,
Alex Krol


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer