CreateFile() handle problem

Say you’re iterating across the list of processes to close down their handles at the same time process A is duplicating a handle owned by process B. You go through A, go through all of its handles, find all the invalid ones and force them into some sort of zombie state (you can’t just close them because you don’t want them reopened until the app realizes they’re dead) and then move on to B after A dup’s B’s handle.

And even once you’ve closed all of the handles you’re still not there. Because closing the handle just triggers a cleanup, and you need an actual close so you know there aren’t any IRPs in flight at the time you unload the module. So now you have to wait for all of those handles to enter the closed state.

I’m sure it’s possible, but thinking about how to coordinate all the parts is already giving me a headache. Process rundown is complex enough and that’s one process running at a time all threads in the process have been terminated (so we know the state can’t be perturbed). Trying to do this to a live process seems difficult.

Balance that against the gain, which is that a kernel module can unload. What’s the big win? How much is it worth slowing down other aspects of the system (making handle dereference more expensive, making process list access more expensive, etc…) in order to allow this? I wouldn’t put much into it myself.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Michal Vodicka
Sent: Wednesday, March 26, 2008 12:14 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] CreateFile() handle problem

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Wednesday, March 26, 2008 3:18 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] CreateFile() handle problem

> Problematic but possible. OS could, for example, search
handle tables. Well, I see. You mean also pointer >
references. In this model only handles could be allowed.

There is no problem with maintaining “referee’s” list -
instead of simply incrementing reference on an object, the
system could add caller’s EPROCESS to the list of processes
that have an outstanding reference to an object in either
pointer or reference form. The question is how the OS has
to inform the processes that a given object is about to get
deleted. The only possible option is APC, so that the target
process has to register a callback with the OS. At this point
we have a logical question arises:

What is the reason to inform the process that a given object
got invalidated via a callback if it is going to find it out
when it tries to access it anyway???

Eh? I never said processes should be informed by OS about object
deletion. It is a problem you invented and finally found it is no
problem :slight_smile:

What I said was OS itself would close handles, invalidate entries in
handle table which’d be finally freed when handle owner closes them.

You’re right current implementation is correct and works. What I see as
a problem is the dependence of kernel modules on user mode apps
behaviour. Driver can’t commit suicide until all apps close all handles.
What I described would allow to make a cut and leave apps alone with
their handles. Which are useless in any case. It’d solve problem with
device change notification which is cumbersome at least.

Anyway, it is academic discussion. I only wanted to show Don current
design isn’t the only possible.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

You see it from the perspective of current design and implementation. No
wonder it’d be a bit complicated. Things would look differently if it
was one of original design goals. It’d probably influence IO manager
design and implementation. Currently it’d be probably unrealistic. As a
major problem I’d see pointer references. With them it wouldn’t make too
much sense.

As for the win, Max gave a good example with volume dismounts. There is
the same problem but a bit worse. When a volume is at a drive which is
surprise removed, you have to dismount regardless of apps which have
this volume opened (and apps can have the current directory there for
ages). There is forced dismount for this purpose and Vista FastFat
always uses it. BTW, is he right about searching handle tables?

For PnP drivers it is similar. Removed devices waiting for apps closing
their handles can be problematic. Maybe not in the ideal world when
everything works as expected but in reality their are. I encountered at
least one case when it caused USB reeumeration problems and we were
asked by MS support to close handle ASAP to solve it. It was probably OS
bug but they weren’t able to find other solution. I was really
uncomfortable with it then because my driver and OS behaviour became
dependent on correct app manners which are harder to ensure.

As for the worse performance, I don’t believe the loss would be even
remotely comparable with the loss caused by moving drivers to user mode
:wink: Sorry, I couldn’t resist. I just do battle with COM in UMDF driver
and grumble about people who made this choice :wink:

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Wieland
Sent: Thursday, March 27, 2008 2:02 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] CreateFile() handle problem

Say you’re iterating across the list of processes to close
down their handles at the same time process A is duplicating
a handle owned by process B. You go through A, go through
all of its handles, find all the invalid ones and force them
into some sort of zombie state (you can’t just close them
because you don’t want them reopened until the app realizes
they’re dead) and then move on to B after A dup’s B’s handle.

And even once you’ve closed all of the handles you’re still
not there. Because closing the handle just triggers a
cleanup, and you need an actual close so you know there
aren’t any IRPs in flight at the time you unload the module.
So now you have to wait for all of those handles to enter the
closed state.

I’m sure it’s possible, but thinking about how to coordinate
all the parts is already giving me a headache. Process
rundown is complex enough and that’s one process running at a
time all threads in the process have been terminated (so we
know the state can’t be perturbed). Trying to do this to a
live process seems difficult.

Balance that against the gain, which is that a kernel module
can unload. What’s the big win? How much is it worth
slowing down other aspects of the system (making handle
dereference more expensive, making process list access more
expensive, etc…) in order to allow this? I wouldn’t put
much into it myself.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Michal Vodicka
Sent: Wednesday, March 26, 2008 12:14 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] CreateFile() handle problem

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@hotmail.com
> Sent: Wednesday, March 26, 2008 3:18 AM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] CreateFile() handle problem
>
> > Problematic but possible. OS could, for example, search
> handle tables. Well, I see. You mean also pointer >
> references. In this model only handles could be allowed.
>
>
> There is no problem with maintaining “referee’s” list -
> instead of simply incrementing reference on an object, the
> system could add caller’s EPROCESS to the list of processes
> that have an outstanding reference to an object in either
> pointer or reference form. The question is how the OS has
> to inform the processes that a given object is about to get
> deleted. The only possible option is APC, so that the target
> process has to register a callback with the OS. At this point
> we have a logical question arises:
>
> What is the reason to inform the process that a given object
> got invalidated via a callback if it is going to find it out
> when it tries to access it anyway???

Eh? I never said processes should be informed by OS about object
deletion. It is a problem you invented and finally found it is no
problem :slight_smile:

What I said was OS itself would close handles, invalidate entries in
handle table which’d be finally freed when handle owner closes them.

You’re right current implementation is correct and works.
What I see as
a problem is the dependence of kernel modules on user mode apps
behaviour. Driver can’t commit suicide until all apps close
all handles.
What I described would allow to make a cut and leave apps alone with
their handles. Which are useless in any case. It’d solve problem with
device change notification which is cumbersome at least.

Anyway, it is academic discussion. I only wanted to show Don current
design isn’t the only possible.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online
at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online
at http://www.osronline.com/page.cfm?name=ListServer

> Eh? I never said processes should be informed by OS about object deletion. It is a

problem you invented and finally found it is no problem :slight_smile: What I said was OS itself would close handles,
invalidate entries in handle table which’d be finally freed when handle owner closes them.

An object may get safely deleted from RAM only when no one has an open reference to it either by pointer or by handle. It is understandable that it does not make sense to design a solution that works only for references by handles but not for the ones by pointers. The thing you speak about may work only with handles, so that its practical usefulness is zero. The only functionality that can make this imaginary API meaningful is informing processes about object’s destruction so that they release their references to it straight away, thus allowing the OS to delete invalidated object from memory. As I said, it can be done, but it will give processes absolutely unnecessary headache without any significant overall performance enhancement…

Anton Bassov

>Interesting. Can you be more specific? I can’t find it quickly in

FastFat sources. Thanks.

They change the file objects to dead state, after which any IRP coming to them
will fail.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

>always uses it. BTW, is he right about searching handle tables?

I do not understand why handle table search is needed anyway. Rendering the
file object to the “zombie” mode where all IRPs to it will be failed is what I
was speaking about.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim
S. Shatskih
Sent: Thursday, March 27, 2008 7:14 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] CreateFile() handle problem

>always uses it. BTW, is he right about searching handle tables?

I do not understand why handle table search is needed anyway.
Rendering the
file object to the “zombie” mode where all IRPs to it will be
failed is what I
was speaking about.

Yep, it seems as an equivallent solution. But you need to find all the
file objects some way. Do they keep the list?

Do you rememner the routine name which does it? I can’t find it quickly
in FastFat sources and don’t have time for deeper investigation just
now. Thanks.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

>Yep, it seems as an equivallent solution. But you need to find all the

file objects some way. Do they keep the list?

Yes.

Do you rememner the routine name which does it?

I think that dismount sets a flag in the VCB, which then causes
FatDecodeFileObject to fail.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com