Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Windows attaches thread to System process during cleanup???

Rade_TodorovicRade_Todorovic Member Posts: 4
I have seen this behaviour on Windows 7 and Windows 10 RS3, probably on everything in between.

When the OS closes a *kernel* handle, it seems to attach the current thread temporarily to the System process. (KeStackAttachProcess or equivalent internal routine.) If the handle happens to be a file handle, *the following IRP_MJ_CLEANUP irp is dispatched while still attached to System*!

1) Is it documented behaviour? It has subtle consequences (e.g. APCs scheduled on that thread cannot run while attached to a different process), so it looks like it *should* be documented, but I could not find any references.

2) Anyone has any explanation why the system does it? Or is it just a bug from some olden times, which cannot be fixed for compatibility reasons?

Many thanks,
R.

Comments

  • Ken_JohnsonKen_Johnson Member - All Emails Posts: 1,556
    As things happen to be implemented today, the kernel handle table referenced when OBJ_KERNEL_HANDLE is requested is the handle table assigned to the initial system process. It's not clear to me that there is any promise that this will always be true, however, so I would recommend not relying on this fact.


    In general, it is inadvisable to ascribe special significance to the process context for IRP_MJ_CLEANUP. There are many cases where this process may be more or less "arbitrary" with respect to the process that first opened a given handle. For example, if a process has a clone forked off by WER for purposes of writing a crash dump in parallel without blocking the "original" process throughout, and then "original" process is terminated and run down, and then (finally) the clone is terminated and run down, then the clone would be the process that sends IRP_MJ_CLEANUP if it still had a handle open at that point in time. Similar considerations apply to DuplicateHandle, handle inheritance to "normal" child processes, and so on.

    - S (Msft)

    -----Original Message-----
    From: xxxxx@lists.osr.com <xxxxx@lists.osr.com> On Behalf Of xxxxx@sophos.com
    Sent: Thursday, March 08, 2018 12:05 PM
    To: Windows File Systems Devs Interest List <xxxxx@lists.osr.com>
    Subject: [ntfsd] Windows attaches thread to System process during cleanup???

    I have seen this behaviour on Windows 7 and Windows 10 RS3, probably on everything in between.

    When the OS closes a *kernel* handle, it seems to attach the current thread temporarily to the System process. (KeStackAttachProcess or equivalent internal routine.) If the handle happens to be a file handle, *the following IRP_MJ_CLEANUP irp is dispatched while still attached to System*!

    1) Is it documented behaviour? It has subtle consequences (e.g. APCs scheduled on that thread cannot run while attached to a different process), so it looks like it *should* be documented, but I could not find any references.

    2) Anyone has any explanation why the system does it? Or is it just a bug from some olden times, which cannot be fixed for compatibility reasons?

    Many thanks,
    R.


    ---
    NTFSD is sponsored by OSR


    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at <https://na01.safelinks.protection.outlook.com/?url=http://www.osr.com/seminars&amp;data=01|01||e791700314ce46f8caaa08d5853151f4|f62b632944a24271bcc1ea45807ab854|1&amp;sdata=wJp+vGpCTgcseQzIhLbmBvkOThT7SMJbvp7toMwUw3s=&amp;reserved=0&gt;

    To unsubscribe, visit the List Server section of OSR Online at <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/page.cfm?name=ListServer&amp;data=01|01||e791700314ce46f8caaa08d5853151f4|f62b632944a24271bcc1ea45807ab854|1&amp;sdata=gdkpUr8rCYO1rpQJJG/3tvjJAxTjc/tOIM+X+2TAqLo=&amp;reserved=0&gt;
  • Juan_Antonio_BarrigasJuan_Antonio_Barrigas Member - All Emails Posts: 18
    Just reading ObOpenObjectByPointer API documentation:

    OBJ_KERNEL_HANDLE
    The handle can only be accessed in kernel mode. This flag must be specified if the caller is not running in the system process context.

    peprocess->ObjectTable

    Every process has it's handle table, system is not an exception. What is so wrong about cleanup been called in the context of system?
  • Rade_TodorovicRade_Todorovic Member Posts: 4
    @JuanAntonioBarrigas It is one thing if it can be called from a thread in the System process (or any other process) - if it happened to close the handle (or the last copy of a duplicated handle). I can understand that.

    It is another matter when it is, as it is, called from a thread belonging to a process which happens to have closed the handle, but *with that thread attached to the system process*. The system seems to internally call KeStackAttachProcess, then issue the cleanup irp, then call KeUnstackDetachProcess. This is contrary to Microsoft's own guidelines (e.g. see MSDN docs for KeStackAttachProcess, https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-kestackattachprocess):

    > Attaching a thread to a different process can prevent asynchronous I/O
    > operations from completing and can potentially cause deadlocks. In
    > general, the lines of code between the call to KeStackAttachProcess and
    > the call to KeUnstackDetachProcess should be very simple and should not
    > call complex routines *or send IRPs to other drivers*.

    Emphasis mine. Still, IRP_MJ_CLEANUP is sent while being attached to a different process. That is what is puzzling me.

    A related note, I would be hopeful that I can at least detect that situation (there is such API as KeIsProcessAttached) ... but this API is undocumented...
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,848
    <quote>
    It is another matter when it is, as it is, called from a thread belonging to a
    process which happens to have closed the handle, but *with that thread attached
    to the system process*
    </quote>

    How does this thread, which IS NOT a system thread, happen to have a kernel handle open and then happen to close that kernel handle? I'm curious.

    And why would closing that handle in the context of the system process NOT be reasonable in this case?

    And, no matter... As Mr. Johnson clearly described, you need to be able to detect that you're receiving a Cleanup in a context OTHER than that in which the handle was opened in any case. So... does this ordinary code not manage this case as well?

    Or am I missing a subtlety of your issue (which is entirely possible)?

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Juan_Antonio_BarrigasJuan_Antonio_Barrigas Member - All Emails Posts: 18
    My point was that windows kernel uses macros such as "PsGetCurrentProcess". It's description is: "Returns a pointer to the process of the current thread." Current thread's process might depend on current apc state.

    If you do a search in fastfat sources you'll see calls to this macro too.
  • Rade_TodorovicRade_Todorovic Member Posts: 4
    @JuanAntonioBarrigas I never looked before in detail what PsGetCurrentProcess did with respect to process attachment/detachment, but now I've had a look, and indeed it seems that the result of *that* call follows the process attachment/detachment (i.e. temporarily it would return the System PEPROCESS). In that respect, while attached to System, the thread resembles a system thread. Other routines, however, are unaffected (PsGetCurrentProcessId, for example returns the original, rather than attached, process ID). There isn't much documentation about which one does which.

    @PeterViscarola I've noticed this behaviour during the call to NtSetInformationFile (FileRenameInformation) from the user mode, when in addition to the user-mode handle passed for this call, the kernel temporarily opens another *kernel* handle to the rename target directory. (Didn't go as far to try to understand what it needs it for.) The whole NtSetInformationFile system call goes on synchronously, and it was a total surprise for me that I got my minifilter's cleanup callback (for that target directory) called while attached to System process. Surely I could get a similar surprise if I handled file I/O from a filter/minifilter "above" me, which could open/close system handles in the context of a user-mode process.

    In addition, it *can* cause subtle issues because APCs that are targeted at OriginalApcEnvironment don't run until detached. If, suppose, in my cleanup routine I wait for a user-mode service which happens to call something that depends on APCs working properly (e.g. NtGetContextThread for that very thread), I encounter a deadlock, which is exactly one of those things predicted by MSDN to happen if people put complex code between KeStackAttachProcess and KeUnstackDetachProcess.

    Now I am suspicious whether deadlock can happen in other circumstances too. For example, if I initiate some I/O in the cleanup routine (e.g. in pre-cleanup) - can I rely on it completing correctly, or will it cause a deadlock in some cases (assuming that the completion uses the APC mechanism)? To put it bluntly: in the cleanup routine, can I "call complex routines or send IRPs to other drivers", as MSDN puts it in that documentation for KeStackAttachProcess?
  • Juan_Antonio_BarrigasJuan_Antonio_Barrigas Member - All Emails Posts: 18
    Hi Rade,

    I understand your point, but you shouldn't assume that APCs are enabled in your minifilter callbacks, and before using APIs that might depend on APCs, you should check for "KeAreApcsDisabled()". This is documented as for "FltQueryInformationFile" just to name a case.
  • Gabriel_BerceaGabriel_Bercea Member - All Emails Posts: 464
    Expect to be more surprised in RS4.

    What you can and cannot do in a KeStackAttach - KeUnstackDetach combo
    depends on a lot more than a one line of documentation from Microsoft over
    there. I take that more of a general guideline of what I should do with my
    filter. Microsoft might have a bit more leverage in issuing this scenario.
    Nevertheless, to be completely honest this does not seem strange behavior
    to me at all. I've seen lot "worse".

    What I would be more worried is, why is my implementation depending on non
    arbitrary thread context in cleanup ? Now that I find a bit strange.
    If you need to do some cleanup "work" why do you care who's the last guy to
    close the handle ? Just do your work in a way that does not depend on that.

    You have filter comm ports if you need user mode intervention, and
    otherwise if it can be done fully in kernel.... You see my point.


    Cheers,
    Gabriel
    www.kasardia.com

    On Thu, Mar 8, 2018, 21:05 xxxxx@sophos.com
    wrote:

    > I have seen this behaviour on Windows 7 and Windows 10 RS3, probably on
    > everything in between.
    >
    > When the OS closes a *kernel* handle, it seems to attach the current
    > thread temporarily to the System process. (KeStackAttachProcess or
    > equivalent internal routine.) If the handle happens to be a file handle,
    > *the following IRP_MJ_CLEANUP irp is dispatched while still attached to
    > System*!
    >
    > 1) Is it documented behaviour? It has subtle consequences (e.g. APCs
    > scheduled on that thread cannot run while attached to a different process),
    > so it looks like it *should* be documented, but I could not find any
    > references.
    >
    > 2) Anyone has any explanation why the system does it? Or is it just a bug
    > from some olden times, which cannot be fixed for compatibility reasons?
    >
    > Many thanks,
    > R.
    >
    >
    > ---
    > NTFSD is sponsored by OSR
    >
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >

    Cheers,
    Gabriel

  • Rade_TodorovicRade_Todorovic Member Posts: 4
    @JuanAntonioBarrigas:

    > ... you shouldn't assume that APCs are enabled in your minifilter callbacks...

    Looking at the latest documentation, indeed, there are no guarantees whatsoever. I admit I *did* assume that cleanup will be synchronous, and that both the pre-operation and the post-operation will be called at PASIVE_LEVEL with APCs enabled (or that at least calling FltDoCompletionProcessingWhenSafe for the latter would make it happen that way). That was my mistake.

    Thanks a lot,

    R.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!