Thread termination in a Win2K driver

I have an inherited project that’s giving me fits. It’s a driver
originally developed on Win98, which I’m moving to Win2K. The problem is
that the driver creates a worker thread (using PsCreateSystemThread()).
When the driver exits (it’s an IEEE1394 driver, so the plug might be
pulled while the computer is up), I get

*** Fatal System Error: 0x000000ce

and ‘!analyze -v’ tells me the driver “unloaded without cancelling timers,
DPCs, worker threads, etc.”

I’ve narrowed it down to this worker thread. If I never start the thread,
I can exit without incident.

The thread does call PsTerminateSystemThread() as it exits, and KdPrint
statements seem to show that the cleanup routines are being called and
completing. The main driver thread waits on an event (which the worker
thread signals as it exits) before completing the StopDevice code, so
everything looks right. Yet the machine still crashes, whether in
response to a surprise removal or from a stop command from the ‘Unplug or
Eject Hardware’ dialog.

Any clues you guys have to offer will be gratefully accepted.

Roy M. Silvernail
Software Engineer
Acroloop Motion Control Systems, Inc.
xxxxx@acroloop.com

I would change the wait logic to wait for the thread to terminate. One
trick for drivers is to have the thread that is waiting to drop its priority
to the lowest and increase the other thread(s) to the highest. You can do a
KeWaitForSingleObject on a thread also and that is the simplest solution.

----- Original Message -----
From: “Roy M. Silvernail”
To: “NT Developers Interest List”
Sent: Monday, March 25, 2002 5:21 PM
Subject: [ntdev] Thread termination in a Win2K driver

> I have an inherited project that’s giving me fits. It’s a driver
> originally developed on Win98, which I’m moving to Win2K. The problem is
> that the driver creates a worker thread (using PsCreateSystemThread()).
> When the driver exits (it’s an IEEE1394 driver, so the plug might be
> pulled while the computer is up), I get
>
> *** Fatal System Error: 0x000000ce
>
> and ‘!analyze -v’ tells me the driver “unloaded without cancelling timers,
> DPCs, worker threads, etc.”
>
> I’ve narrowed it down to this worker thread. If I never start the thread,
> I can exit without incident.
>
> The thread does call PsTerminateSystemThread() as it exits, and KdPrint
> statements seem to show that the cleanup routines are being called and
> completing. The main driver thread waits on an event (which the worker
> thread signals as it exits) before completing the StopDevice code, so
> everything looks right. Yet the machine still crashes, whether in
> response to a surprise removal or from a stop command from the ‘Unplug or
> Eject Hardware’ dialog.
>
> Any clues you guys have to offer will be gratefully accepted.
> –
> Roy M. Silvernail
> Software Engineer
> Acroloop Motion Control Systems, Inc.
> xxxxx@acroloop.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@yoshimuni.com
> To unsubscribe send a blank email to %%email.unsub%%

Roy,
i guess you created a timer to do something periodically in your working thread, and fail to destroy it before you quit. Just check…

----- Original Message -----
From: “Roy M. Silvernail”
To: “NT Developers Interest List”
Sent: Tuesday, March 26, 2002 6:21 AM
Subject: [ntdev] Thread termination in a Win2K driver

> I have an inherited project that’s giving me fits. It’s a driver
> originally developed on Win98, which I’m moving to Win2K. The problem is
> that the driver creates a worker thread (using PsCreateSystemThread()).
> When the driver exits (it’s an IEEE1394 driver, so the plug might be
> pulled while the computer is up), I get
>
> *** Fatal System Error: 0x000000ce
>
> and ‘!analyze -v’ tells me the driver “unloaded without cancelling timers,
> DPCs, worker threads, etc.”
>
> I’ve narrowed it down to this worker thread. If I never start the thread,
> I can exit without incident.
>
> The thread does call PsTerminateSystemThread() as it exits, and KdPrint
> statements seem to show that the cleanup routines are being called and
> completing. The main driver thread waits on an event (which the worker
> thread signals as it exits) before completing the StopDevice code, so
> everything looks right. Yet the machine still crashes, whether in
> response to a surprise removal or from a stop command from the ‘Unplug or
> Eject Hardware’ dialog.
>
> Any clues you guys have to offer will be gratefully accepted.
> –
> Roy M. Silvernail
> Software Engineer
> Acroloop Motion Control Systems, Inc.
> xxxxx@acroloop.com
>
>
> —
> You are currently subscribed to ntdev as: KDriver@163.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

>The thread does call PsTerminateSystemThread() as it exits, and KdPrint

statements seem to show that the cleanup routines are being called and
completing. The main driver thread waits on an event (which the worker
thread signals as it exits) before completing the StopDevice code, so
everything looks right. Yet the machine still crashes, whether in
response to a surprise removal or from a stop command from the ‘Unplug or
Eject Hardware’ dialog.

There is a time window after the thread has signaled the “I’m exiting”
event and the thread is actually gone. You need to wait on the actual
thread object to know when the thread has terminated.

A nasty bug that can happen in DLL’s or drivers is the code for the
DLL/driver get’s unloaded but the thread may still try to execute code at
what used to be valid addresses. The easy to find case is the thread tries
to execute invalid addresses, the harder case to find is the thread
executes some code that is now loaded where the old code used to be. This
could be viewed as a bug in the thread creation API’s, as they don’t take
a parameter that creates a reference to the module. It could also be viewed
as bug in the OS, which doesn’t figure out what code module a thread
function is in, and assure that module doesn’t get unloaded until the
reference from the thread goes away.

For user-mode DLL’s, one strategy is for a DLL that creates a thread to
call LoadLibrary on itself, incrementing it’s reference count. The thread
exit code then calls the API FreeLibraryAndExitThread, which decrements the
reference count on the calling module and never returns, exiting the
current thread. I can’t offhand think of any driver equivalent.

  • Jan

>I would change the wait logic to wait for the thread to terminate. One

trick for drivers is to have the thread that is waiting to drop its priority
to the lowest and increase the other thread(s) to the highest.

Increasing the priority for a threat that’s exiting and lowering priority
for a thread that needs to run after the higher priority thread will not
always work. The thread with the raised priority could take a page fault
and suspend, making the lower priority thread the highest priority runnable
thread. Adjusting thread priorities will NOT achieve synchronization.

  • Jan

Windos9x/me has no safe way to terminate a thread. NT on the other hand
has a simple and yet DDK-undocumented mechanism to safely wait for
thread termination.

  1. Use ObReferenceObjectByHandle to obtain a pointer to the thread
    object you created when you called PsCreateSystemThread. This pointer is
    ‘safe’ as the thread object reference count has been incremented to
    account for its existance. Call ZwClose to dereference the thread handle
  • you don’t need it.
  1. Now inform your worker thread through whatever mechanism you choose
    (an event, a global flag, whatever,) that it is time to die.

  2. Then simply call KeWaitForSingleObject using the object pointer you
    obtained from ObReferenceObjectByHandle. When your thread really has
    terminated the object will be signalled (and not destroyed as you have
    its reference count incremented.)

  3. On return from KeWait you need to dereference the object by calling
    ObDereferenceObject.

Here is a code segment from the w2k ddk floppy.c that illustrates this
technique:

status = PsCreateSystemThread(&threadHandle,
(ACCESS_MASK) 0L,
&ObjAttributes,
(HANDLE) 0L,
NULL,
FloppyThread,
DisketteExtension);

if (!NT_SUCCESS(status)) {
// deleted stuff here for error handling.
return status;
}

status = ObReferenceObjectByHandle( threadHandle,
SYNCHRONIZE,
NULL,
KernelMode,

&DisketteExtension->FloppyThread,
NULL );

ASSERT(NT_SUCCESS(status));

ZwClose(threadHandle);

And later on in the same file, the termination handler:

ASSERT(disketteExtension->FloppyThread != NULL);

KeWaitForSingleObject( disketteExtension->FloppyThread,
Executive,
KernelMode,
FALSE,
NULL );

if (disketteExtension->FloppyThread != NULL) {
ObDereferenceObject( disketteExtension->FloppyThread );
}

disketteExtension->FloppyThread = NULL;

=====================
Mark Roddy
Windows XP/2000/NT Consulting
Hollis Technology Solutions 603-321-1032
www.hollistech.com
xxxxx@hollistech.com
For Windows Device Driver Training: see www.azius.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David J. Craig
Sent: Monday, March 25, 2002 6:09 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Thread termination in a Win2K driver

I would change the wait logic to wait for the thread to
terminate. One trick for drivers is to have the thread that
is waiting to drop its priority to the lowest and increase
the other thread(s) to the highest. You can do a
KeWaitForSingleObject on a thread also and that is the
simplest solution.

----- Original Message -----
From: “Roy M. Silvernail”
> To: “NT Developers Interest List”
> Sent: Monday, March 25, 2002 5:21 PM
> Subject: [ntdev] Thread termination in a Win2K driver
>
>
> > I have an inherited project that’s giving me fits. It’s a driver
> > originally developed on Win98, which I’m moving to Win2K.
> The problem
> > is that the driver creates a worker thread (using
> > PsCreateSystemThread()). When the driver exits (it’s an IEEE1394
> > driver, so the plug might be pulled while the computer is up), I get
> >
> > *** Fatal System Error: 0x000000ce
> >
> > and ‘!analyze -v’ tells me the driver “unloaded without cancelling
> > timers, DPCs, worker threads, etc.”
> >
> > I’ve narrowed it down to this worker thread. If I never start the
> > thread, I can exit without incident.
> >
> > The thread does call PsTerminateSystemThread() as it exits, and
> > KdPrint statements seem to show that the cleanup routines are being
> > called and completing. The main driver thread waits on an event
> > (which the worker thread signals as it exits) before completing the
> > StopDevice code, so everything looks right. Yet the
> machine still
> > crashes, whether in response to a surprise removal or from a stop
> > command from the ‘Unplug or Eject Hardware’ dialog.
> >
> > Any clues you guys have to offer will be gratefully accepted.
> > –
> > Roy M. Silvernail
> > Software Engineer
> > Acroloop Motion Control Systems, Inc.
> > xxxxx@acroloop.com
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@yoshimuni.com To
> > unsubscribe send a blank email to %%email.unsub%%
>
>
>
> —
> You are currently subscribed to ntdev as:
> xxxxx@hollistech.com To unsubscribe send a blank email to
> %%email.unsub%%
>
>
>

> The thread does call PsTerminateSystemThread() as it exits, and KdPrint

statements seem to show that the cleanup routines are being called and
completing. The main driver thread waits on an event (which the worker
thread signals as it exits)

Wait on KTHREAD itself. This is the only 100% reliable way of doing this.
Signaling an event opens a window between event signaling and real thread termination.

Note that waiting on KTHREAD is not supported in Win9x - so, no reliable ways of doing this in Win9x OSes.

Max