Windows Dispatcher Threads

> "The Windows 2000 scheduling code is implemented in the kernel. There’s

no single “scheduler” module or routine, however—the code is spread
throughout the kernel in which scheduling-related events occur. The

At least all this code is with KeXxx and KiXxx prefixes and is running under
KiDispatcherLock (the queued spinlock under the new Windows - from XP up I
think).

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> …and who decided which CPU teh scheduler is to run on? Who decided that it

is time to load the scheduler’s thread, and how? Is it an interrupt?

There is not “scheduler’s thread” in Windows, the scheduler is a set of
functions called by any thread or any DPC.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> ************* AT THIS STAGE, where the the return take the CPU to, which

thread? WHo decides IF the thread needs to be scheduled out?

The interrupt/DPC epilog.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Ignore this idea that there’s a “scheduler” as an independent entity.
It doesn’t exist like that in Windows.

In windows there are basically two functions for scheduling in the
dispatcher. The first selects the most appropriate thread to run on a
processor at any given time based on a number of factors such as thread
priority, thread affinity, ideal processor, etc… . The second
initiates a context switch between the current thread and the new one.

The Ke subsystem implements these functions and uses them to schedule
threads in a distributed manner. Basically any time a Ke function is
called which changes the state of a thread (to block it, to wake it up,
to change its priority, to mark it as having exceeded a quantum) that
function will invoke these two other functions to see if a different
thread should be running and, if so, to swap over to it.

So you’ve got some user-mode code on a processor. That code will
continue to run until it:

* Calls WaitForSingleObject, which calls the
NtWaitForSingleObject system call which transitions to kernel mode and
(eventually) calls KeWaitForSingleObject. KeWaitForSingleObject links
the current thread onto the wait list for the object (say an event) then
selects another thread which could be run and does a context switch

* Gets a page fault. This triggers the kernel trap handler,
which calls the memory manager which will initiate a page-in then calls
KeWaitForSingleObject to block the thread until the I/O completes.
KeWaitForSingleObject will select a new thread to run and switch to it.

* Is interrupted by a timer interrupt. The kernel’s timer
interrupt handler will see if the current thread has run longer than its
quota and, if so, will attempt to select a new thread to run. If the
current thread is the highest priority thread in the system then it will
be left on the processor. Otherwise the kernel code will get select a
new one and switch to it.

* Is interrupted by a device interrupt which completes an I/O
request initiated by another process. Often this involves boosting the
priority of the other process by calling a Ke function. This function
sets the new priority, then calls the selection function to see what
process should be running on the processor. If there’s a better
candidate after the priority boost then Ke will swap to the new thread

There are a number of other conditions which can cause a context switch,
and I’ve dramatically oversimplified some of them, but hopefully this
will help you get the idea. The responsibility for scheduling in NT is
distributed across all threads in the system, which eventually end up in
a Ke routine.

-p

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of A P
Sent: Wednesday, September 06, 2006 5:34 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Windows Dispatcher Threads

“The kernel runs as needed on a many processors as needed. The
scheduler
deals with what threads are available to run”

…and who decided which CPU teh scheduler is to run on? Who decided
that it is time to load the scheduler’s thread, and how? Is it an
interrupt?

— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

> all this is fine, but to adjust these global variables, some code needs to

run!!! if at that time another thread is running, and this update operation
needs to take place how is it done? is that through timer inperrupt and it’s
ISR???

This occurs in:

  • KeWaitForSingleObject
  • KeSetEvent
  • timer interrupt epilog
  • and so on

Lots of places.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

when the machine boots up, just after the boot loader finishes, is it still
an SMP there, of a master slave?

if there are multiple processors, after POST, which processor gets control?
How is that decided?

Well, it is “MP” but one of the processors (the “boot processor”) is
running and the others are in a halted condition waiting for their
startup information. The details are machine specific (of course) but a
good basic guide to how it is often done in smaller MP boxes (2-4
processors) is the MPS 1.4 specification guide. In NUMA architectures
(where you have “nodes” of processors) the simple trick I’ve seen is
that one of the nodes is the “boot node”.

No doubt there are many other specialized solutions, but during booting
the usual rule is to keep it simple.

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com http:</http:>


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of A P
Sent: Friday, September 08, 2006 8:34 AM
To: ntdev redirect
Subject: Re: [ntdev] Windows Dispatcher Threads

when the machine boots up, just after the boot loader finishes, is it
still an SMP there, of a master slave?

if there are multiple processors, after POST, which processor gets
control? How is that decided?

— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

A P wrote:

when the machine boots up, just after the boot loader finishes, is it
still an SMP there, of a master slave?

if there are multiple processors, after POST, which processor gets
control? How is that decided?

When the machine boots up, only one processor is taken out of reset.
The BIOS, the POST, and the early boot loader run with this one
processor. At some point after the processor tables have been
configured and switched to protected mode, the kernel will use BIOS- or
chipset-specific functions to take the other processors out of reset,
one at a time.

There is never a “master/slave” relationship. SMP refers to the
processor capabilities and access to resources, not to the way
processors are used. All processors in an SMP system have equal access
to resources, like memory and I/O bus.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

A P wrote:

I have a basic question:

On a multiprocessor machine, how many dispatcher threads does Windows
have? Is it one, or = the # of processors?

AP

All the excellent replies and question in this thread can be answered
completely by two sources:

Best regards,
Alex Ionescu

> when the machine boots up, just after the boot loader finishes, is it still

an SMP there, of a master slave?

SMP, but only 1 processor is running, others are stalled.

if there are multiple processors, after POST, which processor gets control?

At hardware reset, in some configurations, CPUs automatically elect one of them
to be “main”. In other configurations, this election is hard-coded.

Non-main CPUs are immediately stalled and only will be awakened by IPI sent
from HalStartNextProcessor

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com