System Threads and IRQ Levels

As I understand the concept of micro-kernel, it’s the innermost layer of an
OS, the only layer that truly talks to hardware and can directly affect
hardware. The idea (actually, an academic idea; I believe it was the
Carnegie-Mellon Mach kernel that was the real example of micro-kernel) is
that one or more OSes could be built on top of this layer and express
different behavior.

Well and good. In practice, micro-kernels have a poor reputation (so I
believe), because performance is rotten. That’s due to the transition from
the OS layer to the “real,” micro-kernel layer and back. I knew of a project
when I was at IBM that foundered because of that sort of performance
difficulty.

That’s why I don’t think the Windows OS follows a micro-kernel model: There
are layers, but they’re closely connected. I would go so far as to say they
penetrate (are aware of) each other. That penetration definitely is not in
the micro-kernel way of doing things.

I’m not the only person who thinks this way. See, for example, Russinovich
and Solomon, “Microsoft Windows Internals,” fourth edition, page 36.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no rights.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Head on over to building 26 and tell that to Cutler… :wink:

While Microsoft doesn’t distribute a separate binary image that is
called the “micro-kernel” the structure of the OS was geared towards
that model, and even today the kernel components are structurally
separable - providing services to the OS executive through a small set
of core services. The only piece with which this breaks down is in
areas that are highly machine specific (read: virtual memory).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 2:01 PM
To: ntfsd redirect
Subject: Re:[ntfsd] System Threads and IRQ Levels

The Windows OS doesn’t have a micro-kernel.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Ken,

Basically, yes. Thread scheduling is done entirely inside the micro
kernel and as far as it is concerned, the two different types of threads
are indistinguishable (they both have KTHREAD structures, which is all
it cares about…)

Of course I said basically because on an MP (or SMT/Hyperthread or
dual-core) system those threads could both be running, albeit on
different processors. Further, you will occasionally see situations
where a higher priority thread is in standby (ready to run) but a lower
priority thread is running on the processor. Presumably, this is
because in general it is more efficient to let the lower priority thread
keep running versus sending an IPI to the other processor telling it to
switch to the higher priority thread.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Friday, August 19, 2005 12:53 PM
To: ntfsd redirect
Subject: RE: [ntfsd] System Threads and IRQ Levels

Tony:

So, assuming all other things are equal (i.e., running at
PASSIVE_LEVEL), is
it correct to assume that a user-mode program running at priority 4 will
get
scheduled before a system thread running at priority 3?

This is the crux of my original (related) question.

Ken

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, August 19, 2005 12:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] System Threads and IRQ Levels

No, system threads are created by calling PsCreateSystemThread. I
believe you are confusing a “system thread” with a “work queue thread”.
The Executive creates a pool of system threads for servicing work items
that are inserted into the work queues; the ExQueueWorkItem function is
traditionally used, although the current preference is that you use
IoQueueWorkItem. But when discussing the executive worker threads, work
items should leave the thread state comparable to what it was on entry.

With that said, in my experience, it is not safe for file system filter
drivers to use system work queues. I generally build my own even in a
file system because I find the high priority assigned to the executive
work queues does not provide me with the background style processing I
am seeking. In a filter driver, I’ve had far too many deadlock issues
between my filter and the underlying file system using the same work
queue - best to avoid them entirely by using a different work queue.

Notice that filter manager has its own work queues as well.

So perhaps the OP was asking “at what IRQL level will work items that I
post to system work queues be called”, to which the answer should always
be: “IRQL PASSIVE_LEVEL”.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@Home
Sent: Friday, August 19, 2005 12:05 PM
To: ntfsd redirect
Subject: Re: [ntfsd] System Threads and IRQ Levels

> For example, they are not prohibited from acquiring
> spin locks (and raising IRQL to DISPATCH_LEVEL)
Correct me if I’m wrong, but I thought that system threads
are in a pool. Think about a horse stable where you
can rent a horse for some time and then return it so that
someone else has the next ride.

Now, what if I LEAVE the horse on an elevated level
after I’m done?

The next rider might not expect that - work items are
supposed to run at PASSIVE, and they are run by
threads from a system pool.

Can it be said then that yes, your code that runs in a
system pool is allowed to elevate the IRQL, but only
temporarily?

Meaning “clean your horse before returning it back to
the stable”.

Is this a correct picture?

[Alternatively, someone does it before allowing the next
rider in (which I think does not happen).]

Regards,
Alex Shvedov

----- Original Message -----
From: “Tony Mason”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 11:16 AM
Subject: RE: [ntfsd] System Threads and IRQ Levels

System threads run at any IRQL. For example, they are not prohibited
from acquiring spin locks (and raising IRQL to DISPATCH_LEVEL) or
acquiring fast mutexes (and raising IRQL to APC_LEVEL). They can even
call ISR synchronize routines (and raise to DIRQL).

The implicit question (and seemingly the answers) seems to be “at what
IRQL do system threads START executing?” The answer to THAT is:
PASSIVE_LEVEL. From a driver perspective, code executing in a user
thread versus one executing in a system thread is generally identical.

Scheduling has to do with the management of thread priorities as they
use the processor; the scheduler runs at IRQL DISPATCH_LEVEL and thus
any driver code (regardless of thread context) that raises IRQL to
DISPATCH_LEVEL is non-preemptable (but, oddly enough, not quite
non-dispatchable - check out KeDelayExecutionThread which does work at
DISPATCH_LEVEL). Scheduling, while conceptually simple is fiendishly
complicated in application because it is (from a computational
complexity perspective) a “hard problem” (well, perfect scheduling is a
“hard problem”). Thus scheduling priority only comes into play when a
thread is running at IRQL < DISPATCH_LEVEL (e.g., PASSIVE_LEVEL or
APC_LEVEL).

And, just as a side-note, the IDLE thread is not scheduled it is
“chosen” because there are no schedulable threads to run. It actually
executes at DISPATCH_LEVEL.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Satya Das
Sent: Friday, August 19, 2005 10:58 AM
To: ntfsd redirect
Subject: RE: [ntfsd] System Threads and IRQ Levels

PASSIVE_LEVEL.

No it does not.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neil Weicher
Sent: Thursday, August 18, 2005 2:03 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] System Threads and IRQ Levels

This may have been answered before, but what IRQL do System Threads run
at?
Does raising the priority affect that? Thanks.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@appstream.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
Really ?? You are taking what Tony is saying too literally. From MSDN
“The Windows NT operating system is based on a design principle often
referred to as a microkernel architecture”

This thread is getting kinda long. Can you say which “this statement”
you’re referring to?

Ken

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 5:43 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] System Threads and IRQ Levels

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
Really ?? You are taking what Tony is saying too literally. From MSDN
“The Windows NT operating system is based on a design principle often
referred to as a microkernel architecture”


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@comcast.net
To unsubscribe send a blank email to xxxxx@lists.osr.com

No it is not if we follow the definition ( as well as the intention )
correctly.

If you also read the os/2 inside from datel ( i think ), you will see
similar comment(s). Started out with microkernel idea, probably ahead of
their time, dropped out the pure idea.

None of them are pure microkernel based .

-pro

This thread is getting kinda long. Can you say which “this statement”
you’re referring to?

Ken

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 5:43 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] System Threads and IRQ Levels

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
> Really ?? You are taking what Tony is saying too literally. From MSDN
> “The Windows NT operating system is based on a design principle often
> referred to as a microkernel architecture”
>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@comcast.net
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@garlic.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

5 seconds with Google…

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpr
o/html/msdn_realtime.asp

Scroll down the page, you’ll see a big color diagram of the OS. It is
in the paragraph just above.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 5:43 PM
To: ntfsd redirect
Subject: Re:[ntfsd] System Threads and IRQ Levels

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
Really ?? You are taking what Tony is saying too literally. From MSDN
“The Windows NT operating system is based on a design principle often
referred to as a microkernel architecture”


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

And I quote from said article: “To ensure higher levels of performance, the
design time for Windows NT modified the microkernel design by eliminating
the clearly defined break between the kernel code and other system
components.”

“Eliminating the clearly defined break,” most especially for performance
purposes, means that Windows is not micro-kernel. The break was the whole
point of the micro-kernel concept.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no rights.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
5 seconds with Google…

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpr
o/html/msdn_realtime.asp

Scroll down the page, you’ll see a big color diagram of the OS. It is
in the paragraph just above.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 5:43 PM
To: ntfsd redirect
Subject: Re:[ntfsd] System Threads and IRQ Levels

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
Really ?? You are taking what Tony is saying too literally. From MSDN
“The Windows NT operating system is based on a design principle often
referred to as a microkernel architecture”


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Here is another good one:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndevice/html/IRQL_Sched.asp

Here they recommend using Work Items instead of System Threads for
performance purposes. Has anyone seen or done any benchmarks for Work Items
vs. System Threads?

As I mentioned before, one reason I have come across for using System
Threads vs. Work Items is that Work Items queued from dispatch routines
APPEAR to be limited to the CPU affinity of the calling process. Has anyone
else seen this?

This is probably unimportant for most developers because most processes have
affinity masks of all 1s. However I came across this little detail when I
discovered that even though I was queing multiple Work Items, they were all
sharing a single CPU.

  • Neil

----- Original Message -----
From: “Tony Mason”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 6:00 PM
Subject: RE: [ntfsd] System Threads and IRQ Levels

5 seconds with Google…

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpr
o/html/msdn_realtime.asp

Scroll down the page, you’ll see a big color diagram of the OS. It is
in the paragraph just above.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 5:43 PM
To: ntfsd redirect
Subject: Re:[ntfsd] System Threads and IRQ Levels

As I just said, I disagree. But where exactly is this statement made? I
couldn’t find it.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Satya Das” wrote in message news:xxxxx@ntfsd…
Really ?? You are taking what Tony is saying too literally. From MSDN
“The Windows NT operating system is based on a design principle often
referred to as a microkernel architecture”


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

It seems to me that the overhead is in repeatedly
creating the threads. Do what Tony suggested: create
your own pool of “worker” threads.

— Neil Weicher wrote:

> Here is another good one:
>
>
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndevice/html/IRQL_Sched.asp
>
> Here they recommend using Work Items instead of
> System Threads for
> performance purposes. Has anyone seen or done any
> benchmarks for Work Items
> vs. System Threads?
>
> As I mentioned before, one reason I have come across
> for using System
> Threads vs. Work Items is that Work Items queued
> from dispatch routines
> APPEAR to be limited to the CPU affinity of the
> calling process. Has anyone
> else seen this?
>
> This is probably unimportant for most developers
> because most processes have
> affinity masks of all 1s. However I came across this
> little detail when I
> discovered that even though I was queing multiple
> Work Items, they were all
> sharing a single CPU.
>
> - Neil
>
> ----- Original Message -----
> From: “Tony Mason”
> To: “Windows File Systems Devs Interest List”
>
> Sent: Friday, August 19, 2005 6:00 PM
> Subject: RE: [ntfsd] System Threads and IRQ Levels
>
>
> 5 seconds with Google…
>
>
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpr
> o/html/msdn_realtime.asp
>
> Scroll down the page, you’ll see a big color diagram
> of the OS. It is
> in the paragraph just above.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> James Antognini
> [MSFT]
> Sent: Friday, August 19, 2005 5:43 PM
> To: ntfsd redirect
> Subject: Re:[ntfsd] System Threads and IRQ Levels
>
> As I just said, I disagree. But where exactly is
> this statement made? I
> couldn’t find it.
>
> –
> James Antognini
> Windows DDK and WDK Support
>
>
> This posting is provided “AS IS” with no warranties,
> and confers no
> rights.
>
>
>
> “Satya Das” wrote in message
> news:xxxxx@ntfsd…
> Really ?? You are taking what Tony is saying too
> literally. From MSDN
> “The Windows NT operating system is based on a
> design principle often
> referred to as a microkernel architecture”
>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as:
> xxxxx@osr.com
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: unknown
> lmsubst tag argument: ‘’
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as:
> xxxxx@yahoo.com
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>

By worker threads, I assume you mean via ExInitializeWorkItem? That is the
preferrable method, but like I said, it seems to hit up against the CPU
affinity issue. Granted it will not affect most people.

Thanks for the reply.

----- Original Message -----
From: “Randy Cook”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 8:15 PM
Subject: Re: [ntfsd] System Threads and IRQ Levels

> It seems to me that the overhead is in repeatedly
> creating the threads. Do what Tony suggested: create
> your own pool of “worker” threads.
>
> — Neil Weicher wrote:
>
>> Here is another good one:
>>
>>
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndevice/html/IRQL_Sched.asp
>>
>> Here they recommend using Work Items instead of
>> System Threads for
>> performance purposes. Has anyone seen or done any
>> benchmarks for Work Items
>> vs. System Threads?
>>
>> As I mentioned before, one reason I have come across
>> for using System
>> Threads vs. Work Items is that Work Items queued
>> from dispatch routines
>> APPEAR to be limited to the CPU affinity of the
>> calling process. Has anyone
>> else seen this?
>>
>> This is probably unimportant for most developers
>> because most processes have
>> affinity masks of all 1s. However I came across this
>> little detail when I
>> discovered that even though I was queing multiple
>> Work Items, they were all
>> sharing a single CPU.
>>
>> - Neil
>>

Neil,

Maybe I misunderstood Tony’s suggestion, but I thought
it was that you could start a few system threads, keep
them around, and hand off stuff to them when you had
work to do. Don’t tear them down, just put them in a
queue and reuse them.

Randy

— Neil Weicher wrote:

> By worker threads, I assume you mean via
> ExInitializeWorkItem? That is the
> preferrable method, but like I said, it seems to hit
> up against the CPU
> affinity issue. Granted it will not affect most
> people.
>
> Thanks for the reply.
>
> ----- Original Message -----
> From: “Randy Cook”
> To: “Windows File Systems Devs Interest List”
>
> Sent: Friday, August 19, 2005 8:15 PM
> Subject: Re: [ntfsd] System Threads and IRQ Levels
>
>
> > It seems to me that the overhead is in repeatedly
> > creating the threads. Do what Tony suggested:
> create
> > your own pool of “worker” threads.
> >
> > — Neil Weicher wrote:
> >
> >> Here is another good one:
> >>
> >>
> >
>
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndevice/html/IRQL_Sched.asp
> >>
> >> Here they recommend using Work Items instead of
> >> System Threads for
> >> performance purposes. Has anyone seen or done
> any
> >> benchmarks for Work Items
> >> vs. System Threads?
> >>
> >> As I mentioned before, one reason I have come
> across
> >> for using System
> >> Threads vs. Work Items is that Work Items queued
> >> from dispatch routines
> >> APPEAR to be limited to the CPU affinity of the
> >> calling process. Has anyone
> >> else seen this?
> >>
> >> This is probably unimportant for most developers
> >> because most processes have
> >> affinity masks of all 1s. However I came across
> this
> >> little detail when I
> >> discovered that even though I was queing
> multiple
> >> Work Items, they were all
> >> sharing a single CPU.
> >>
> >> - Neil
> >>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as:
> xxxxx@yahoo.com
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>

> Here they recommend using Work Items instead of System Threads for

performance purposes.

This is probably that the Work items has limited number of threads
for the whole system (for all clients of work items in the systems).
Af far as I can rememner, the number of threads was 2-3-5
for small-medium-large system. If every component in the system will
create big (or at least some) amount of additional workers,
it will degrade performance. The question is - how much.

Has anyone seen or done any benchmarks for Work Items

:-). I quess that if you do any benchmarks, the system threads will win,
because will be ready sooner than System threads (they must not
divide their work between your driver and the rest of the system.

L.

IRQL are another concept which is more fundamental and low-level then the
dispatcher priority. For instance, the priority system is defunct if you’re on
a high IRQL.

System threads run on PASSIVE_LEVEL.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Neil Weicher”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 1:03 AM
Subject: [ntfsd] System Threads and IRQ Levels

> This may have been answered before, but what IRQL do System Threads run at?
> Does raising the priority affect that? Thanks.
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

>non-dispatchable - check out KeDelayExecutionThread which does work at

DISPATCH_LEVEL).

Hmm… why does the documentation say KeDelayExecutionThread is callable
at <= APC_LEVEL ?

APC_LEVEL < DISPATCH_LEVEL

:slight_smile:

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> Thanks for the lowdown. So you are saying that it is better to use a System

Thread (PsCreateSystemThread) rather than a Work Item (ExQueueWorkItem)
in a
filter driver? I use both at different times but have not run into any
problems with Work Items. Maybe I have been lucky so far?

No. Usually, PsCreateSystemThread gives you the same functionality as work
item, but with more coding efforts :).

I would use PsCreateSystemThread only for some very non-trivial tasks, usually
something related to a security context in which the work item must run.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Usually “micro-kernel” means - the OS design where the drivers and
subsystems like TCP/IP and FSDs are implemented as special kinds of processes
and not as kernel modules, and LPCs are heavily uses to request IO from them.

In this aspect, Windows is NOT micro-kernel, not more then Linux and other
traditional UNIXen. Even RSX-11 and VMS were more microkernel then Windows.

True microkernel OSes are Windows CE, QNX and some others.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Tony Mason”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 10:36 PM
Subject: RE: [ntfsd] System Threads and IRQ Levels

Head on over to building 26 and tell that to Cutler… :wink:

While Microsoft doesn’t distribute a separate binary image that is
called the “micro-kernel” the structure of the OS was geared towards
that model, and even today the kernel components are structurally
separable - providing services to the OS executive through a small set
of core services. The only piece with which this breaks down is in
areas that are highly machine specific (read: virtual memory).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Friday, August 19, 2005 2:01 PM
To: ntfsd redirect
Subject: Re:[ntfsd] System Threads and IRQ Levels

The Windows OS doesn’t have a micro-kernel.


James Antognini
Windows DDK and WDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Ken,

Basically, yes. Thread scheduling is done entirely inside the micro
kernel and as far as it is concerned, the two different types of threads
are indistinguishable (they both have KTHREAD structures, which is all
it cares about…)

Of course I said basically because on an MP (or SMT/Hyperthread or
dual-core) system those threads could both be running, albeit on
different processors. Further, you will occasionally see situations
where a higher priority thread is in standby (ready to run) but a lower
priority thread is running on the processor. Presumably, this is
because in general it is more efficient to let the lower priority thread
keep running versus sending an IPI to the other processor telling it to
switch to the higher priority thread.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Friday, August 19, 2005 12:53 PM
To: ntfsd redirect
Subject: RE: [ntfsd] System Threads and IRQ Levels

Tony:

So, assuming all other things are equal (i.e., running at
PASSIVE_LEVEL), is
it correct to assume that a user-mode program running at priority 4 will
get
scheduled before a system thread running at priority 3?

This is the crux of my original (related) question.

Ken

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, August 19, 2005 12:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] System Threads and IRQ Levels

No, system threads are created by calling PsCreateSystemThread. I
believe you are confusing a “system thread” with a “work queue thread”.
The Executive creates a pool of system threads for servicing work items
that are inserted into the work queues; the ExQueueWorkItem function is
traditionally used, although the current preference is that you use
IoQueueWorkItem. But when discussing the executive worker threads, work
items should leave the thread state comparable to what it was on entry.

With that said, in my experience, it is not safe for file system filter
drivers to use system work queues. I generally build my own even in a
file system because I find the high priority assigned to the executive
work queues does not provide me with the background style processing I
am seeking. In a filter driver, I’ve had far too many deadlock issues
between my filter and the underlying file system using the same work
queue - best to avoid them entirely by using a different work queue.

Notice that filter manager has its own work queues as well.

So perhaps the OP was asking “at what IRQL level will work items that I
post to system work queues be called”, to which the answer should always
be: “IRQL PASSIVE_LEVEL”.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@Home
Sent: Friday, August 19, 2005 12:05 PM
To: ntfsd redirect
Subject: Re: [ntfsd] System Threads and IRQ Levels

> For example, they are not prohibited from acquiring
> spin locks (and raising IRQL to DISPATCH_LEVEL)
Correct me if I’m wrong, but I thought that system threads
are in a pool. Think about a horse stable where you
can rent a horse for some time and then return it so that
someone else has the next ride.

Now, what if I LEAVE the horse on an elevated level
after I’m done?

The next rider might not expect that - work items are
supposed to run at PASSIVE, and they are run by
threads from a system pool.

Can it be said then that yes, your code that runs in a
system pool is allowed to elevate the IRQL, but only
temporarily?

Meaning “clean your horse before returning it back to
the stable”.

Is this a correct picture?

[Alternatively, someone does it before allowing the next
rider in (which I think does not happen).]

Regards,
Alex Shvedov

----- Original Message -----
From: “Tony Mason”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 19, 2005 11:16 AM
Subject: RE: [ntfsd] System Threads and IRQ Levels

System threads run at any IRQL. For example, they are not prohibited
from acquiring spin locks (and raising IRQL to DISPATCH_LEVEL) or
acquiring fast mutexes (and raising IRQL to APC_LEVEL). They can even
call ISR synchronize routines (and raise to DIRQL).

The implicit question (and seemingly the answers) seems to be “at what
IRQL do system threads START executing?” The answer to THAT is:
PASSIVE_LEVEL. From a driver perspective, code executing in a user
thread versus one executing in a system thread is generally identical.

Scheduling has to do with the management of thread priorities as they
use the processor; the scheduler runs at IRQL DISPATCH_LEVEL and thus
any driver code (regardless of thread context) that raises IRQL to
DISPATCH_LEVEL is non-preemptable (but, oddly enough, not quite
non-dispatchable - check out KeDelayExecutionThread which does work at
DISPATCH_LEVEL). Scheduling, while conceptually simple is fiendishly
complicated in application because it is (from a computational
complexity perspective) a “hard problem” (well, perfect scheduling is a
“hard problem”). Thus scheduling priority only comes into play when a
thread is running at IRQL < DISPATCH_LEVEL (e.g., PASSIVE_LEVEL or
APC_LEVEL).

And, just as a side-note, the IDLE thread is not scheduled it is
“chosen” because there are no schedulable threads to run. It actually
executes at DISPATCH_LEVEL.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Satya Das
Sent: Friday, August 19, 2005 10:58 AM
To: ntfsd redirect
Subject: RE: [ntfsd] System Threads and IRQ Levels

PASSIVE_LEVEL.

No it does not.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neil Weicher
Sent: Thursday, August 18, 2005 2:03 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] System Threads and IRQ Levels

This may have been answered before, but what IRQL do System Threads run
at?
Does raising the priority affect that? Thanks.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@appstream.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

> hardware. The idea (actually, an academic idea; I believe it was the

Carnegie-Mellon Mach kernel that was the real example of micro-kernel)

Mach contains:

  • interrupt and DPC (dunno the Mach name for them) handling
  • dispatcher and thread object implementation
  • low-level VM - relies on external “pager” object implementation IIRC

The rest - like process object implementation, user security contexts,
filesystems, device drivers and their namespace, TCP/IP - are modules external
to Mach.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com