RE: OS Design Philosophy

Interestingly, the OS that runs on the X-BOX uses a modified version of the memory-manager than is included in with std XP. IIRC, the system provides easy access to physical memory, and that virtual memory is either not available, or seldom used. Just an interesting aside – maybe someone knows more about this. But anyway, it’s perfect for a system where performance, especially memory bandwidth, is critical and is *unlikely* to be running mission-critical or life-support systems :slight_smile:

Regards,

Paul Bunn, UltraBac Software, 425-644-6000
Microsoft MVP - Windows NT/2000/XP
http://www.ultrabac.com

-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 11:02 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

It’s a question of OS philosophy, no ? Expanding the stack requires (1)
allocating a slab of physical memory, (2) mapping it to a page table
somewhere, and (3) setting up a stack segment with that slab. This ends the
memory management issues, and all three functions are pretty low level that
can be easily handled without any additional locking complication. Not a big
deal, eh ? Definitely not rocket science, at least not if one includes the
functionality within the guts of the system. And for a great way of managing
memory, look at the Linux slab allocator.

Alberto.

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 22, 2002 1:43 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

The kernel stack size is limited because it is using real physical memory
and it isused under operating conditions where it is not possible to extend
it.

Suppose, for example, the MM is writing data back out to the paging file (in
a nested call, no doubt) and that causes a stack overflow error. We jump
back into the memory manager to grow the stack. This assumes a LOT about
the locking model (which is already hideously complicated), the operations
of other things on the system, the amount of available physical memory, etc.

The file systems (for example) have special case code for handling this
issue - they actually post to the stack overflow worker thread.

The choice was intentional - very few things at this level of the OS were
unintentional. There are numerous trade-offs that must be made when
developing an operating system and it shows hubris to sit here and criticize
design decisions made in 1989 and 1990. Sure the world has changed, and the
OS has changed quite a bit as well. But in 1990, could you have predicted
the way the world is today?

Of course, you can always make suggestions to the base team about what
changes you’d like to see them make and why. Doesn’t mean they will make
them - but there are enough people here with hooks into that team that at
least your suggestion has a chance of being heard.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http: http://www.osr.com

Hope to see you at the next OSR file systems class October 7, 2002!

-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 1:04 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

If this was done intentionally, it’s a pretty poor way of achieving improved
safety. Heck, this is kernel dev and not Java beans, how do they know I’m
running on their stack, and not on my own ? Besides, give me a decent-sized
stack, please ! Or at least put the stack in its own segment so that a stack
overflow triggers an exception and the OS can expand the stack on the fly to
fit my need. And how can one make such a runtime decision from a compile
time analysis ? I run things with BoundsChecker, which detects this kind of
issue at runtime, I find it a better idea. Furthermore, if what they want is
to detect a stack overuse condition at compile time, why don’t they just
issue an error message in the compile ? I can think of better ways of
frigging an error than to link one’s code with an unexisting library
function.

Alberto.

-----Original Message-----
From: Graham, Simon [mailto:xxxxx@stratus.com]
Sent: Wednesday, May 22, 2002 11:55 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

More importantly - there is NO implementation of __chkstk for k-mode
drivers; this is why the link fails when you start using huge local
variables. Personally, I think it’s good to build drivers with this turned
on since it lets you know when you start using huge amounts of stack -
always an issue for drivers!

/simgr

-----Original Message-----
From: Arup Banerjee [mailto:xxxxx@quark.co.in]
Sent: Wednesday, May 22, 2002 11:04 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

Arup, do
> you happen to
> know which library has that entry point ?
>

Address Publics by Value Rva+Base Lib:Object
0001:00003b10__chkstk 00404b10 f LIBC:chkstk.obj

> Alberto.
>
>
> -----Original Message-----
> From: Arup Banerjee [mailto:xxxxx@quark.co.in
mailto:xxxxx]
> Sent: Wednesday, May 22, 2002 9:54 AM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Problem with arrays
>
>
> kernel stack is 12KB if no GUI-related calls, 64KB for a GUI
> thread. By
> default, the compiler inserts a call to the __chkstk() function in the
> prologue code for each function to compares the amount of
> stack space a
> function requires with the amount of stack space available.
>
> In NT atleast this is not explicit but probe is made if
> requested allocation
> is above 4k from the current stack pointer. In fact if u use
> the compiler
> option /Gs with threshold this check is made if local stack allocation
> exceeds the specified threshold…
>
> Ur array declaration is somehow crossing the default
> threshold which is not
> detected when u’ve declared it as a pointer instead . In
> order to supress
> the__chkstk call use an imaginary high threshold value like
> /Gs9999999999999999999. This would supress the compiler from calling
> “__chkstk”.
>
> --Arup
>
>
> > -----Original Message-----
> > From: xxxxx@ieee.org [mailto:xxxxx@ieee.org
mailto:xxxxx]
> > Sent: Wednesday, May 22, 2002 6:39 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] Problem with arrays
> >
> >
> >
> > Guys,
> >
> > In the OS course I just gave, we bumped into a problem with a
> > student’s driver: anytime I used an array of quadwords (to hold 64-
> > bit timestamps) in the driver, the linker would complain that
> > it could
> > not resolve the symbol “__chkstk”. This happens both with the
> > Win2K and the XP DDKs. Change it from an array to a pointer, and
> > the problem disappears. Did any of you ever bump into this kind of
> > problem ?
> >
> >
> > Alberto.
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@quark.co.in
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
> —
> You are currently subscribed to ntdev as:
> xxxxx@compuware.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> The contents of this e-mail are intended for the named
> addressee only. It
> contains information that may be confidential. Unless you are
> the named
> addressee or an authorized designee, you may not copy or use
> it, or disclose
> it to anyone else. If you received it in error please notify
> us immediately
> and then destroy it.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@quark.co.in
> To unsubscribe send a blank email to %%email.unsub%%
>


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.


You are currently subscribed to ntdev as: xxxxx@ultrabac.com
To unsubscribe send a blank email to %%email.unsub%%</mailto:xxxxx></mailto:xxxxx></http:>

But graphics is overwhelmingly done inside the graphics chip - you could run
without a Pentium in there, and no XP, and it’d probably be even faster.

Alberto.

-----Original Message-----
From: Paul Bunn [mailto:xxxxx@UltraBac.com]
Sent: Wednesday, May 22, 2002 2:48 PM
To: NT Developers Interest List
Subject: [ntdev] RE: OS Design Philosophy

Interestingly, the OS that runs on the X-BOX uses a modified version of the
memory-manager than is included in with std XP. IIRC, the system provides
easy access to physical memory, and that virtual memory is either not
available, or seldom used. Just an interesting aside – maybe someone knows
more about this. But anyway, it’s perfect for a system where performance,
especially memory bandwidth, is critical and is *unlikely* to be running
mission-critical or life-support systems :slight_smile:

Regards,

Paul Bunn, UltraBac Software, 425-644-6000
Microsoft MVP - Windows NT/2000/XP
http://www.ultrabac.com

-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 11:02 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

It’s a question of OS philosophy, no ? Expanding the stack requires (1)
allocating a slab of physical memory, (2) mapping it to a page table
somewhere, and (3) setting up a stack segment with that slab. This ends the
memory management issues, and all three functions are pretty low level that
can be easily handled without any additional locking complication. Not a big
deal, eh ? Definitely not rocket science, at least not if one includes the
functionality within the guts of the system. And for a great way of managing
memory, look at the Linux slab allocator.

Alberto.

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 22, 2002 1:43 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

The kernel stack size is limited because it is using real physical memory
and it isused under operating conditions where it is not possible to extend
it.

Suppose, for example, the MM is writing data back out to the paging file (in
a nested call, no doubt) and that causes a stack overflow error. We jump
back into the memory manager to grow the stack. This assumes a LOT about
the locking model (which is already hideously complicated), the operations
of other things on the system, the amount of available physical memory, etc.

The file systems (for example) have special case code for handling this
issue - they actually post to the stack overflow worker thread.

The choice was intentional - very few things at this level of the OS were
unintentional. There are numerous trade-offs that must be made when
developing an operating system and it shows hubris to sit here and criticize
design decisions made in 1989 and 1990. Sure the world has changed, and the
OS has changed quite a bit as well. But in 1990, could you have predicted
the way the world is today?

Of course, you can always make suggestions to the base team about what
changes you’d like to see them make and why. Doesn’t mean they will make
them - but there are enough people here with hooks into that team that at
least your suggestion has a chance of being heard.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http: http://www.osr.com

Hope to see you at the next OSR file systems class October 7, 2002!

-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 1:04 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

If this was done intentionally, it’s a pretty poor way of achieving improved
safety. Heck, this is kernel dev and not Java beans, how do they know I’m
running on their stack, and not on my own ? Besides, give me a decent-sized
stack, please ! Or at least put the stack in its own segment so that a stack
overflow triggers an exception and the OS can expand the stack on the fly to
fit my need. And how can one make such a runtime decision from a compile
time analysis ? I run things with BoundsChecker, which detects this kind of
issue at runtime, I find it a better idea. Furthermore, if what they want is
to detect a stack overuse condition at compile time, why don’t they just
issue an error message in the compile ? I can think of better ways of
frigging an error than to link one’s code with an unexisting library
function.

Alberto.

-----Original Message-----
From: Graham, Simon [mailto:xxxxx@stratus.com]
Sent: Wednesday, May 22, 2002 11:55 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

More importantly - there is NO implementation of __chkstk for k-mode
drivers; this is why the link fails when you start using huge local
variables. Personally, I think it’s good to build drivers with this turned
on since it lets you know when you start using huge amounts of stack -
always an issue for drivers!

/simgr

-----Original Message-----
From: Arup Banerjee [mailto:xxxxx@quark.co.in]
Sent: Wednesday, May 22, 2002 11:04 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays

Arup, do
> you happen to
> know which library has that entry point ?
>

Address Publics by Value Rva+Base Lib:Object
0001:00003b10__chkstk 00404b10 f LIBC:chkstk.obj

> Alberto.
>
>
> -----Original Message-----
> From: Arup Banerjee [mailto:xxxxx@quark.co.in
mailto:xxxxx]
> Sent: Wednesday, May 22, 2002 9:54 AM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Problem with arrays
>
>
> kernel stack is 12KB if no GUI-related calls, 64KB for a GUI
> thread. By
> default, the compiler inserts a call to the __chkstk() function in the
> prologue code for each function to compares the amount of
> stack space a
> function requires with the amount of stack space available.
>
> In NT atleast this is not explicit but probe is made if
> requested allocation
> is above 4k from the current stack pointer. In fact if u use
> the compiler
> option /Gs with threshold this check is made if local stack allocation
> exceeds the specified threshold…
>
> Ur array declaration is somehow crossing the default
> threshold which is not
> detected when u’ve declared it as a pointer instead . In
> order to supress
> the__chkstk call use an imaginary high threshold value like
> /Gs9999999999999999999. This would supress the compiler from calling
> “__chkstk”.
>
> --Arup
>
>
> > -----Original Message-----
> > From: xxxxx@ieee.org [mailto:xxxxx@ieee.org
mailto:xxxxx]
> > Sent: Wednesday, May 22, 2002 6:39 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] Problem with arrays
> >
> >
> >
> > Guys,
> >
> > In the OS course I just gave, we bumped into a problem with a
> > student’s driver: anytime I used an array of quadwords (to hold 64-
> > bit timestamps) in the driver, the linker would complain that
> > it could
> > not resolve the symbol “__chkstk”. This happens both with the
> > Win2K and the XP DDKs. Change it from an array to a pointer, and
> > the problem disappears. Did any of you ever bump into this kind of
> > problem ?
> >
> >
> > Alberto.
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@quark.co.in
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
> —
> You are currently subscribed to ntdev as:
> xxxxx@compuware.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> The contents of this e-mail are intended for the named
> addressee only. It
> contains information that may be confidential. Unless you are
> the named
> addressee or an authorized designee, you may not copy or use
> it, or disclose
> it to anyone else. If you received it in error please notify
> us immediately
> and then destroy it.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@quark.co.in
> To unsubscribe send a blank email to %%email.unsub%%
>


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.


You are currently subscribed to ntdev as: xxxxx@ultrabac.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.</mailto:xxxxx></mailto:xxxxx></http:>