A physical memory allocator can run at the lowest level and not have to know
about paging issues. So, we can write a slab allocator, for example, Linux
did it. In this case, however, it’s probably not even needed because the
stack doesn’t need to be physically contiguous anyway. If resources aren’t
available, resources aren’t available, and we do whatever we normally do
when resources aren’t available. Modifying page tables may require locking,
yes, but that’s part of life: shared resources must be protected, and
synchronization constructs, at least in theory, should belong to the
resources being synchronized and not to anything else. Moving the contents
of the stack can be done inside the kernel - we’re assuming that running out
of stack is not that frequent an event, so, we may be justified in spending
some resources in order to get it right, even if I stop everything on its
tracks and do a hard systemwide cli before I move the data, I’ll probably
still be justified even in performance terms.
The assumption under Linux reentrancy is basically the same in any OS: code
that must run atomically will. The only difference is the means utilized to
achieve the ends. But I’m not too sure what you mean by “recursive stack
utilization”, can you expand ?
And I don’t see why stack swapping done at the right level will break
anything. As a matter of fact, the Pentium architecture does it when calling
between privilege levels, if nothing else I can see myself trying to fool
the architecture to swap stacks for me.
Alberto.
-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 22, 2002 2:27 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
It is a question of OS philosophy. Unfortunately, your analysis is a “hand
wave” over the low-level details. (1) Allocate a “slab” of physical memory.
This is going to require some form of serialization. It also assumes a
“slab” is available. What do you do when one is not available? Where does
it come from? Do you start page reclamation as part of this? (2) Modifying
the page tables again would require some form of serialization. (3) Moving
the contents of stack. That is thread local, so the only serialization
would be something like a raise to APC_LEVEL.
And how do you decide to deal with recursive stack utilization. Linux
doesn’t have the same problem because, being UNIX based, there are
assumptions about OS reentrancy that differ.
It is also difficult to do spot-comparisons such as this, as tempting as it
is to do so. What are the knock-on effects of changing this behavior? I
know (for example) that there were changes in the way stack is managed in
Windows XP and it broke some third party products that played games (like
swapping the stack). Of course, Linux has a far smaller application base
and thus we hear of fewer incompatibility issues - both because there are
fewer, and because the volume of the complaints is smaller.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Hope to see you at the next OSR file systems class October 7, 2002!
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 2:02 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
It’s a question of OS philosophy, no ? Expanding the stack requires (1)
allocating a slab of physical memory, (2) mapping it to a page table
somewhere, and (3) setting up a stack segment with that slab. This ends the
memory management issues, and all three functions are pretty low level that
can be easily handled without any additional locking complication. Not a big
deal, eh ? Definitely not rocket science, at least not if one includes the
functionality within the guts of the system. And for a great way of managing
memory, look at the Linux slab allocator.
Alberto.
-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 22, 2002 1:43 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
The kernel stack size is limited because it is using real physical memory
and it isused under operating conditions where it is not possible to extend
it.
Suppose, for example, the MM is writing data back out to the paging file (in
a nested call, no doubt) and that causes a stack overflow error. We jump
back into the memory manager to grow the stack. This assumes a LOT about
the locking model (which is already hideously complicated), the operations
of other things on the system, the amount of available physical memory, etc.
The file systems (for example) have special case code for handling this
issue - they actually post to the stack overflow worker thread.
The choice was intentional - very few things at this level of the OS were
unintentional. There are numerous trade-offs that must be made when
developing an operating system and it shows hubris to sit here and criticize
design decisions made in 1989 and 1990. Sure the world has changed, and the
OS has changed quite a bit as well. But in 1990, could you have predicted
the way the world is today?
Of course, you can always make suggestions to the base team about what
changes you’d like to see them make and why. Doesn’t mean they will make
them - but there are enough people here with hooks into that team that at
least your suggestion has a chance of being heard.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http: http://www.osr.com
Hope to see you at the next OSR file systems class October 7, 2002!
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Wednesday, May 22, 2002 1:04 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
If this was done intentionally, it’s a pretty poor way of achieving improved
safety. Heck, this is kernel dev and not Java beans, how do they know I’m
running on their stack, and not on my own ? Besides, give me a decent-sized
stack, please ! Or at least put the stack in its own segment so that a stack
overflow triggers an exception and the OS can expand the stack on the fly to
fit my need. And how can one make such a runtime decision from a compile
time analysis ? I run things with BoundsChecker, which detects this kind of
issue at runtime, I find it a better idea. Furthermore, if what they want is
to detect a stack overuse condition at compile time, why don’t they just
issue an error message in the compile ? I can think of better ways of
frigging an error than to link one’s code with an unexisting library
function.
Alberto.
-----Original Message-----
From: Graham, Simon [mailto:xxxxx@stratus.com]
Sent: Wednesday, May 22, 2002 11:55 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
More importantly - there is NO implementation of __chkstk for k-mode
drivers; this is why the link fails when you start using huge local
variables. Personally, I think it’s good to build drivers with this turned
on since it lets you know when you start using huge amounts of stack -
always an issue for drivers!
/simgr
-----Original Message-----
From: Arup Banerjee [mailto:xxxxx@quark.co.in]
Sent: Wednesday, May 22, 2002 11:04 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Problem with arrays
Arup, do
> you happen to
> know which library has that entry point ?
>
Address Publics by Value Rva+Base Lib:Object
0001:00003b10__chkstk 00404b10 f LIBC:chkstk.obj
> Alberto.
>
>
> -----Original Message-----
> From: Arup Banerjee [mailto:xxxxx@quark.co.in
mailto:xxxxx]
> Sent: Wednesday, May 22, 2002 9:54 AM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Problem with arrays
>
>
> kernel stack is 12KB if no GUI-related calls, 64KB for a GUI
> thread. By
> default, the compiler inserts a call to the __chkstk() function in the
> prologue code for each function to compares the amount of
> stack space a
> function requires with the amount of stack space available.
>
> In NT atleast this is not explicit but probe is made if
> requested allocation
> is above 4k from the current stack pointer. In fact if u use
> the compiler
> option /Gs with threshold this check is made if local stack allocation
> exceeds the specified threshold…
>
> Ur array declaration is somehow crossing the default
> threshold which is not
> detected when u’ve declared it as a pointer instead . In
> order to supress
> the__chkstk call use an imaginary high threshold value like
> /Gs9999999999999999999. This would supress the compiler from calling
> “__chkstk”.
>
> --Arup
>
>
> > -----Original Message-----
> > From: xxxxx@ieee.org [mailto:xxxxx@ieee.org
mailto:xxxxx]
> > Sent: Wednesday, May 22, 2002 6:39 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] Problem with arrays
> >
> >
> >
> > Guys,
> >
> > In the OS course I just gave, we bumped into a problem with a
> > student’s driver: anytime I used an array of quadwords (to hold 64-
> > bit timestamps) in the driver, the linker would complain that
> > it could
> > not resolve the symbol “__chkstk”. This happens both with the
> > Win2K and the XP DDKs. Change it from an array to a pointer, and
> > the problem disappears. Did any of you ever bump into this kind of
> > problem ?
> >
> >
> > Alberto.
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@quark.co.in
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
> —
> You are currently subscribed to ntdev as:
> xxxxx@compuware.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> The contents of this e-mail are intended for the named
> addressee only. It
> contains information that may be confidential. Unless you are
> the named
> addressee or an authorized designee, you may not copy or use
> it, or disclose
> it to anyone else. If you received it in error please notify
> us immediately
> and then destroy it.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@quark.co.in
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
—
You are currently subscribed to ntdev as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.</mailto:xxxxx></mailto:xxxxx></http:>