WDK/DDK free build and (very) large arrays

NTDEV
1

Hi,

I’ve encountered a code in a driver that for some reason (and let’s put this reason outside of this thread), initialize huge arrays in the code…
It is an inline function resides in header file (Let’s say imp.h) and function is called of course from a C file lets say called call.c, the function looks like:

(imp.h)

__inline void my_strange_func(int a, int b, int c, int d)
{
some code…

static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array

some code…
}

I know it is a strange code but let’s don’t deal with here, the question is the differences between checked/free builds

Now, when I compile this code in checked build environment, it takes me less than few seconds in order to compile & Link the code to a working .sys file!

When I run free environment (WDK 6000/DDK 3790.1830), It takes me a very long time (couple of minutes) to get the following error:
call.obj : error LNK2019: unresolved external symbol __chkstk referenced in function my_strange_func@16

than I replaced __inline with __forceinline

and than I got this error (again, after very long time):

imp.h(92) : error C1002: compiler is out of heap space in pass 2

The long time takes by cl.exe the use ~50% cpu and more than 500,000Kb of memory…

What I see in this meantime in the build window is “Compiling - generating code…”

Now, the questions:

  1. why is the difference between checked and free builds? I would expect them to behave the same in this manner…
    I just add that there are not #ifdef or something around this code…there are also no special definitions for DEBUG build (except the DBG define)

  2. my conclusion is that it is bad idea to put these very large arrays + not realistic idea - am I correct ?

Thanks !

2

Compiler optimizations? I’d hazard a guess that rather than initializing the
arrays at declaration, initilializing them after declaration would reduce
compile time:

int 64KB = 64 * 1024;
static const int array1[64KB];

for (int i=0; i++; i < 64KB)
array1[i] = i;

This is just a guess and a code snippet off of the top of my head, but I’d
bet it would alleviate the problem you’re seeing (as far as compile time). I
could be wrong, but anyway, the next point is more important -

The “chkstk” unresolved external is caused by the compiler checking to see
if you’ve occupied more than (I think 4K on an x86 system) stack space for
local variables - in this case, you certainly are; maybe allocating memory
and then assigning that memory to the array variable would be a better
approach. Kernel mode stack space is a very scarce commodity! In other
words, your assumption that statically defined large arrays (winding up on
the stack rather than using dynamically allocated memory) are a bad idea in
kernel mode is correct.

I think there is a compiler option that enforces stack checking (maybe -Gh
?), and that would explain the difference between the checked and free
builds (as far as the error goes); the different optimization options would
explain your compile times.

Ed Lau

Artera Group, Inc.
Middlebury, CT 06762

----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Wednesday, December 20, 2006 10:05 AM
Subject: [ntdev] WDK/DDK free build and (very) large arrays

> Hi,
>
> I’ve encountered a code in a driver that for some reason (and let’s put
this reason outside of this thread), initialize huge arrays in the code…
> It is an inline function resides in header file (Let’s say imp.h) and
function is called of course from a C file lets say called call.c, the
function looks like:
>
> (imp.h)
>
> __inline void my_strange_func(int a, int b, int c, int d)
> {
> some code…
>
> static const int Array1 = {0,1,…,64KB} ; // total size is 64KB
elements in array
> static const int Array1 = {0,1,…,64KB} ; // total size is 64KB
elements in array
> static const int Array1 = {0,1,…,64KB} ; // total size is 64KB
elements in array
> static const int Array1 = {0,1,…,64KB} ; // total size is 64KB
elements in array
>
> some code…
> }
>
> I know it is a strange code but let’s don’t deal with here, the question
is the differences between checked/free builds
>
> Now, when I compile this code in checked build environment, it takes me
less than few seconds in order to compile & Link the code to a working .sys
file!
>
> When I run free environment (WDK 6000/DDK 3790.1830), It takes me a very
long time (couple of minutes) to get the following error:
> call.obj : error LNK2019: unresolved external symbol__chkstk referenced
in function my_strange_func@16
>
> than I replaced inline with forceinline
>
> and than I got this error (again, after very long time):
>
> imp.h(92) : error C1002: compiler is out of heap space in pass 2
>
> The long time takes by cl.exe the use ~50% cpu and more than 500,000Kb of
memory…
>
> What I see in this meantime in the build window is “Compiling - generating
code…”
>
> Now, the questions:
>
> 1) why is the difference between checked and free builds? I would expect
them to behave the same in this manner…
> I just add that there are not #ifdef or something around this code…there
are also no special definitions for DEBUG build (except the DBG define)
>
> 2) my conclusion is that it is bad idea to put these very large arrays +
not realistic idea - am I correct ?
>
> Thanks !
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>

3

Hi,

Well, I’m very aware to the kernel stack size issue and to the Gs flag, I do know that the chkstk related to this issue, but thses arrays are static const taht shouldn’t be allcoated at stack, but on data segment or something…

4

The difference between checked and free build can be inline functions handling. Try to remove __inline to see if free build changes. It is possible checked build (no optimalization) does it automatically whereas free build tries to make it inline and optimalizator can’t handle it. As another experiment you can inline function by hand; just copy whole block of code to the place where function is called.

Anyway, using such code in the inline function is more than strange :slight_smile:

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

From: xxxxx@lists.osr.com[SMTP:xxxxx@lists.osr.com] on behalf of xxxxx@yahoo.com[SMTP:xxxxx@yahoo.com]
Reply To: Windows System Software Devs Interest List
Sent: Wednesday, December 20, 2006 4:05 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] WDK/DDK free build and (very) large arrays

Hi,

I’ve encountered a code in a driver that for some reason (and let’s put this reason outside of this thread), initialize huge arrays in the code…
It is an inline function resides in header file (Let’s say imp.h) and function is called of course from a C file lets say called call.c, the function looks like:

(imp.h)

__inline void my_strange_func(int a, int b, int c, int d)
{
some code…

static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array

some code…
}

I know it is a strange code but let’s don’t deal with here, the question is the differences between checked/free builds

Now, when I compile this code in checked build environment, it takes me less than few seconds in order to compile & Link the code to a working .sys file!

When I run free environment (WDK 6000/DDK 3790.1830), It takes me a very long time (couple of minutes) to get the following error:
call.obj : error LNK2019: unresolved external symbol __chkstk referenced in function my_strange_func@16

than I replaced __inline with __forceinline

and than I got this error (again, after very long time):

imp.h(92) : error C1002: compiler is out of heap space in pass 2

The long time takes by cl.exe the use ~50% cpu and more than 500,000Kb of memory…

What I see in this meantime in the build window is “Compiling - generating code…”

Now, the questions:

  1. why is the difference between checked and free builds? I would expect them to behave the same in this manner…
    I just add that there are not #ifdef or something around this code…there are also no special definitions for DEBUG build (except the DBG define)

  2. my conclusion is that it is bad idea to put these very large arrays + not realistic idea - am I correct ?

Thanks !

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

I’m not a language expert, but I think in C the “static” storage specifier
only implies that the variable is local to the translation unit (not
accessible from other modules or functions, in this case) - I do not think
that it implies the variable is allocated once at a fixed address (which I
think is the case in C++). The “const” declaration doesn’t have any effect
on the problems that you are seeing, in my opinion: I think that the
const-ness of the variable will only affect your ability to modify the
variable. The fact that the compiler/linker is emitting the “chkstk” error I
think proves my point, unless the compiler is in error (which I doubt). If
anyone can shed more light on my interpretation of the “static” storage
class specifier, I’d love to hear it -

Ed Lau

----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Wednesday, December 20, 2006 11:40 AM
Subject: RE:[ntdev] WDK/DDK free build and (very) large arrays

> Hi,
>
> Well, I’m very aware to the kernel stack size issue and to the Gs flag, I
do know that the chkstk related to this issue, but thses arrays are static
const taht shouldn’t be allcoated at stack, but on data segment or
something…
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>

6

A static variable is a *GLOBAL* which is only visible in the context in which it’s defined.

If defined at the module level it’s only visible to that module. If defined at a function or block level it’s only visible within that function.

However, since this global is also constant, it’s possible the compiler decided that it could reinitialize the value each time on the stack rather than putting it into .CODE or .DATA (I think constants actually end up in .CODE). If the caller can’t change the state there’s no need to persist it globally.

I love compilers.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ed Lau
Sent: Wednesday, December 20, 2006 9:35 AM
To: Windows System Software Devs Interest List
Subject: Re: RE:[ntdev] WDK/DDK free build and (very) large arrays

I’m not a language expert, but I think in C the “static” storage specifier
only implies that the variable is local to the translation unit (not
accessible from other modules or functions, in this case) - I do not think
that it implies the variable is allocated once at a fixed address (which I
think is the case in C++). The “const” declaration doesn’t have any effect
on the problems that you are seeing, in my opinion: I think that the
const-ness of the variable will only affect your ability to modify the
variable. The fact that the compiler/linker is emitting the “chkstk” error I
think proves my point, unless the compiler is in error (which I doubt). If
anyone can shed more light on my interpretation of the “static” storage
class specifier, I’d love to hear it -

Ed Lau

----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Wednesday, December 20, 2006 11:40 AM
Subject: RE:[ntdev] WDK/DDK free build and (very) large arrays

> Hi,
>
> Well, I’m very aware to the kernel stack size issue and to the Gs flag, I
do know that the chkstk related to this issue, but thses arrays are static
const taht shouldn’t be allcoated at stack, but on data segment or
something…
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

xxxxx@yahoo.com wrote:

Hi,

I’ve encountered a code in a driver that for some reason (and let’s put this reason outside of this thread), initialize huge arrays in the code…
It is an inline function resides in header file (Let’s say imp.h) and function is called of course from a C file lets say called call.c, the function looks like:

(imp.h)

__inline void my_strange_func(int a, int b, int c, int d)
{
some code…

static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array
static const int Array1 = {0,1,…,64KB} ; // total size is 64KB elements in array

some code…
}

I’m not sure that the C++ standard defines the meaning of “static const
int” in an inline function. It is entirely possible that you’re getting
another copy of the static data every time the inline is instantiated.

__inline should only be used for trivial functions. It is WAY too easy
to end up with lots and lots of generated code.

I know it is a strange code but let’s don’t deal with here, the question is the differences between checked/free builds

Now, when I compile this code in checked build environment, it takes me less than few seconds in order to compile & Link the code to a working .sys file!

When I run free environment (WDK 6000/DDK 3790.1830), It takes me a very long time (couple of minutes) to get the following error:
call.obj : error LNK2019: unresolved external symbol __chkstk referenced in function my_strange_func@16

than I replaced __inline with __forceinline
and than I got this error (again, after very long time):
imp.h(92) : error C1002: compiler is out of heap space in pass 2
The long time takes by cl.exe the use ~50% cpu and more than 500,000Kb of memory…

What I see in this meantime in the build window is “Compiling - generating code…”

Now, the questions:

  1. why is the difference between checked and free builds? I would expect them to behave the same in this manner…

Quite the contrary! Checked builds turn off optimization. In a checked
build, inline functions will not be expanded inline; you only get one
copy, and the callers all use the normal “call” instruction. In a free
build, the compiler will actually move the code inline, and probably
generate a new copy of the static data each time.

  1. my conclusion is that it is bad idea to put these very large arrays + not realistic idea - am I correct ?

The “bad idea” here is abuse of __inline. There is nothing wrong with
static data per se.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

8

Ed Lau wrote:

I’m not a language expert, but I think in C the “static” storage specifier
only implies that the variable is local to the translation unit (not
accessible from other modules or functions, in this case) - I do not think
that it implies the variable is allocated once at a fixed address (which I
think is the case in C++).

No, static variables and data structures behave the same in C++ and C.
They are initialized once, and the contents survive from call to call.

Note, however that is use of “const” automatically means that he is
using C++.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

9

Well - Tim,

According to what you are saying - the compiler should not fail since it is not on the stack.

Another question (A Potential solution)
I know there is a way on user mode dll’s and module to integrate resources to an executable or a dll, is there any documented way to do it in a .sys file? (beside using resHacker or something)

10

xxxxx@yahoo.com wrote:

Well - Tim,

According to what you are saying - the compiler should not fail since it is not on the stack.

That’s a non sequitur. If you were using up stack space, your PROGRAM
would fail at run-time. The fact that the COMPILER fails is a
completely separate and unrelated issue. If you were calling this
function 500 times, and IF my theory is correct, you can imagine that
the compiler might get a bit boggled trying to create 500 different
instances of your several-hundred-kilobyte static tables.

I’m not excusing it – it is a compiler bug, and trying to find rational
explanations for bugs only leads to madness. I’m just trying to point
out a potential trigger.

Another question (A Potential solution)
I know there is a way on user mode dll’s and module to integrate resources to an executable or a dll, is there any documented way to do it in a .sys file? (beside using resHacker or something)

You can link resources into a .sys file just like any other .dll (I put
a version resource in every driver), but it is a useless exercise –
there are no APIs to fetch them.

There’s nothing wrong with static data. My suggestion is that you
either remove the __inline attribute, or move the static data out of the
__inline function and into file scope.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

11

Well, some more insights:

When I tried to move these static const arrays outside of the function scope to the global scope, it still didn’t help me (only on free build)
Than, I tried to see what make the difference(checked/free)
so I explored the WinDDK\6000\bin\setenv.bat file:
So it’s the MSC_OPTIMIZATION (not surprising)
checked build: MSC_OPTIMIZATION=/Od /Oi
free buikd: MSC_OPTIMIZATION=

So just for testing, the answer is to add the “/Oi” in free and than it acts like checked build
vise versa - if I remove the “/Oi” on checked build - than again the linker is getting lost…

From documentation:
/Oi (Generate Intrinsic Functions):
Replaces some function calls with intrinsic or otherwise special forms of the function that help your application run faster.
/Oi is only a request to the compiler to replace some function calls with intrinsics; the compiler may call the function (and not replace the function call with an intrinsic) if it will result in better performance.

I tried to use some options from documentation like:

Add before my function:
#pragma intrinsic(my_strange_func)
when I did so I got:
warning C4163: ‘my_strange_func’ : not available as an intrinsic function

and wrap the function with
#pragma intrinsic(_enable)

#pragma intrinsic(_disable)

But it didn’t help.

So, bottom line, I have a fucntion, that uses global scope const hufe arrays (64Kb), that compiles OK when “/Oi” flag is on and can not compile/link when the “/Oi” flag is off…

Any suggestions?

12

I think the basic answer has already been suggested, but if you’re
interested in the behavior itself, I would get rid of BUILD and go to CL
directly, because they’re are a few other options (-Ob being the one
that comes to mind, although it probably doesn’t affect it as your
function is most unlikely to actually be inlined by the compiler) that
could be affecting it, and, while certainly possible via the log file,
it is generally difficult to say what BUILD is doing behind the scenes.

mm

>> xxxxx@yahoo.com 2006-12-21 10:18 >>>
Well, some more insights:

When I tried to move these static const arrays outside of the function
scope to the global scope, it still didn’t help me (only on free build)
Than, I tried to see what make the difference(checked/free)
so I explored the WinDDK\6000\bin\setenv.bat file:
So it’s the MSC_OPTIMIZATION (not surprising)
checked build: MSC_OPTIMIZATION=/Od /Oi
free buikd: MSC_OPTIMIZATION=

So just for testing, the answer is to add the “/Oi” in free and than it
acts like checked build
vise versa - if I remove the “/Oi” on checked build - than again the
linker is getting lost…

From documentation:
/Oi (Generate Intrinsic Functions):
Replaces some function calls with intrinsic or otherwise special forms
of the function that help your application run faster.
/Oi is only a request to the compiler to replace some function calls
with intrinsics; the compiler may call the function (and not replace the
function call with an intrinsic) if it will result in better
performance.

I tried to use some options from documentation like:

Add before my function:
#pragma intrinsic(my_strange_func)
when I did so I got:
warning C4163: ‘my_strange_func’ : not available as an intrinsic
function

and wrap the function with
#pragma intrinsic(_enable)

#pragma intrinsic(_disable)

But it didn’t help.

So, bottom line, I have a fucntion, that uses global scope const hufe
arrays (64Kb), that compiles OK when “/Oi” flag is on and can not
compile/link when the “/Oi” flag is off…

Any suggestions?

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

13

When you tell me to “go to CL directly” how do you suggest me to do it? look at the log file?
BTW cl.exe says:
/Ob inline expansion (default n=0)
/Oi[-] enable intrinsic functions

Gennerally
Still getting error LNK2019: unresolved external symbol __chkstk referenced in function _my_strange_func@16

Only when /Oi flag is off

14

I mean type whatever it is you wish at the command line, just to ensure
that BUILD is not doing anything behind your back. This probably was
not what you were looking for, retrospectively. Sorry about that.
Certainly, you can get this information just the way you have been, but
it is a lot easier to both experiment and verify directly. I don’t mean
that this will solve your problems. It just sounds like your not
especially familiar with the options, and there are a few options that I
think could affect this, perhaps providing other permutations that might
also produce the behavior, which seems to be what you are after. In
particular, off the top of my head, in addition to -Ob, the group O
options (-O1,-O2,-Os,-Ot,-Ox and -Od) all either set or disable a number
of options at once that might impact this issue, and I personally find
them very hard to keep straight, even though they are documented.
BUILD, by design, insulates you from seeing directly what the options
are; if you want to understand this behavior, which I can not claim to
(personally, I agree with Peter, who knows a great deal more than me on
the subject), but I can certainly understand why you want to, BUILD is
the first thing I would lose, as it just gets in the way in this
particular situation.

I would just try the barest command line that will compile and work
from there. As Peter and a few others who really know their stuff have
weighed in already, I don’t think you’re going to get any specifics that
are better than what you have already determined on your own, because
this looks like it might be degenerate behavior by the compiler, and, as
such, there really is nothing to do but try combinations and rule out
what you can. I would start with the options I mentioned and see what
you get.

This would apply to LINK as well. If you try switching the build of
the CRT (or do so indirectly using -MT/-MTd or the dynamic versions), I
think the error will go away.

mm

>> xxxxx@yahoo.com 2006-12-21 11:56 >>>
When you tell me to “go to CL directly” how do you suggest me to do it?
look at the log file?
BTW cl.exe says:
/Ob inline expansion (default n=0)
/Oi[-] enable intrinsic functions

Gennerally
Still getting error LNK2019: unresolved external symbol __chkstk
referenced in function _my_strange_func@16

Only when /Oi flag is off

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

15

Just write an .RC file and list it among other source files in SOURCES=

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
> Well - Tim,
>
> According to what you are saying - the compiler should not fail since it is
not on the stack.
>
> Another question (A Potential solution)
> I know there is a way on user mode dll’s and module to integrate resources to
an executable or a dll, is there any documented way to do it in a .sys file?
(beside using resHacker or something)
>
>

16

Thank you Maxim,
Any way to fetch it out on runtime?