Funky compiler issue

NTDEV
1

MessageI hear so many people arguing that one should use the DDK compiler to build drivers, well, because, and inevitably the bogeyman follows promptly: do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large volume-rendering graphics engine that implements voxel rendering in hardware - requires a set of relatively involuted data structures that are shared by the user-side library, by the hardware-level diagnostics, and by the device driver. The problem is, the library is and the diags are built with MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is pretty standard, we use the standard makefile and build.exe for the driver and an off-the-shelf .sln and .dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of reasons. As soon as that was done, the diags started to fail. I went down to trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence its offset is 8. In other words, field a is extended with 4 bytes of slop. Now, I could go to either compiler and fiddle with command line options, but hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line options, or at least with the differences clearly and well documented, to build all three pieces of software, I wouldn’t have that problem. While I’m busy protecting myself of a chicken little issue that may never become true, and insist in only using the DDK compiler for driver development, I’m leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making sure that it can correctly build drivers too ? If nothing else, how about a build button that, when clicked, runs build.exe on the solution ? Or better, a “Kernel Driver Project” pick when we choose New/Project ? With a kernel-side safe version of the standard C/C++ runtime library ? Generating kernel-side safe code ? Golly, that would be nice.

Alberto.

2

While I agree with all of the tool feature requests in Alberto’s statement
below I can’t help but throw in 2 cents and say that regardless of the
compiler choices, passing information between two disparate components,
especially betweeen usermode and kernel mode, is always a design problem
requiring a precise and unambiguous ‘in memory representation’ of the
information. A language like C or C++ is a way of expressing the
representation and a compiler is a tool for interpreting the expression. If
you want it to be precise (and you must for IOCTL buffers or on wire
representations) you cannot leave *anything* to chance, whim, or default
settings in the compiler. This is not a class of problem to be fixed by
having one compiler. It is an example of a problem to be fixed by having
one precise representation that is expressed in a way that the compiler has
no wiggle room to misintrepret, regardless of the compiler, revision, or
settings. It may be ugly (due to the language extensions required to
control it) but alignment directives and packing (or dummy fields to
explicitly express the packing and force alignment) would be required to be
precise enough even if only one compiler existed. This becomes even more
critical with 32bit/64bit hybrid environments where alignment and packing
rules may differ between user mode and kernel mode. The statement that the
DDK compiler ‘correctly’ set the offset to 4 for the 64bit field is a matter
of opinion and likely true only in a 32bit kernel mode. In my unqualified
opinion, neither 4 nor 8 is ‘correct’ unless one of them is the one that was
designed for and specified. If the same driver were compiled for AMD64 of
IA64, suddenly the MSVC.NET choice of aligning to 8 bytes would not seem
like such a bad idea. If the design called for not wasting the 4 bytes,
both the AMD64 and IA64 drivers could be coerced into dealing with the
misalligned field (although with a performance penalty). The trade offs in
are not about what the compiler does but what the design calls for. It is
systems level software. Don’t leave it up to the compiler to decide what
that means.

Respectfully,
Dave Cattley
Consulting Engineer
Systems Software Development

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: Thursday, June 09, 2005 11:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to build
drivers, well, because, and inevitably the bogeyman follows promptly: do it
the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large
volume-rendering graphics engine that implements voxel rendering in hardware

  • requires a set of relatively involuted data structures that are shared by
    the user-side library, by the hardware-level diagnostics, and by the device
    driver. The problem is, the library is and the diags are built with
    MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is
    pretty standard, we use the standard makefile and build.exe for the driver
    and an off-the-shelf .sln and .dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of
reasons. As soon as that was done, the diags started to fail. I went down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence
its offset is 8. In other words, field a is extended with 4 bytes of slop.
Now, I could go to either compiler and fiddle with command line options, but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented, to
build all three pieces of software, I wouldn’t have that problem. While I’m
busy protecting myself of a chicken little issue that may never become true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making
sure that it can correctly build drivers too ? If nothing else, how about a
build button that, when clicked, runs build.exe on the solution ? Or better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ? Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

MessageIt’s as simple as using

#pragma pack( push, 4 ) // or 8
struct BLAH { uint32 a; uint64 b; … };
#pragma pack( pop )

–
http://www.cristalink.com

“Alberto Moreira” wrote in message news:xxxxx@ntdev…
I hear so many people arguing that one should use the DDK compiler to build
drivers, well, because, and inevitably the bogeyman follows promptly: do it
the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large
volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that are
shared by the user-side library, by the hardware-level diagnostics, and by
the device driver. The problem is, the library is and the diags are built
with MSVC.NET; the driver uses the DDK compiler. In all three cases, the
build is pretty standard, we use the standard makefile and build.exe for the
driver and an off-the-shelf .sln and .dsp made for the app and for the
diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of
reasons. As soon as that was done, the diags started to fail. I went down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence
its offset is 8. In other words, field a is extended with 4 bytes of slop.
Now, I could go to either compiler and fiddle with command line options, but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented, to
build all three pieces of software, I wouldn’t have that problem. While I’m
busy protecting myself of a chicken little issue that may never become true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making
sure that it can correctly build drivers too ? If nothing else, how about a
build button that, when clicked, runs build.exe on the solution ? Or better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ? Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

4

You can, in cases such as this, use the offsetof macro to cause a compiler
error if the data items don’t align. I also use the sizeof structure test
in an assert to make sure the total size matches what I need. A little
paranoid, but sometimes you just have to do it.

“Alberto Moreira” wrote in message news:xxxxx@ntdev…
MessageI hear so many people arguing that one should use the DDK compiler to
build drivers, well, because, and inevitably the bogeyman follows promptly:
do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large
volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that are
shared by the user-side library, by the hardware-level diagnostics, and by
the device driver. The problem is, the library is and the diags are built
with MSVC.NET; the driver uses the DDK compiler. In all three cases, the
build is pretty standard, we use the standard makefile and build.exe for the
driver and an off-the-shelf .sln and .dsp made for the app and for the
diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of
reasons. As soon as that was done, the diags started to fail. I went down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence
its offset is 8. In other words, field a is extended with 4 bytes of slop.
Now, I could go to either compiler and fiddle with command line options, but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented, to
build all three pieces of software, I wouldn’t have that problem. While I’m
busy protecting myself of a chicken little issue that may never become true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making
sure that it can correctly build drivers too ? If nothing else, how about a
build button that, when clicked, runs build.exe on the solution ? Or better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ? Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

5

Based on alignment rules for 64 bit integers that I know of, the DDK compiled version of the structure is the incorrect layout. A 64 bit integer is to be aligned on an 8 byte boundary. Something else in your headers / includes before the structure definition is changing the layout.

I tested this out. I have just created this structure and inserted it into a working driver and loaded the image via windbg to inspect the symbols.

typedef struct _TEST_OFFSETS {
ULONG a;
ULONGLONG b;
} TEST_OFFSETS;

0:000> dt TEST_OFFSETS
+0x000 a : Uint4B
+0x008 b : Uint8B

What is the typedef for uint64? Perhaps that is the cause of this offset difference.

d

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of David R. Cattley
Sent: Thursday, June 09, 2005 8:38 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Funky compiler issue

While I agree with all of the tool feature requests in Alberto’s statement below I can’t help but throw in 2 cents and say that regardless of the compiler choices, passing information between two disparate components, especially betweeen usermode and kernel mode, is always a? design problem requiring a precise and unambiguous ‘in memory representation’ of the information.? A language like C or C++ is a way of expressing the representation and a compiler is a tool for interpreting the expression.? If you want it to be precise (and you must for IOCTL buffers or on wire representations) you cannot leave *anything* to chance, whim, or default settings in the compiler.? This is not a class of?problem to be fixed by having one compiler.? It is an example of a problem to be fixed by having one precise representation that is expressed in a way that the compiler has no wiggle room to misintrepret, regardless of the compiler, revision, or settings.? It may be ugly (due to the language extensions required to control it) but alignment directives and packing (or dummy fields to explicitly express the packing and force alignment) would be required to be precise enough even if only one compiler existed.? This becomes even more critical with 32bit/64bit hybrid environments where alignment and packing rules may differ between user mode and kernel mode.? The statement that the DDK compiler ‘correctly’ set the offset to 4 for the 64bit field is a matter of opinion and likely true only in a 32bit kernel mode.? In my unqualified opinion, neither 4 nor 8 is ‘correct’ unless one of them is the one that was designed for and specified.? If the same driver were compiled for AMD64 of IA64, suddenly the MSVC.NET choice of aligning to 8 bytes would not seem like such a bad idea.? If the design called for not wasting the 4 bytes, both the AMD64 and IA64 drivers could be coerced into dealing with the misalligned field (although with a performance penalty).? The trade offs in are not about what the compiler does but what the design calls for.? It is systems level software.? Don’t leave it up to the compiler to decide what that means.
?
Respectfully,
Dave Cattley
Consulting Engineer
Systems Software Development

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: Thursday, June 09, 2005 11:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue
I hear so many people arguing that one should use the DDK compiler to build drivers, well, because, and inevitably the bogeyman follows promptly: do it the legit way or it’ll bite you good and hard.
?
Well, I just bumped into this situation, bear with me. My?chip - a large volume-rendering graphics engine that implements voxel rendering in hardware - requires a?set of relatively involuted data structures that are shared by the user-side library, by the hardware-level diagnostics, and by the device driver. The problem is, the library is and the diags are built with MSVC.NET;?the driver uses the DDK compiler. In all three cases, the build is pretty standard,?we use the standard makefile and build.exe for the driver and an off-the-shelf .sln and dsp made for the app and for the diags.
?
Now,? a 32-bit data field was recently upgraded to 64-bit, for a number of reasons. As soon as that was done,?the diags started to fail. I went down to trace the error, and I had a number of structs that look like this:
?
? struct BLAH { uint32 a; uint64 b; … };
?
Guess what ? The offset of field b is correctly generated in the DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence its offset is 8. In other words, field a is extended with 4 bytes of slop. Now, I could go to either compiler and fiddle with command line options, but hey, that’s not the way it should be, is it ?
?
If I could use the same compiler,? with the same default command line options, or at least with the differences clearly and well documented, to build all three pieces of software, I wouldn’t have that problem. While I’m busy protecting myself of a chicken little issue that may never become true, and insist in only using the DDK compiler for driver development, I’m leaving my house open to the sneaky thief to come in and cause me harm.
?
So, Microsoft, one suggestion: next release of MSVC.NET, how about making sure that it can correctly build drivers too ? If nothing else, how about a build button that, when clicked, runs build.exe on the solution ? Or better, a “Kernel Driver Project” pick when?we choose New/Project ? With a kernel-side safe version of the standard C/C++ runtime library ? Generating kernel-side safe code ? Golly, that would be nice.
?
?
Alberto.
?
?

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com —
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

6

How do you do this in a maintainable manner that catches new errors?
Unless you have a hardcoded constant in your ASSERT (ie sizoef(FOO) ==
24), what are you comparing the sizeof the structure against? Also, I
use CASSERTs for these types of checks whenever possible, this way you
can’t even link :wink:

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David J. Craig
Sent: Thursday, June 09, 2005 9:02 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Funky compiler issue

You can, in cases such as this, use the offsetof macro to cause a
compiler
error if the data items don’t align. I also use the sizeof structure
test
in an assert to make sure the total size matches what I need. A little
paranoid, but sometimes you just have to do it.

“Alberto Moreira” wrote in message
news:xxxxx@ntdev…
MessageI hear so many people arguing that one should use the DDK
compiler to
build drivers, well, because, and inevitably the bogeyman follows
promptly:
do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large

volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that
are
shared by the user-side library, by the hardware-level diagnostics, and
by
the device driver. The problem is, the library is and the diags are
built
with MSVC.NET; the driver uses the DDK compiler. In all three cases, the

build is pretty standard, we use the standard makefile and build.exe for
the
driver and an off-the-shelf .sln and .dsp made for the app and for the
diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number
of
reasons. As soon as that was done, the diags started to fail. I went
down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary,
hence
its offset is 8. In other words, field a is extended with 4 bytes of
slop.
Now, I could go to either compiler and fiddle with command line options,
but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented,
to
build all three pieces of software, I wouldn’t have that problem. While
I’m
busy protecting myself of a chicken little issue that may never become
true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about
making
sure that it can correctly build drivers too ? If nothing else, how
about a
build button that, when clicked, runs build.exe on the solution ? Or
better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ?
Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

7

Message Yes, this is exactly why it is a good idea to build all low-level components of the project (if not the whole project) by the DDK.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: Alberto Moreira
To: Windows System Software Devs Interest List
Sent: Friday, June 10, 2005 7:08 AM
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to build drivers, well, because, and inevitably the bogeyman follows promptly: do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large volume-rendering graphics engine that implements voxel rendering in hardware - requires a set of relatively involuted data structures that are shared by the user-side library, by the hardware-level diagnostics, and by the device driver. The problem is, the library is and the diags are built with MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is pretty standard, we use the standard makefile and build.exe for the driver and an off-the-shelf .sln and .dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of reasons. As soon as that was done, the diags started to fail. I went down to trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence its offset is 8. In other words, field a is extended with 4 bytes of slop. Now, I could go to either compiler and fiddle with command line options, but hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line options, or at least with the differences clearly and well documented, to build all three pieces of software, I wouldn’t have that problem. While I’m busy protecting myself of a chicken little issue that may never become true, and insist in only using the DDK compiler for driver development, I’m leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making sure that it can correctly build drivers too ? If nothing else, how about a build button that, when clicked, runs build.exe on the solution ? Or better, a “Kernel Driver Project” pick when we choose New/Project ? With a kernel-side safe version of the standard C/C++ runtime library ? Generating kernel-side safe code ? Golly, that would be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

8

Usually you know the size of the data being returned. From a sector such as
the MBR or Partition boot record to data being sent from a device such as
those defined in the SCSI or ATAPI specifications. If it really doesn’t
make any difference because you have structures that are being shared
between programs that are not defined by some outside entity you can place
the value of sizeof in a member and have the recipient check it against
their knowledge of that structure.

“Doron Holan” wrote in message
news:xxxxx@ntdev…
How do you do this in a maintainable manner that catches new errors?
Unless you have a hardcoded constant in your ASSERT (ie sizoef(FOO) ==
24), what are you comparing the sizeof the structure against? Also, I
use CASSERTs for these types of checks whenever possible, this way you
can’t even link :wink:

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David J. Craig
Sent: Thursday, June 09, 2005 9:02 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Funky compiler issue

You can, in cases such as this, use the offsetof macro to cause a
compiler
error if the data items don’t align. I also use the sizeof structure
test
in an assert to make sure the total size matches what I need. A little
paranoid, but sometimes you just have to do it.

“Alberto Moreira” wrote in message
news:xxxxx@ntdev…
MessageI hear so many people arguing that one should use the DDK
compiler to
build drivers, well, because, and inevitably the bogeyman follows
promptly:
do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large

volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that
are
shared by the user-side library, by the hardware-level diagnostics, and
by
the device driver. The problem is, the library is and the diags are
built
with MSVC.NET; the driver uses the DDK compiler. In all three cases, the

build is pretty standard, we use the standard makefile and build.exe for
the
driver and an off-the-shelf .sln and .dsp made for the app and for the
diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number
of
reasons. As soon as that was done, the diags started to fail. I went
down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary,
hence
its offset is 8. In other words, field a is extended with 4 bytes of
slop.
Now, I could go to either compiler and fiddle with command line options,
but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented,
to
build all three pieces of software, I wouldn’t have that problem. While
I’m
busy protecting myself of a chicken little issue that may never become
true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about
making
sure that it can correctly build drivers too ? If nothing else, how
about a
build button that, when clicked, runs build.exe on the solution ? Or
better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ?
Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

9

Got it. It was unclear to me where the size was coming from. Certainly
spec defined structures (by hw or some SIG) can be checked in this
fashion. Outside of the Size field, this breaks down for your own
structures.

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David J. Craig
Sent: Thursday, June 09, 2005 11:24 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Funky compiler issue

Usually you know the size of the data being returned. From a sector
such as
the MBR or Partition boot record to data being sent from a device such
as
those defined in the SCSI or ATAPI specifications. If it really doesn’t

make any difference because you have structures that are being shared
between programs that are not defined by some outside entity you can
place
the value of sizeof in a member and have the recipient check it against
their knowledge of that structure.

“Doron Holan” wrote in message
news:xxxxx@ntdev…
How do you do this in a maintainable manner that catches new errors?
Unless you have a hardcoded constant in your ASSERT (ie sizoef(FOO) ==
24), what are you comparing the sizeof the structure against? Also, I
use CASSERTs for these types of checks whenever possible, this way you
can’t even link :wink:

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David J. Craig
Sent: Thursday, June 09, 2005 9:02 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Funky compiler issue

You can, in cases such as this, use the offsetof macro to cause a
compiler
error if the data items don’t align. I also use the sizeof structure
test
in an assert to make sure the total size matches what I need. A little
paranoid, but sometimes you just have to do it.

“Alberto Moreira” wrote in message
news:xxxxx@ntdev…
MessageI hear so many people arguing that one should use the DDK
compiler to
build drivers, well, because, and inevitably the bogeyman follows
promptly:
do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large

volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that
are
shared by the user-side library, by the hardware-level diagnostics, and
by
the device driver. The problem is, the library is and the diags are
built
with MSVC.NET; the driver uses the DDK compiler. In all three cases, the

build is pretty standard, we use the standard makefile and build.exe for
the
driver and an off-the-shelf .sln and .dsp made for the app and for the
diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number
of
reasons. As soon as that was done, the diags started to fail. I went
down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary,
hence
its offset is 8. In other words, field a is extended with 4 bytes of
slop.
Now, I could go to either compiler and fiddle with command line options,
but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented,
to
build all three pieces of software, I wouldn’t have that problem. While
I’m
busy protecting myself of a chicken little issue that may never become
true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about
making
sure that it can correctly build drivers too ? If nothing else, how
about a
build button that, when clicked, runs build.exe on the solution ? Or
better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ?
Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

10

Alberto,

The default packing in the Microsoft compilers is 8 bytes regardless of
whether the compiler is from the DDK or MSVC++. If you are observing any
packing other than 8, then it could be that some header file is changing it
with #pragma pack or that the /Zp compiler switch is being specified to
change it.

Here are some references relating to this issue:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html
/core.2f.zp.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html
/_clang_Storage_and_Alignment_of_Structures.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html
/_core_structure_alignment.asp

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html
/_pluslang_sizeof_operator.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html
/_predir_pack.asp

  • Danilo

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: Thursday, June 09, 2005 8:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to build
drivers, well, because, and inevitably the bogeyman follows promptly: do it
the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large
volume-rendering graphics engine that implements voxel rendering in hardware

  • requires a set of relatively involuted data structures that are shared by
    the user-side library, by the hardware-level diagnostics, and by the device
    driver. The problem is, the library is and the diags are built with
    MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is
    pretty standard, we use the standard makefile and build.exe for the driver
    and an off-the-shelf .sln and .dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of
reasons. As soon as that was done, the diags started to fail. I went down to
trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence
its offset is 8. In other words, field a is extended with 4 bytes of slop.
Now, I could go to either compiler and fiddle with command line options, but
hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented, to
build all three pieces of software, I wouldn’t have that problem. While I’m
busy protecting myself of a chicken little issue that may never become true,
and insist in only using the DDK compiler for driver development, I’m
leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making
sure that it can correctly build drivers too ? If nothing else, how about a
build button that, when clicked, runs build.exe on the solution ? Or better,
a “Kernel Driver Project” pick when we choose New/Project ? With a
kernel-side safe version of the standard C/C++ runtime library ? Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

11

this message makes my eyes hurt :smiley:

…

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: 10 June 2005 04:08
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to
build drivers, well, because, and inevitably the bogeyman follows
promptly: do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large
volume-rendering graphics engine that implements voxel rendering in
hardware - requires a set of relatively involuted data structures that
are shared by the user-side library, by the hardware-level diagnostics,
and by the device driver. The problem is, the library is and the diags
are built with MSVC.NET; the driver uses the DDK compiler. In all three
cases, the build is pretty standard, we use the standard makefile and
build.exe for the driver and an off-the-shelf .sln and .dsp made for the
app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number
of reasons. As soon as that was done, the diags started to fail. I went
down to trace the error, and I had a number of structs that look like
this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary,
hence its offset is 8. In other words, field a is extended with 4 bytes
of slop. Now, I could go to either compiler and fiddle with command line
options, but hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line
options, or at least with the differences clearly and well documented,
to build all three pieces of software, I wouldn’t have that problem.
While I’m busy protecting myself of a chicken little issue that may
never become true, and insist in only using the DDK compiler for driver
development, I’m leaving my house open to the sneaky thief to come in
and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about
making sure that it can correctly build drivers too ? If nothing else,
how about a build button that, when clicked, runs build.exe on the
solution ? Or better, a “Kernel Driver Project” pick when we choose
New/Project ? With a kernel-side safe version of the standard C/C++
runtime library ? Generating kernel-side safe code ? Golly, that would
be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

12

Let me check the code today at work. That I remember, the 64-bit
field is an __int64, but I may be wrong.

Alberto.

----- Original Message -----
From: “Doron Holan”
To: “Windows System Software Devs Interest List”

Sent: Friday, June 10, 2005 12:08 AM
Subject: RE: [ntdev] Funky compiler issue

Based on alignment rules for 64 bit integers that I know of, the
DDK compiled version of the structure is the incorrect layout.
A 64 bit integer is to be aligned on an 8 byte boundary.
Something else in your headers / includes before the structure
definition is changing the layout.

I tested this out. I have just created this structure and
inserted it into a working driver and loaded the image via
windbg to inspect the symbols.

typedef struct TEST_OFFSETS {
ULONG a;
ULONGLONG b;
} TEST_OFFSETS;

0:000> dt TEST_OFFSETS
+0x000 a : Uint4B
+0x008 b : Uint8B

What is the typedef for uint64? Perhaps that is the cause of
this offset difference.

d


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David R.
Cattley
Sent: Thursday, June 09, 2005 8:38 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Funky compiler issue

While I agree with all of the tool feature requests in Alberto’s
statement below I can’t help but throw in 2 cents and say that
regardless of the compiler choices, passing information between
two disparate components, especially betweeen usermode and
kernel mode, is always a design problem requiring a precise and
unambiguous ‘in memory representation’ of the information. A
language like C or C++ is a way of expressing the representation
and a compiler is a tool for interpreting the expression. If you
want it to be precise (and you must for IOCTL buffers or on wire
representations) you cannot leave anything to chance, whim, or
default settings in the compiler. This is not a class of problem
to be fixed by having one compiler. It is an example of a
problem to be fixed by having one precise representation that is
expressed in a way that the compiler has no wiggle room to
misintrepret, regardless of the compiler, revision, or settings.
It may be ugly (due to the language extensions required to
control it) but alignment directives and packing (or dummy
fields to explicitly express the packing and force alignment)
would be required to be precise enough even if only one compiler
existed. This becomes even more critical with 32bit/64bit hybrid
environments where alignment and packing rules may differ
between user mode and kernel mode. The statement that the DDK
compiler ‘correctly’ set the offset to 4 for the 64bit field is
a matter of opinion and likely true only in a 32bit kernel mode.
In my unqualified opinion, neither 4 nor 8 is ‘correct’ unless
one of them is the one that was designed for and specified. If
the same driver were compiled for AMD64 of IA64, suddenly the
MSVC.NET choice of aligning to 8 bytes would not seem like such
a bad idea. If the design called for not wasting the 4 bytes,
both the AMD64 and IA64 drivers could be coerced into dealing
with the misalligned field (although with a performance
penalty). The trade offs in are not about what the compiler does
but what the design calls for. It is systems level software.
Don’t leave it up to the compiler to decide what that means.

Respectfully,
Dave Cattley
Consulting Engineer
Systems Software Development

_
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alberto
Moreira
Sent: Thursday, June 09, 2005 11:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue
I hear so many people arguing that one should use the DDK
compiler to build drivers, well, because, and inevitably the
bogeyman follows promptly: do it the legit way or it’ll bite you
good and hard.

Well, I just bumped into this situation, bear with me. My chip -
a large volume-rendering graphics engine that implements voxel
rendering in hardware - requires a set of relatively involuted
data structures that are shared by the user-side library, by the
hardware-level diagnostics, and by the device driver. The
problem is, the library is and the diags are built with
MSVC.NET; the driver uses the DDK compiler. In all three cases,
the build is pretty standard, we use the standard makefile and
build.exe for the driver and an off-the-shelf .sln and dsp made
for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a
number of reasons. As soon as that was done, the diags started
to fail. I went down to trace the error, and I had a number of
structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the
DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword
boundary, hence its offset is 8. In other words, field a is
extended with 4 bytes of slop. Now, I could go to either
compiler and fiddle with command line options, but hey, that’s
not the way it should be, is it ?

If I could use the same compiler, with the same default command
line options, or at least with the differences clearly and well
documented, to build all three pieces of software, I wouldn’t
have that problem. While I’m busy protecting myself of a chicken
little issue that may never become true, and insist in only
using the DDK compiler for driver development, I’m leaving my
house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how
about making sure that it can correctly build drivers too ? If
nothing else, how about a build button that, when clicked, runs
build.exe on the solution ? Or better, a “Kernel Driver Project”
pick when we choose New/Project ? With a kernel-side safe
version of the standard C/C++ runtime library ? Generating
kernel-side safe code ? Golly, that would be nice.

Alberto.

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to
xxxxx@lists.osr.com —
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to
xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to
xxxxx@lists.osr.com

13

MessageWell, this happens to be 32-bit mode. The need for a 64-bit field comes from the need to correctly represent video memory entities such as buffers (the current board has a maximum of 4Gb of memory on board, and we can expect this value to grow in the years to come) and other board-side objects. This is a unique situation, where the video board has way more memory than the motherboard! That particular 64-bit field doesn’t need to be aligned but it’d have been nice to have the same alignment in both compilers! My solution was to sort the fields in those data structures by width, 64-bit first, 32-bit next, and so on: that fixes the problem, and it only took one day of work or so to do the conversion. But still, I’d have felt better if I hadn’t had the problem to begin with.

Alberto.
----- Original Message -----
From: David R. Cattley
To: Windows System Software Devs Interest List
Sent: Thursday, June 09, 2005 11:37 PM
Subject: RE: [ntdev] Funky compiler issue

While I agree with all of the tool feature requests in Alberto’s statement below I can’t help but throw in 2 cents and say that regardless of the compiler choices, passing information between two disparate components, especially betweeen usermode and kernel mode, is always a design problem requiring a precise and unambiguous ‘in memory representation’ of the information. A language like C or C++ is a way of expressing the representation and a compiler is a tool for interpreting the expression. If you want it to be precise (and you must for IOCTL buffers or on wire representations) you cannot leave *anything* to chance, whim, or default settings in the compiler. This is not a class of problem to be fixed by having one compiler. It is an example of a problem to be fixed by having one precise representation that is expressed in a way that the compiler has no wiggle room to misintrepret, regardless of the compiler, revision, or settings. It may be ugly (due to the language extensions required to control it) but alignment directives and packing (or dummy fields to explicitly express the packing and force alignment) would be required to be precise enough even if only one compiler existed. This becomes even more critical with 32bit/64bit hybrid environments where alignment and packing rules may differ between user mode and kernel mode. The statement that the DDK compiler ‘correctly’ set the offset to 4 for the 64bit field is a matter of opinion and likely true only in a 32bit kernel mode. In my unqualified opinion, neither 4 nor 8 is ‘correct’ unless one of them is the one that was designed for and specified. If the same driver were compiled for AMD64 of IA64, suddenly the MSVC.NET choice of aligning to 8 bytes would not seem like such a bad idea. If the design called for not wasting the 4 bytes, both the AMD64 and IA64 drivers could be coerced into dealing with the misalligned field (although with a performance penalty). The trade offs in are not about what the compiler does but what the design calls for. It is systems level software. Don’t leave it up to the compiler to decide what that means.

Respectfully,
Dave Cattley
Consulting Engineer
Systems Software Development

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: Thursday, June 09, 2005 11:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to build drivers, well, because, and inevitably the bogeyman follows promptly: do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large volume-rendering graphics engine that implements voxel rendering in hardware - requires a set of relatively involuted data structures that are shared by the user-side library, by the hardware-level diagnostics, and by the device driver. The problem is, the library is and the diags are built with MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is pretty standard, we use the standard makefile and build.exe for the driver and an off-the-shelf .sln and dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of reasons. As soon as that was done, the diags started to fail. I went down to trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence its offset is 8. In other words, field a is extended with 4 bytes of slop. Now, I could go to either compiler and fiddle with command line options, but hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line options, or at least with the differences clearly and well documented, to build all three pieces of software, I wouldn’t have that problem. While I’m busy protecting myself of a chicken little issue that may never become true, and insist in only using the DDK compiler for driver development, I’m leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making sure that it can correctly build drivers too ? If nothing else, how about a build button that, when clicked, runs build.exe on the solution ? Or better, a “Kernel Driver Project” pick when we choose New/Project ? With a kernel-side safe version of the standard C/C++ runtime library ? Generating kernel-side safe code ? Golly, that would be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com —
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

14

MessageHi, Danilo,

I checked the /Zp option and it says “Default Alignment”, whatever it means. The driver is large enough as it is, and unfortunately I inherited the code, so, searching for pragmas right now isn’t really an option! But I hear you, and sometime I’ll try to go down to the bottom of this issue. Meanwhile, I moved all 64-bit fields to the beginning of the structure, and I padded all structures to a multiple of 64-bits, so, it won’t happen again, or at least I hope so.

Alberto.
----- Original Message -----
From: Danilo Almeida
To: Windows System Software Devs Interest List
Sent: Friday, June 10, 2005 3:33 AM
Subject: RE: [ntdev] Funky compiler issue

Alberto,

The default packing in the Microsoft compilers is 8 bytes regardless of whether the compiler is from the DDK or MSVC++. If you are observing any packing other than 8, then it could be that some header file is changing it with #pragma pack or that the /Zp compiler switch is being specified to change it.

Here are some references relating to this issue:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_.2f.zp.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/_clang_Storage_and_Alignment_of_Structures.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_structure_alignment.asp

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/_pluslang_sizeof_operator.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclang/html/_predir_pack.asp

  • Danilo

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alberto Moreira
Sent: Thursday, June 09, 2005 8:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Funky compiler issue

I hear so many people arguing that one should use the DDK compiler to build drivers, well, because, and inevitably the bogeyman follows promptly: do it the legit way or it’ll bite you good and hard.

Well, I just bumped into this situation, bear with me. My chip - a large volume-rendering graphics engine that implements voxel rendering in hardware - requires a set of relatively involuted data structures that are shared by the user-side library, by the hardware-level diagnostics, and by the device driver. The problem is, the library is and the diags are built with MSVC.NET; the driver uses the DDK compiler. In all three cases, the build is pretty standard, we use the standard makefile and build.exe for the driver and an off-the-shelf .sln and .dsp made for the app and for the diags.

Now, a 32-bit data field was recently upgraded to 64-bit, for a number of reasons. As soon as that was done, the diags started to fail. I went down to trace the error, and I had a number of structs that look like this:

struct BLAH { uint32 a; uint64 b; … };

Guess what ? The offset of field b is correctly generated in the DDK compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, hence its offset is 8. In other words, field a is extended with 4 bytes of slop. Now, I could go to either compiler and fiddle with command line options, but hey, that’s not the way it should be, is it ?

If I could use the same compiler, with the same default command line options, or at least with the differences clearly and well documented, to build all three pieces of software, I wouldn’t have that problem. While I’m busy protecting myself of a chicken little issue that may never become true, and insist in only using the DDK compiler for driver development, I’m leaving my house open to the sneaky thief to come in and cause me harm.

So, Microsoft, one suggestion: next release of MSVC.NET, how about making sure that it can correctly build drivers too ? If nothing else, how about a build button that, when clicked, runs build.exe on the solution ? Or better, a “Kernel Driver Project” pick when we choose New/Project ? With a kernel-side safe version of the standard C/C++ runtime library ? Generating kernel-side safe code ? Golly, that would be nice.

Alberto.

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com —
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

15

Alberto,

This problem is as old as compilers (I first encountered it more than
30 years ago personally and it was old then). The only safe solution is
assume that the alignment of an element is the sizre of an element, and pad
the structure to force this alignment. At least, Microsoft has been
consistent within a target, a nomber of the PC compiler verndors in the
early days changed alignment depending on compiler revision, and at least
one did it in a bug fix patch.

Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

----- Original Message -----
From: “Alberto Moreira”
To: “Windows System Software Devs Interest List”
Sent: Friday, June 10, 2005 8:08 AM
Subject: Re: [ntdev] Funky compiler issue

MessageWell, this happens to be 32-bit mode. The need for a 64-bit field
comes from the need to correctly represent video memory entities such as
buffers (the current board has a maximum of 4Gb of memory on board, and we
can expect this value to grow in the years to come) and other board-side
objects. This is a unique situation, where the video board has way more
memory than the motherboard! That particular 64-bit field doesn’t need to be
aligned but it’d have been nice to have the same alignment in both
compilers! My solution was to sort the fields in those data structures by
width, 64-bit first, 32-bit next, and so on: that fixes the problem, and it
only took one day of work or so to do the conversion. But still, I’d have
felt better if I hadn’t had the problem to begin with.

Alberto.

16

Well, I didn’t expect to bump into this kind of issue, not in a
32-bit environment anyway; in a 64-bit environent, that first
32-bit integer would have been compiled as 64-bit anyway, so,
the point would have been moot. I happen not to like macros very
much because they disturb the listing’s natural flow, because
they make debugging with SoftICE a rather iffy proposition (you
don’t see the code generated by the macro in your source unless
you do special things, and I rarely have the time to bother
about that sort of thing), and because, well, it’s additional
work that I’d have to put in and I’m typically not willing to do
it! Specially on an issue that I believe shouldn’t have happened
to begin with. Not to be overcritical, but a driver must talk to
its app(s), and the least one expects is that a fully-packaged
solution-oriented development environment such as the DDK would
give you a default environment that’s fully compatible between
driver and app. What I did was to sort the fields in the data
structures, 64-bit fields first, 32-bit fields next, and I also
padded all structures to a 64-bit boundary. But my point was,
using the DDK compiler instead of, say, MSVC.NET, also has its
pitfalls: it’s no panacea against creeping runtime issues.

Alberto.

----- Original Message -----
From: “David J. Craig”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”

Sent: Friday, June 10, 2005 12:01 AM
Subject: Re:[ntdev] Funky compiler issue

> You can, in cases such as this, use the offsetof macro to
> cause a compiler error if the data items don’t align. I also
> use the sizeof structure test in an assert to make sure the
> total size matches what I need. A little paranoid, but
> sometimes you just have to do it.
>
> “Alberto Moreira” wrote in message
> news:xxxxx@ntdev…
> MessageI hear so many people arguing that one should use the
> DDK compiler to build drivers, well, because, and inevitably
> the bogeyman follows promptly: do it the legit way or it’ll
> bite you good and hard.
>
> Well, I just bumped into this situation, bear with me. My
> chip - a large volume-rendering graphics engine that
> implements voxel rendering in hardware - requires a set of
> relatively involuted data structures that are shared by the
> user-side library, by the hardware-level diagnostics, and by
> the device driver. The problem is, the library is and the
> diags are built with MSVC.NET; the driver uses the DDK
> compiler. In all three cases, the build is pretty standard, we
> use the standard makefile and build.exe for the driver and an
> off-the-shelf .sln and .dsp made for the app and for the
> diags.
>
> Now, a 32-bit data field was recently upgraded to 64-bit, for
> a number of reasons. As soon as that was done, the diags
> started to fail. I went down to trace the error, and I had a
> number of structs that look like this:
>
> struct BLAH { uint32 a; uint64 b; … };
>
> Guess what ? The offset of field b is correctly generated in
> the DDK compiler (equal to 4) but MSVC.NET aligns it to a
> quadword boundary, hence its offset is 8. In other words,
> field a is extended with 4 bytes of slop. Now, I could go to
> either compiler and fiddle with command line options, but hey,
> that’s not the way it should be, is it ?
>
> If I could use the same compiler, with the same default
> command line options, or at least with the differences clearly
> and well documented, to build all three pieces of software, I
> wouldn’t have that problem. While I’m busy protecting myself
> of a chicken little issue that may never become true, and
> insist in only using the DDK compiler for driver development,
> I’m leaving my house open to the sneaky thief to come in and
> cause me harm.
>
> So, Microsoft, one suggestion: next release of MSVC.NET, how
> about making sure that it can correctly build drivers too ? If
> nothing else, how about a build button that, when clicked,
> runs build.exe on the solution ? Or better, a “Kernel Driver
> Project” pick when we choose New/Project ? With a kernel-side
> safe version of the standard C/C++ runtime library ?
> Generating kernel-side safe code ? Golly, that would be nice.
>
>
> Alberto.
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ieee.org
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com

17

Hi, Don,

Yes, you’re right of course, the issue is old. But my point is,
Microsoft distributes the DDK as the preferred development
environment to develop drivers, but drivers talk to applications
and applications are built with other compilers using other
preferred development environments. Drivers must talk to their
applications, so, one would expect that if one builds a driver
and an app using Microsoft’s own development environments with
basically default compiler and link switches, one should get
compatibility between a driver and its app. Note that I’m not
blaming Microsoft for that, because as other posters pointed
out, other macros in the program might have thrown the alignment
off balance; yet, I don’t know, even a simple compiler warning
would have been better than nothing.

I still believe that the best solution is to integrate the DDK’s
environment into MSVC.NET, with a build button for those who
prefer it.

Alberto.

----- Original Message -----
From: “Don Burn”
To: “Windows System Software Devs Interest List”

Sent: Friday, June 10, 2005 8:17 AM
Subject: Re: [ntdev] Funky compiler issue

> Alberto,
>
> This problem is as old as compilers (I first encountered
> it more than 30 years ago personally and it was old then).
> The only safe solution is assume that the alignment of an
> element is the sizre of an element, and pad the structure to
> force this alignment. At least, Microsoft has been consistent
> within a target, a nomber of the PC compiler verndors in the
> early days changed alignment depending on compiler revision,
> and at least one did it in a bug fix patch.
>
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Remove StopSpam from the email to reply
>
>
> ----- Original Message -----
> From: “Alberto Moreira”
> To: “Windows System Software Devs Interest List”
>
> Sent: Friday, June 10, 2005 8:08 AM
> Subject: Re: [ntdev] Funky compiler issue
>
>
> MessageWell, this happens to be 32-bit mode. The need for a
> 64-bit field comes from the need to correctly represent video
> memory entities such as buffers (the current board has a
> maximum of 4Gb of memory on board, and we can expect this
> value to grow in the years to come) and other board-side
> objects. This is a unique situation, where the video board has
> way more memory than the motherboard! That particular 64-bit
> field doesn’t need to be aligned but it’d have been nice to
> have the same alignment in both compilers! My solution was to
> sort the fields in those data structures by width, 64-bit
> first, 32-bit next, and so on: that fixes the problem, and it
> only took one day of work or so to do the conversion. But
> still, I’d have felt better if I hadn’t had the problem to
> begin with.
>
> Alberto.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ieee.org
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com

18

At the risk of “stirring the pot”, I’ll add my two cents here…

It would be nice if I could use the DDK build environment to build my
driver’s configuration applets and related software without hacking include
files. Last time I did this (Windows Server 2003 3790 DDK), I had to copy
richedit.h and zmouse.h to my project directory to get things to build.
It’s strange those two include files weren’t in one of the DDK’s include
directories when references to them were.

I suppose I should file a bug on this (and check 3790.1830 and the lastest
WDK to see if it has the same problem). Should I file bugs against both the
DDK and WDK? Or is one report sufficient?

… and where’d I put that list of feedback aliases … ?

-Dan

----- Original Message -----

Subject: Funky compiler issue
From: “Alberto Moreira”
> Date: Thu, 9 Jun 2005 23:08:18 -0400
>
> MessageI hear so many people arguing that one should use the DDK =
> compiler to build drivers, well, because, and inevitably the bogeyman =
> follows promptly: do it the legit way or it’ll bite you good and hard.=20
>
> Well, I just bumped into this situation, bear with me. My chip - a large =
> volume-rendering graphics engine that implements voxel rendering in =
> hardware - requires a set of relatively involuted data structures that =
> are shared by the user-side library, by the hardware-level diagnostics, =
> and by the device driver. The problem is, the library is and the diags =
> are built with MSVC.NET; the driver uses the DDK compiler. In all three =
> cases, the build is pretty standard, we use the standard makefile and =
> build.exe for the driver and an off-the-shelf .sln and .dsp made for the =
> app and for the diags.=20
>
> Now, a 32-bit data field was recently upgraded to 64-bit, for a number =
> of reasons. As soon as that was done, the diags started to fail. I went =
> down to trace the error, and I had a number of structs that look like =
> this:
>
> struct BLAH { uint32 a; uint64 b; … };
>
> Guess what ? The offset of field b is correctly generated in the DDK =
> compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary, =
> hence its offset is 8. In other words, field a is extended with 4 bytes =
> of slop. Now, I could go to either compiler and fiddle with command line =
> options, but hey, that’s not the way it should be, is it ?=20
>
> If I could use the same compiler, with the same default command line =
> options, or at least with the differences clearly and well documented, =
> to build all three pieces of software, I wouldn’t have that problem. =
> While I’m busy protecting myself of a chicken little issue that may =
> never become true, and insist in only using the DDK compiler for driver =
> development, I’m leaving my house open to the sneaky thief to come in =
> and cause me harm.=20
>
> So, Microsoft, one suggestion: next release of MSVC.NET, how about =
> making sure that it can correctly build drivers too ? If nothing else, =
> how about a build button that, when clicked, runs build.exe on the =
> solution ? Or better, a “Kernel Driver Project” pick when we choose =
> New/Project ? With a kernel-side safe version of the standard C/C++ =
> runtime library ? Generating kernel-side safe code ? Golly, that would =
> be nice.=20
>
>
> Alberto.

19

Daniel E. Germann wrote:

I suppose I should file a bug on this (and check 3790.1830 and the
lastest WDK to see if it has the same problem).
>

Yes, please.

Should I file bugs
against both the DDK and WDK? Or is one report sufficient?

One report is usually sufficient. Given that the S03 SP1 DDK has now
been released, the next DDK release will be the WDK. So please file a
bug against the DDK.

… and where’d I put that list of feedback aliases … ?

Best, if you’re in the beta program (for anything) file the bug via the
beta program. Just be SURE to mark is clearly as a bug in the DDK.
Keep the external bug number that you get for follow-up.

Second best, send email to WDKFB. When sending email to WDKFB, make it
clear that this is a BUG REPORT, the release of the WDK that you’ve
confirmed it in, and ask (nicely) for the (internal) bug number so later
you can “followup either directly or through PSS” to find out what happens.

You can also file DDK bugs via your TAM (if you company has one) or via
PSS DDK Support.

Peter
OSR

20

Hi All,

Ok, this thread is especially interesting to me so I’d like the chance
to throw in my $0.02 also. I don’t think I ever completely trust a
compiler to always to the right thing (too many bad experiences with
small embedded compilers). And when it comes to passing data between
multiple components built with different compilers (or versions of
compilers), I never would have considered it, because as I recall (maybe
incorrectly), the C / C++ standards do not specify alignment or
packing of structs. I would have used the following approach to
transfer the data between the 2 components and taken the compiler(s) out
of the picture:

//************************************************
// Shared defines and declaration for required buffer on each end
//************************************************
#define START_a 0 // uint32
#define START_b 4 // uint64
#define BLAH_SIZE 12

extern UCHAR BlahXferBuffer[BLAH_SIZE];

//************************************************
// Routine to send data somewhere
//************************************************
VOID SendData(struct Blah *myblah)
{
//
// Make sure the data is in a known format
//
RtlCopyMemory(&BlahXferBuffer[START_a], myblah->a, 4);
RtlCopyMemory(&BlahXferBuffer[START_b], myblah->b, 8);

//
// Send the known formatted buffer on it’s way
//
XferData(BlahXferBuffer);
}

//************************************************
// Routine to get this data
//************************************************
VOID Receive Data(struct Blah *myblah)
{
//
// Get the data from wherever
//
GetData(BlahXferBuffer);

//
// Pull it back out and populate whatever data struct you want
//
RtlCopyMemory( myblah->a, &BlahXferBuffer[START_a], 4);
RtlCopyMemory( myblah->b, &BlahXferBuffer[START_b], 8);
}

Yes, it’s ugly and involves memcpys, but it works, is unambiguous, and
leaves nothing to chance. (Well, ok, more could be done to account for
sizes of the parameters, but I just wanted to get the idea out.)

Anyway, just my opinion.

Best Regards,
-Mike

David R. Cattley wrote:

While I agree with all of the tool feature requests in Alberto’s
statement below I can’t help but throw in 2 cents and say that
regardless of the compiler choices, passing information between two
disparate components, especially betweeen usermode and kernel mode, is
always a design problem requiring a precise and unambiguous ‘in
memory representation’ of the information. A language like C or C++
is a way of expressing the representation and a compiler is a tool for
interpreting the expression. If you want it to be precise (and you
must for IOCTL buffers or on wire representations) you cannot leave
*anything* to chance, whim, or default settings in the compiler. This
is not a class of problem to be fixed by having one compiler. It is
an example of a problem to be fixed by having one precise
representation that is expressed in a way that the compiler has no
wiggle room to misintrepret, regardless of the compiler, revision, or
settings. It may be ugly (due to the language extensions required to
control it) but alignment directives and packing (or dummy fields to
explicitly express the packing and force alignment) would be required
to be precise enough even if only one compiler existed. This becomes
even more critical with 32bit/64bit hybrid environments where
alignment and packing rules may differ between user mode and kernel
mode. The statement that the DDK compiler ‘correctly’ set the offset
to 4 for the 64bit field is a matter of opinion and likely true only
in a 32bit kernel mode. In my unqualified opinion, neither 4 nor 8 is
‘correct’ unless one of them is the one that was designed for and
specified. If the same driver were compiled for AMD64 of IA64,
suddenly the MSVC.NET choice of aligning to 8 bytes would not seem
like such a bad idea. If the design called for not wasting the 4
bytes, both the AMD64 and IA64 drivers could be coerced into dealing
with the misalligned field (although with a performance penalty). The
trade offs in are not about what the compiler does but what the design
calls for. It is systems level software. Don’t leave it up to the
compiler to decide what that means.

Respectfully,
Dave Cattley
Consulting Engineer
Systems Software Development

*From:* xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] *On Behalf Of *Alberto Moreira
*Sent:* Thursday, June 09, 2005 11:08 PM
*To:* Windows System Software Devs Interest List
*Subject:* [ntdev] Funky compiler issue

*/I hear so many people arguing that one should use the DDK compiler
to build drivers, well, because, and inevitably the bogeyman follows
promptly: do it the legit way or it’ll bite you good and hard. /*
*//*
*/Well, I just bumped into this situation, bear with me. My chip - a
large volume-rendering graphics engine that implements voxel rendering
in hardware - requires a set of relatively involuted data structures
that are shared by the user-side library, by the hardware-level
diagnostics, and by the device driver. The problem is, the library is
and the diags are built with MSVC.NET; the driver uses the DDK
compiler. In all three cases, the build is pretty standard, we use the
standard makefile and build.exe for the driver and an off-the-shelf
.sln and dsp made for the app and for the diags. /*
*//*
*/Now, a 32-bit data field was recently upgraded to 64-bit, for a
number of reasons. As soon as that was done, the diags started to
fail. I went down to trace the error, and I had a number of structs
that look like this:/*
*//*
*/ struct BLAH { uint32 a; uint64 b; … };/*
*//*
*/Guess what ? The offset of field b is correctly generated in the DDK
compiler (equal to 4) but MSVC.NET aligns it to a quadword boundary,
hence its offset is 8. In other words, field a is extended with 4
bytes of slop. Now, I could go to either compiler and fiddle with
command line options, but hey, that’s not the way it should be, is it ? /*
*//*
*/If I could use the same compiler, with the same default command
line options, or at least with the differences clearly and well
documented, to build all three pieces of software, I wouldn’t have
that problem. While I’m busy protecting myself of a chicken little
issue that may never become true, and insist in only using the DDK
compiler for driver development, I’m leaving my house open to the
sneaky thief to come in and cause me harm. /*
*//*
*/So, Microsoft, one suggestion: next release of MSVC.NET, how about
making sure that it can correctly build drivers too ? If nothing else,
how about a build button that, when clicked, runs build.exe on the
solution ? Or better, a “Kernel Driver Project” pick when we choose
New/Project ? With a kernel-side safe version of the standard C/C++
runtime library ? Generating kernel-side safe code ? Golly, that would
be nice. /*
*//*
*//*
*/Alberto./*
*//*
*//*

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com —
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com