BSOD!!! (bugcheck 0x3b)

NTDEV
21

> There is no possible way to allocate user memory at location 0.

It was not possible in Win16 and it was never possible in Win32.
Several APIs critically depend on this behavior.

It wasn’t exposed through any win32 API but by calling NT APIs
directly you could in fact allocate memory at NULL.

As far as I know, the only people using this feature were exploit
writers, which is why it was disabled in win8:

http://blogs.msdn.com/b/b8/archive/2011/09/15/protecting-you-from-malware.aspx

“…we now prevent user-mode processes from allocating the low 64K
of process memory, which prevents a whole class of kernel-mode NULL
dereference vulnerabilities from being exploited.”

22

Jan Bottorff wrote:

That’s curious. I wonder how many pieces of library code think 00000000 means invalid address. About a million lines of C code also go:

If (somePointer) {
… valid path
} else {
… invalid path
}

Sounds like they should say:

If (somePointer != INVALID_POINTER)…

That depends on what you mean. This is a complicated
philosophical/theoretical situation with implications both for the
language and the operating system. The C standard requires that the
null pointer act AS IF it has the value 0. So:

if (!somePointer) {
// This path must be taken if somePointer is the null pointer.
}

The C standard does NOT require that the null pointer actually be a bit
pattern of all zeros, although virtually every implementation does that.

The C standard also does NOT require that dereferencing a null pointer
must fail. The behavior in that case is undefined. The compiler can do
what it wants. In Microsoft’s 16-bit C compiler, dereferencing a null
pointer produced legitimate results. The fact that 32-bit Windows makes
a reference to address 0 fail is a programming and debugging aid.

The net result of all of this is that, IF you actually store something
at address 0, the compiler is NOT required by the standard to let you
access it. 0 is an invalid address in Microsoft’s compiler, because
that it the value if the null pointer.

Is there some specific value that is DEFINED to be an invalid address? Like for uninitialized pointers?

Yes: 0.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

23

C code can compare against NULL because malloc (or any other supported win32/CRT API) will never return a “valid” NULL pointer.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jan Bottorff
Sent: Thursday, November 15, 2012 9:03 PM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

That’s curious. I wonder how many pieces of library code think 00000000 means invalid address. About a million lines of C code also go:

If (somePointer) {
… valid path
} else {
… invalid path
}

Sounds like they should say:

If (somePointer != INVALID_POINTER)…

Is there some specific value that is DEFINED to be an invalid address? Like for uninitialized pointers?

24

xxxxx@flounder.com wrote

By definition, an invalid address is any address which is not valid.

By whose definition, and for what purpose? This statement is simply way
too broad. There are too many competing contexts. To a user-mode
Windows application, any address within an unmapped page is “invalid”
because it causes a page fault. Any address within kernel space is
“invalid” because it causes a protection fault. However, all of that is
merely a by-product of using a virtual memory operating system. But as
far as C is concerned, every one of those 4 billion addresses are valid,
except the one with a bit pattern that matches the “null pointer”.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

25

Jan Bottorff wrote:

I also see that C++11 has fixed this problem, by the introduction of nullptr, which is a unique type.

That doesn’t fix this problem at all. It makes it easier to abstract
the null pointer concept, and easier for the compiler to diagnose null
pointer issues, but you still have exactly the same issue at the
compiler/implementation interface. If I store the nullptr in a char*
and dereference it, what happens? The standard doesn’t address that.
Really, it CAN’T address that. It’s an implementation detail.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

26

Pavel Lebedynskiy wrote:

C code can compare against NULL because malloc (or any other supported win32/CRT API) will never return a “valid” NULL pointer.

For philosophical purposes, you MUST differentiate between the compiler
and the underlying operating system. It’s a CRITICAL difference. The C
runtime library will never return 0 for a valid pointer – that’s
required by the standard. However, the Win32 API is not constrained by
the standards of any mere language committee. It is perfectly free to
return 0 as a legitimate address, although that would be unexpected.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

27

xxxxx@gmail.com wrote:

Thank you all.

Umm…
DLPMediaControlPassThrough and FsFilterDispatchPassThrough is passthrough dispatch routine.
This routine uses only IoSkipCurrentIrpStackLocation & IoCallDriver.
…
DLPMediaControlPassThrough’s IoCallDriver is fltmgr!FltpDispatch is called.
Then IoCallDriver problem?

You mean a bug in IoCallDriver? No, the bug is in your code,
somewhere. You said this was at system shutdown, right? If I had to
guess (and it’s only a wild guess), I’d guess you are clearing out some
field in a data structure before all of the system components were
through using it. Perhaps some interlock problem.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

28

> xxxxx@flounder.com wrote

> By definition, an invalid address is any address which is not valid.

By whose definition, and for what purpose? This statement is simply way
too broad. There are too many competing contexts. To a user-mode
Windows application, any address within an unmapped page is “invalid”
because it causes a page fault. Any address within kernel space is
“invalid” because it causes a protection fault. However, all of that is
merely a by-product of using a virtual memory operating system. But as
far as C is concerned, every one of those 4 billion addresses are valid,
except the one with a bit pattern that matches the “null pointer”.

“By whose definition” is easy: the configuration of the page tables at the
time the reference is made. That’s it. There are four things that can
happen:

  1. the address is mapped from virtual to physical space, and the physical
    memory is read or written.
  2. The address is mapped from virtual to physical space, and the
    attributes in the page table say it is “write protected” and the operation
    is a write operation. An access-denied exception is generated
  3. The address in the page table has the “not present” bit set, and an
    exception is generated. The operating system determines what page needs
    to be paged in, pages it in, and go back to the top of this list and start
    over now that the page table has been fixed up
  4. The address is not in the page table; that is, it is either marked as
    “not present” or, given the hierarchical structure of page tables, a
    higher-level node which has no allocated pages says “not present”, in
    which case an access-denied exception is taken.

Any address which is not defined as valid is necessarily invalid.

Note that the reason a user-mode process cannot access a kernel page is
that the page is not in the process’s page tables. That’s what all this
“I want to map kernel space into user space” foofraw is about: getting the
page map to allow writing to that set of pages.

In the case of Windows, several specific addresses are seen as “invalid”,
for example, kernel pages from user space (not in the user space map),
0x0000…0xFFFF. Windows has well-defined behavior for accessing these
locations (which is not the same as C, which says
“implementation-specific” or similar wording). Any given environment will
have rules for what happens; for example, on the PDP-10/20, location 0 was
also register 0, so if you dereferenced a NULL pointer and stored through
it, you could clobber some other register, but it was only noticeable if
you clobbered whatever register the language used as its stack register.
Again, the behavior was well-defined in that particular environment.

Since this forum discusses Microsoft Windows and kernel mode in
particular, we can use the definition of “invalid” as implemented by the
operating system. So ultimately, the operating environment determines
what pages are invalid and the hardware is configured so that pages which
are not mapped are invalid.
joe

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

29

> xxxxx@gmail.com wrote:

> Thank you all.
>
> Umm…
> DLPMediaControlPassThrough and FsFilterDispatchPassThrough is
> passthrough dispatch routine.
> This routine uses only IoSkipCurrentIrpStackLocation & IoCallDriver.
> …
> DLPMediaControlPassThrough’s IoCallDriver is fltmgr!FltpDispatch is
> called.
> Then IoCallDriver problem?

You mean a bug in IoCallDriver? No, the bug is in your code,
somewhere. You said this was at system shutdown, right? If I had to
guess (and it’s only a wild guess), I’d guess you are clearing out some
field in a data structure before all of the system components were
through using it. Perhaps some interlock problem.

Or, freeing a structure while someone else has what they think is a valid
pointer to it; the act of freeing storage can cause some fields to be
overwritten by the allocator’s internal bookkeeping fields.
joe

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

30

> Pavel Lebedynskiy wrote:

> C code can compare against NULL because malloc (or any other supported
> win32/CRT API) will never return a “valid” NULL pointer.

For philosophical purposes, you MUST differentiate between the compiler
and the underlying operating system. It’s a CRITICAL difference. The C
runtime library will never return 0 for a valid pointer – that’s
required by the standard. However, the Win32 API is not constrained by
the standards of any mere language committee. It is perfectly free to
return 0 as a legitimate address, although that would be unexpected.

But it never, ever would because a great deal of functionality is
predicated on the fact that locations 0x0000…0xFFFF are invalid memory
addresses. So there is no way Windows would or could return a “valid
pointer” whose value was in this range.
joe

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

31

> Jan Bottorff wrote:

> That’s curious. I wonder how many pieces of library code think 00000000
> means invalid address. About a million lines of C code also go:
>
> If (somePointer) {
> … valid path
> } else {
> … invalid path
> }
>
> Sounds like they should say:
>
> If (somePointer != INVALID_POINTER)…

That depends on what you mean. This is a complicated
philosophical/theoretical situation with implications both for the
language and the operating system. The C standard requires that the
null pointer act AS IF it has the value 0. So:

if (!somePointer) {
// This path must be taken if somePointer is the null pointer.
}

The C standard does NOT require that the null pointer actually be a bit
pattern of all zeros, although virtually every implementation does that.

The C standard also does NOT require that dereferencing a null pointer
must fail. The behavior in that case is undefined. The compiler can do
what it wants. In Microsoft’s 16-bit C compiler, dereferencing a null
pointer produced legitimate results. The fact that 32-bit Windows makes
a reference to address 0 fail is a programming and debugging aid.

Actually, that only happened if you dereferenced a NEAR (16-bit) NULL
pointer; it gave you offset 0 in the default data segment, which was
limited to 64K. If you dereferenced a FAR (32-bit) NULL pointer, you got
a GPF (General Protection Fault).

I use the same example when I discuss the use of __based pointers; a 0
__based pointer is a legitimate pointer, and must be distinguished from
NULL in various ways. This requires great care in coding, and I have
wonderful counterexamples to show why apparently correct code will fail if
converted to use __based pointers. But we are not talking about __based
pointers or Win16 NEAR pointers here; we are talking about the behavior of
Win32/64.

The net result of all of this is that, IF you actually store something
at address 0, the compiler is NOT required by the standard to let you
access it. 0 is an invalid address in Microsoft’s compiler, because
that it the value if the null pointer.

You can’t store anything at location 0 in Windows because it does not
exist. Nor do the following 65,535 locations.
joe

> Is there some specific value that is DEFINED to be an invalid address?
> Like for uninitialized pointers?

Yes: 0.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

32

xxxxx@flounder.com wrote:

> xxxxx@flounder.com wrote
>> By definition, an invalid address is any address which is not valid.
> By whose definition, and for what purpose? This statement is simply way
> too broad. There are too many competing contexts. …
“By whose definition” is easy: the configuration of the page tables at the
time the reference is made.

…in Windows. In Win32, specifically.

So when you say “By definition”, what you’re really saying is “By
definition, in Windows,…” That’s really the point I was trying to
make. Your statement was global and unconditional, but in fact your
comment assumes a very specific and very detailed mental context. The
original question addressed aspects of C and aspect of Windows, and the
two contexts have very different assumptions.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

33

Pavel is completely correct in his statement. Not a supported behavior that is part of the documented API (and never was), however.

  • S (Msft), who personally contributed some of the code to forbid this in Win8.

From: xxxxx@flounder.com
Sent: 11/15/2012 23:15
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

> In user space, addresses 00000000 to 0000FFFF do not exist.

That’s the case usually, but not always. On Windows versions prior to win8
user apps can allocate memory at NULL.

No. This was never possible. It was not possible in Win16 and it was
never possible in Win32. Several APIs critically depend on this behavior.

There is no possible way to allocate user memory at location 0. What is
your evidence that this could be done?

> You have a bug. You have to find and fix the bug. There is no
> alternative.

We are in agreement here.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@flounder.com
Sent: Thursday, November 15, 2012 7:55 PM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

No, 0 is an address. In user space, addresses 00000000 to 0000FFFF do not
exist. In kernel mode, this is interpreted according to the active
mapping, which means that 00000000 to 0000FFFF do not exist. The fact
that they are interpreted in the lower section of memory (<2GB, <3GB, <8TB
depending on the OS configuration) doesn’t (a) change the fact that they
are invalid and (b) adding an exception handler will somehow magically
will make your defective driver “right”.

You have a bug. You have to find and fix the bug. There is no
alternative.

When I had to do exception handling based on illegal addresses, it was
entirely due to hardware failure. The program, as far as we could tell,
was correct. But the hardware was experimental, and we had to keep
running in spite of defective hardware. But in this case, it is clear
that the driver is defective. So you have to fix it.
joe

> NULL is a user space address so technically you *can* handle access
> violations resulting from dereferencing a NULL pointer.
>
> The answer to whether you *should* be doing this is, of course, a big
> NO.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

34

NULL is off limits to those who follow the rules. An allocator that follows the rules and uses the documented support in Win32 will never hand NULL back from a successful allocation.

  • S (Msft)

From: Jan Bottorff
Sent: 11/15/2012 21:05
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

That’s curious. I wonder how many pieces of library code think 00000000 means invalid address. About a million lines of C code also go:

If (somePointer) {
… valid path
} else {
… invalid path
}

Sounds like they should say:

If (somePointer != INVALID_POINTER)…

Is there some specific value that is DEFINED to be an invalid address? Like for uninitialized pointers?

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Pavel Lebedynskiy
Sent: Thursday, November 15, 2012 8:25 PM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

In user space, addresses 00000000 to 0000FFFF do not exist.

That’s the case usually, but not always. On Windows versions prior to win8 user apps can allocate memory at NULL.

You have a bug. You have to find and fix the bug. There is no alternative.

We are in agreement here.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@flounder.com
Sent: Thursday, November 15, 2012 7:55 PM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] BSOD!!! (bugcheck 0x3b)

No, 0 is an address. In user space, addresses 00000000 to 0000FFFF do not exist. In kernel mode, this is interpreted according to the active mapping, which means that 00000000 to 0000FFFF do not exist. The fact that they are interpreted in the lower section of memory (<2GB, <3GB, <8TB depending on the OS configuration) doesn’t (a) change the fact that they are invalid and (b) adding an exception handler will somehow magically will make your defective driver “right”.

You have a bug. You have to find and fix the bug. There is no alternative.

When I had to do exception handling based on illegal addresses, it was entirely due to hardware failure. The program, as far as we could tell, was correct. But the hardware was experimental, and we had to keep running in spite of defective hardware. But in this case, it is clear that the driver is defective. So you have to fix it.
joe

NULL is a user space address so technically you *can* handle access
violations resulting from dereferencing a NULL pointer.

The answer to whether you *should* be doing this is, of course, a big NO.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

35

> Note that the reason a user-mode process cannot access a

kernel page is that the page is not in the process’s page tables.

No, it’s because page table entries that describe kernel space have the supervisor bit set.

CR3 doesn’t change when a user thread makes a system call and switches into the kernel. Kernel PTEs are always there, the only reason threads can’t use them while executing in user mode is the supervisor bit.

36

> The C standard does NOT require that the null pointer actually be a bit

pattern of all zeros, although virtually every implementation does that.

C++ requires (PointerType)0 to be the null pointer. Yes, the bit value can become nonzero.

The C standard also does NOT require that dereferencing a null pointer
must fail. The behavior in that case is undefined.

C++ requires that null pointer is different in == comparison from any pointer to a valid object.

This can be achieved by avoiding of placement of any objects at 0. Crash is not a must.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

37

On 16/11/2012 16:38, Maxim S. Shatskih wrote:

>From what Pavel says (and I usually think of him as a pretty reliable source), the address 0 MIGHT be
> a valid address on pre Win8 systems

C and C++ languages define (PVOID)0 to be different from any valid pointer.

That’s true, but they don’t require the bit pattern of the address
generated by that expression to be 0.