Code Executing at or above DIPATCH_LEVEL must not cause page faults

NTDEV
1

why is this limitaion. what happens if i access paged memory at or above dispatch level?

2

This is a limitation because paging I/O runs below DISPATCH_LEVEL. If you
access paged memory at DISPATCH_LEVEL or higher you will crash if the memory
is paged out. To help you find these, Driver Verifier will page out all the
memory so you are sure to crash if you reference paged memory at those
level.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@yahoo.in
Sent: Monday, March 31, 2014 8:47 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Code Executing at or above DIPATCH_LEVEL must not cause
page faults

why is this limitaion. what happens if i access paged memory at or above
dispatch level?

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

3

> why is this limitaion. what happens if i access paged memory at or above

dispatch level?

Death.

Of the Blue variety.

But here’s the thing: MAYBE. Sort of like Russian Roulette. You can spin
and click 100 times, and nothing happens. It doesn’t mean you are engaged
in a safe activity.

If you access a page, and it is in memory and the map is still valid,
nothing happens; the data is read (or written, if the privileges on the
page allow that). So you say “Gee, this works!” Now, you tested it on
your 16GB development machine, running nothing but your test app. But
then a customer installs it on their server. They are running Apache/IIS
or SQL Server/Oracle, supporting several hundred queries per second. That
is likely to generate a lot of “memory pressure”, and guess what? That
page that remained resident on your machine has been paged out. You take
a page fault.

What happens on a page fault? Well, the thread that generated it is put
on a queue associated with that page image. It sits there until the page
is brought back into memory and the page map is updated. Then that thread
is allowed to run again.

But to deschedule the thread and reschedule it, the thread must be running
at PASSIVE_LEVEL. If it were to happen at DISPATCH_LEVEL, the thread
would be enqueued, but that requires that the thread has blocked the CPU
until that page is brought in. No other I/O can take place, because you
have blocked (indefinitely) all DPC activity on that core. So what
happens if (a) you have only one core and (b) if you have as many page
faults as cores? Well, the machine locks up. Critical devices cannot
process their DPCs. Transfer latency goes up, unbounded. The world
crumbles to dust. You are in Hell, and must abandon all hope. A function
running at DISPATCH_LEVEL must complete, and complete within a specified
upper bound (I think 50us or something on that order). Tens of
milliseconds waiting for disk transfers (remember: to make room for the
page you want, some other page might first have to be written out to
disk!) exceed this bound by too many orders of magnitude to be reasonable.

So rather than try to cope with this bizarre situation, the rule is very
simple: you Do Not Touch Paged Memory At Elevated IRQL. This is not
negotiable. Furthermore, there is never a reason to need to touch paged
memory at elevated IRQL. MmProbeAndLockPages, among other calls, will
temporarily “promote” pageable memory to the non-paged pool. When the MDL
that is used to accomplish this is freed, the memory reverts back to being
pageable again. The MDL that comes in for direct I/O modes is created by
the I/O Manager and is freed by the I/O Manager.

Life gets more interesting when you have a problem such as “I want to be
able to write any size of buffer on any machine”. This means that if you
have a 1GB application buffer, you want to support the ability to write it
out on a machine with 512MB memory (no, don’t say that you are unlikely to
find one that small today; instead, take these numbers as illustrative; it
might be a 16GB buffer on a 16GB machine, which is a little awkward if you
want memory to run the OS).

What you have to do is “Mode Neither”, lock down some block of pages,
write them out (typically using DMA), then unlock them, create a MDL for
the next block, repeat until all pages written. So sometimes the built-in
MDL creation works against your drive requirements.

But until those pages are locked down, they are untouchable by
DISPATCH_LEVEL. This means that when you are at DISPATCH_LEVEL, you have
to get to PASSIVE_LEVEL to lock down the next batch of pages; one way to
do this is to have a driver thread that does MmProbeAndLockPages (and can
legitimately take page faults), creates a MDL, initiates the next I/O
transaction, and goes back to sleep until it is needed again.
joe

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

4

I thought DV only removed the pageable entries in the map, rather than
physically removing the pages.
joe

This is a limitation because paging I/O runs below DISPATCH_LEVEL. If
you
access paged memory at DISPATCH_LEVEL or higher you will crash if the
memory
is paged out. To help you find these, Driver Verifier will page out all
the
memory so you are sure to crash if you reference paged memory at those
level.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@yahoo.in
Sent: Monday, March 31, 2014 8:47 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Code Executing at or above DIPATCH_LEVEL must not cause
page faults

why is this limitaion. what happens if i access paged memory at or above
dispatch level?

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

5

Inpage IO operation includes waiting on an event, which is prohibited on DISPATCH

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
> why is this limitaion. what happens if i access paged memory at or above dispatch level?
>

6

followed by…

A detail of implementation, but yes. It “pages out all the memory” logically, by marking the page as being non-resident.

Peter
OSR

7

As Peter pointed out it is a logical “page out”. Since the OP was asking an
extremely basic question, I wanted the answer to be basic also. I’ve been
encountering a number of inexperienced driver writers lately who went off
the rails, because they got too hung up on the details of the answers from
this and other groups. I did not want the OP thinking “How do I reset the
resident flag, so I can access paged memory at DISPATCH_LEVEL?”, this is the
type of thing I am seeing lately in some of my consults.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Tuesday, April 01, 2014 8:56 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Code Executing at or above DIPATCH_LEVEL must not cause
page faults

[quote]
Driver Verifier will page out all the memory [/quote]

followed by…

[quote]
I thought DV only removed the pageable entries in the map [/quote]

A detail of implementation, but yes. It “pages out all the memory”
logically, by marking the page as being non-resident.

Peter
OSR

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

8

+1

Peter
OSR
@OSRDrivers

9

I’ve had a different reaction: “We couldn’t possibly use that feature,
even for testing, because it pages all those pages out, then they have to
all be paged back in again, and that would take just *forever*”. So I
have to explain that all DV does is “hide” the page.
joe

As Peter pointed out it is a logical “page out”. Since the OP was asking
an
extremely basic question, I wanted the answer to be basic also. I’ve been
encountering a number of inexperienced driver writers lately who went off
the rails, because they got too hung up on the details of the answers from
this and other groups. I did not want the OP thinking “How do I reset
the
resident flag, so I can access paged memory at DISPATCH_LEVEL?”, this is
the
type of thing I am seeing lately in some of my consults.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Tuesday, April 01, 2014 8:56 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Code Executing at or above DIPATCH_LEVEL must not
cause
page faults

[quote]
Driver Verifier will page out all the memory [/quote]

followed by…

[quote]
I thought DV only removed the pageable entries in the map [/quote]

A detail of implementation, but yes. It “pages out all the memory”
logically, by marking the page as being non-resident.

Peter
OSR

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer