When MmMapLockedPagesSpecifyCache fails...

NTFSD
1

I’m using MmMapLockedPagesSpecifyCache to temporarily map an I/O buffer
from my filter driver out to a helper app. When the app is finished, I
unmap the pages (in my read dispatch, I also unlock any mappings I had to
do and free the MDL - only for the NOT IRP_PAGING_IO case).

I know I’m not failing to do any of the unmappings. My question is…
What ultimately causes the MmMapLockedPagesSpecifyCache to start failing?
Once it does, it no longer ever succeeds. I have it wrapped in a
try-except, and do what I can to fail gracefully, but the system seems
never to recover.

This occurs after about 30 minutes of hammering on the system and my
filter with several applications running loops of data whose files I am
intercepting and manipulating (mostly multi-media files run in loops in
their respective applications).

2

If Access Mode is UserMode, this API will raise an exception. Get the
status code,
and youll have the reason of failure.

----- Original Message -----
From: “Bill”
To: “File Systems Developers”
Sent: Wednesday, February 19, 2003 11:14 PM
Subject: [ntfsd] When MmMapLockedPagesSpecifyCache fails…

> I’m using MmMapLockedPagesSpecifyCache to temporarily map an I/O buffer
> from my filter driver out to a helper app. When the app is finished, I
> unmap the pages (in my read dispatch, I also unlock any mappings I had to
> do and free the MDL - only for the NOT IRP_PAGING_IO case).
>
> I know I’m not failing to do any of the unmappings. My question is…
> What ultimately causes the MmMapLockedPagesSpecifyCache to start failing?
> Once it does, it no longer ever succeeds. I have it wrapped in a
> try-except, and do what I can to fail gracefully, but the system seems
> never to recover.
>
> This occurs after about 30 minutes of hammering on the system and my
> filter with several applications running loops of data whose files I am
> intercepting and manipulating (mostly multi-media files run in loops in
> their respective applications).
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

3

Ooops. Forgot to mention that data. The status code is C000009A,
STATUS_INSUFFICIENT_RESOURCES. As I stated, I’m pretty confident of the
logic path in this case. After the helper is done, I MmUnmapLockedPages.
The “resource” should then be availale again, no?

Would repeated allocates and frees from the non-paged pool precipitate
this problem? I’m in the process of creating my own look-aside lists for
data structures I need over and over to eliminate the need of asking the
system for them.

This happens with and without DriverVerifier enabled (takes about the same
length of time either way).

If Access Mode is UserMode, this API will raise an exception. Get the
status code,
and youll have the reason of failure.

4

This status code is mainly raised either when the non-paged pool is
exhausted and the MM
cannot allocate enough space for its internal bookkeeping, which I really
dont think is your case,
either when the Mm is unable to find enough continous space to map the pages
into the user address
space.

Dump the VADs of the target process in Windbg (dumping the VAD tree will
allow you to see the complete
memory layout of the user address space), and see if you really unmap the
pages when needed, and be sure
you dont exhaust the process address space.

----- Original Message -----
From: “Bill”
To: “File Systems Developers”
Sent: Thursday, February 20, 2003 4:38 PM
Subject: [ntfsd] Re: When MmMapLockedPagesSpecifyCache fails…

> Ooops. Forgot to mention that data. The status code is C000009A,
> STATUS_INSUFFICIENT_RESOURCES. As I stated, I’m pretty confident of the
> logic path in this case. After the helper is done, I MmUnmapLockedPages.
> The “resource” should then be availale again, no?
>
> Would repeated allocates and frees from the non-paged pool precipitate
> this problem? I’m in the process of creating my own look-aside lists for
> data structures I need over and over to eliminate the need of asking the
> system for them.
>
> This happens with and without DriverVerifier enabled (takes about the same
> length of time either way).
>
> > If Access Mode is UserMode, this API will raise an exception. Get the
> > status code,
> > and youll have the reason of failure.
> >
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

5

I didn’t get the dump you asked for (I’m using softICE, but have it
disabled as it seemed to be interfering with the memory.dmp creation - I
get corrupted .DMP in this error case if ntice is active), but have a
different observation to offer.

When the system got into this state last, I was able to terminate the
applications that were resulting in the file I/O and bring the system to a
quiesced state. Once I had done so, I was able to restart things and they
were again working normally with the MmMapLockedPagesSpecifyCache no
longer tossing the exception. That fact seems to support my assertion
that I am not leaking mappings, no?

I had made a change to do most of my run-time allocations in DriverEntry
and keep them in a private pool. I say most, because there are still many
others. The end result was that it took much longer to get into the fail
state.

Is is possible that this test, as strenuous as it is, keeps the system
from re-organizing memory (i.e. the NonPagedPool) as required and that
this is the cause of the Map failure?

This status code is mainly raised either when the non-paged pool is
exhausted and the MM
cannot allocate enough space for its internal bookkeeping, which I really
dont think is your case,
either when the Mm is unable to find enough continous space to map the pages
into the user address
space.

Dump the VADs of the target process in Windbg (dumping the VAD tree will
allow you to see the complete
memory layout of the user address space), and see if you really unmap the
pages when needed, and be sure
you dont exhaust the process address space.

6

A simple Q: What does Driver Verifier say about memory consumption
of your
driver, and what does the Task Manager say about how much memory is used
(of how
much available on the system - RAM, and page file)
Also, during testing, never use lookasides. They will prevent you
from seeing
certain errors for sure.

Bill wrote:

I didn’t get the dump you asked for (I’m using softICE, but have it
disabled as it seemed to be interfering with the memory.dmp creation - I
get corrupted .DMP in this error case if ntice is active), but have a
different observation to offer.

When the system got into this state last, I was able to terminate the
applications that were resulting in the file I/O and bring the system to a
quiesced state. Once I had done so, I was able to restart things and they
were again working normally with the MmMapLockedPagesSpecifyCache no
longer tossing the exception. That fact seems to support my assertion
that I am not leaking mappings, no?

I had made a change to do most of my run-time allocations in DriverEntry
and keep them in a private pool. I say most, because there are still many
others. The end result was that it took much longer to get into the fail
state.

Is is possible that this test, as strenuous as it is, keeps the system
from re-organizing memory (i.e. the NonPagedPool) as required and that
this is the cause of the Map failure?

> This status code is mainly raised either when the non-paged pool is
> exhausted and the MM
> cannot allocate enough space for its internal bookkeeping, which I really
> dont think is your case,
> either when the Mm is unable to find enough continous space to map the pages
> into the user address
> space.
>
> Dump the VADs of the target process in Windbg (dumping the VAD tree will
> allow you to see the complete
> memory layout of the user address space), and see if you really unmap the
> pages when needed, and be sure
> you dont exhaust the process address space.

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa File Monitor - File monitoring library for Win32 developers.

7

Good questions…

DriverVerifier shows my driver’s nonpaged allocations right about where I
would expect them, and I do not see them increase over time. I really
don’t think I’m leaking.

Never looked at TaskManager while this was going on. I’ll try to do so
next test run.

I mis-spoke when I said “lookaside”. I know that is a particular name for
a kernel functionality. What I meant was that I create my own small queue
of needed structure allocations upon DriverEntry and keep them reserved
for my use. Rather than ExAllocatePool and ExFreePool, I just get a
needed structure from my private queue and return it there when finished.
If the queue runs dry, I then do ExAllocate… (and make a log entry
indicating that I had to go back to the well). I think this is better for
two reason:

  1. If I see log entries indicating I repeatedly had to allocate more
    structures, I know something is leaking, right?

  2. I eliminate the suspected churn on the system caused by repeated
    ExAllocate/ExFree cycles.

As I said previously, I’m working now to add additional such private
allocation queues to eliminate all the repetitive alloc/free
possibilities. I think the total amount of memory my driver requires from
the nonpaged pool is not too aggressive, but I still am concerned about
the effects of allocating and freeing it on (nearly) every I/O request.

A simple Q: What does Driver Verifier say about memory consumption
of your
driver, and what does the Task Manager say about how much memory is used
(of how
much available on the system - RAM, and page file)
Also, during testing, never use lookasides. They will prevent you
from seeing
certain errors for sure.

8

> Never looked at TaskManager while this was going on. I’ll try to do so next

test run.

One down, one to go:-) Curious - if you allocate MDLs, do you free
them?:slight_smile: (You’d be surprised how often this happens)

I mis-spoke when I said “lookaside”. I know that is a particular name for a
kernel functionality. What I meant was that I create my own small queue of
needed structure allocations upon DriverEntry and keep them reserved for my
use.

That’s exactly what I thought you’d do - do constant allocate/free
during testing! That will allow you to trace any memory corruption error.
Doing your own queue will prevent error on free catches, such as double frees

  • and if you use a lookaside type queue, you’d have the same buffer twice!
    Leading to certain corruption of memory.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.

Alfa File Monitor - File monitoring library for Win32 developers.

9

> > Never looked at TaskManager while this was going on. I’ll try to do so next

> test run.

One down, one to go:-) Curious - if you allocate MDLs, do you free
them?:slight_smile: (You’d be surprised how often this happens)

I do allocate an MDL during read operations, in the NOT IRP_PAGING_IO
case. In that case, I alloc the MDL and do a MmProbeAndLock. I note in
my context structure that there is a free that needs to be done. All
indications are that it is getting done.

OK. DriverVerifier does not show an increase in NP Pool by my filter
driver, but I took a look at TaskManager and I do see a monotonic increase
in the NP Pool assigned to my test program. It works with my filter -
they exchange info on file open/close/read/rename etc. I used
METHOD_BUFFERED for all those ins and outs. No mapping gets down in my
ioctl routines. Since it is METHOD_BUFFERED, the I/O manager is
allocating buffer space in the system pool. Is it possible that these
allocations are not being freed? Under what circumstances?

> I mis-spoke when I said “lookaside”. I know that is a particular name for a
> kernel functionality. What I meant was that I create my own small queue of
> needed structure allocations upon DriverEntry and keep them reserved for my
> use.

That’s exactly what I thought you’d do - do constant allocate/free
during testing! That will allow you to trace any memory corruption error.
Doing your own queue will prevent error on free catches, such as double frees

  • and if you use a lookaside type queue, you’d have the same buffer twice!
    Leading to certain corruption of memory.

I had that mindset at the beginning. I ‘aggresively’ alloc’ed and freed
everything I needed as needed. The long stress runs that caused me
failures led me to implement my own private queuing. I do not see in my
logs any evidence of having to grow that pool - backing my claim that I’m
not leaking (at least not on those objects).

10

Silly question, you are UNlocking those locked pages?

-Jeff

-----Original Message-----
From: Bill [mailto:xxxxx@endwell.net]
Sent: Friday, February 21, 2003 1:20 PM
To: File Systems Developers
Subject: [ntfsd] Re: When MmMapLockedPagesSpecifyCache fails…

> Never looked at TaskManager while this was going on. I’ll try to do so
next
> test run.

One down, one to go:-) Curious - if you allocate MDLs, do you free
them?:slight_smile: (You’d be surprised how often this happens)

I do allocate an MDL during read operations, in the NOT IRP_PAGING_IO
case. In that case, I alloc the MDL and do a MmProbeAndLock. I note in
my context structure that there is a free that needs to be done. All
indications are that it is getting done.

OK. DriverVerifier does not show an increase in NP Pool by my filter
driver, but I took a look at TaskManager and I do see a monotonic increase
in the NP Pool assigned to my test program. It works with my filter -
they exchange info on file open/close/read/rename etc. I used
METHOD_BUFFERED for all those ins and outs. No mapping gets down in my
ioctl routines. Since it is METHOD_BUFFERED, the I/O manager is
allocating buffer space in the system pool. Is it possible that these
allocations are not being freed? Under what circumstances?

> I mis-spoke when I said “lookaside”. I know that is a particular name
for a
> kernel functionality. What I meant was that I create my own small queue
of
> needed structure allocations upon DriverEntry and keep them reserved
for my
> use.

That’s exactly what I thought you’d do - do constant allocate/free
during testing! That will allow you to trace any memory corruption error.
Doing your own queue will prevent error on free catches, such as double
frees

  • and if you use a lookaside type queue, you’d have the same buffer twice!
    Leading to certain corruption of memory.

I had that mindset at the beginning. I ‘aggresively’ alloc’ed and freed
everything I needed as needed. The long stress runs that caused me
failures led me to implement my own private queuing. I do not see in my
logs any evidence of having to grow that pool - backing my claim that I’m
not leaking (at least not on those objects).

You are currently subscribed to ntfsd as: xxxxx@concord.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
the latest virus scan software available for the presence of computer
viruses.
**********************************************************************

11

No question is silly. There are only silly answers… :wink:

Yes, I am unlocking those pages. My logic flow there looks pretty good.
I also think I would trap when I unloaded my driver or terminated the
helper app if I had not. (I know you’re not supposed to unload a filter,
but I have allowed this in the checked build only).

If I watch in TaskManager and DriverVerifier simultaneously, the DV shows
the NP pool allocations by my driver holds steady at about 16K bytes. As
stated, TM shows that the test program is being charged with an ever
increasing amount of NP pool quota. When TM shows the total NP pool use
at about 3MB, the error begins.

I’m sure this is something I’m doing (or not) in my driver.

Silly question, you are UNlocking those locked pages?

-Jeff

12

Will it do this even if BugCheckOnFailure is false?

13

> Will it do this even if BugCheckOnFailure is false?

Yes it does (did). I had put this one on the back burner (many other fish
to fry) and revisited it this week-end. Amazing what a fresh set of eyes
will do… I found a bug in my filter. I was not actually unmapping the
correct pages… a typo “left-over” from converting my Ring 0 code to
using the services of a Ring 3 helper. I actually had a double mapping…
one for Ring 3 and one for Ring 0 (due to MmGetSystemAddressForMdlSafe().
I was releasing the map for the latter but not the former, hence the
memory “leak”.

Thanks to all who offered comments and suggestions.

14

Yes, it will return NULL.

----- Original Message -----
From: “Lyndon J. Clarke”
To: “File Systems Developers”
Sent: Tuesday, March 04, 2003 6:27 PM
Subject: [ntfsd] Re: When MmMapLockedPagesSpecifyCache fails…

> Will it do this even if BugCheckOnFailure is false?
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

15

Maxim. I meant will it bug check. I saw this once. W2KSP2.

16

IIRC either the function or the Mdl->Flags field had the “no bugcheck
on failure” parameter.

Max

----- Original Message -----
From: “Lyndon J. Clarke”
To: “File Systems Developers”
Sent: Wednesday, March 05, 2003 9:28 PM
Subject: [ntfsd] Re: When MmMapLockedPagesSpecifyCache fails…

> Maxim. I meant will it bug check. I saw this once. W2KSP2.
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>