ExAllocatePoolWithTag won't allocate more than 0x70ffff size of NonPagedPool of Data.

NTDEV
1

Hello there,

I am trying to allocate large data buffer using ExAllocatePoolWithTag on my kernel driver program and it did allocate until 0x70fffff on my machine. But the problem is beyond this size value, it won’t allocate anymore. Here’s what I did on my code:

PUCHAR DataBuf;

DataBuf = ExAllocatePoolWithTag(
NonPagedPool,
DataBufSize,
‘bBUF’);

if ( DataBuf == NULL )
{
return STATUS_INSUFFICIENT_RESOURCES;
}

DataBufSize can be set and when I try to set it more than 0x70fffff sizes, it will only return NULL on my DataBuf. My machine has a 1GB RAM and my OS is Windows XP SP2 freshly installed when no third party programs installed yet.

Any idea on how this code can allocate unlimited size based on my RAM resource available? I want my DataBuf to be allocated on a memory resident which should not be paged by the OS.

2

Your ability to allocate pool is limited by more than the number of GB of RAM you have.

Kernel pool can become fragmented over time and this will limit the size of any particular allocation you can get. Also kernel pool has to share the kernel virtual address space with many, many other things and so its size is limited by that as well. Finally other kernel components allocate from the Pool so you won’t be able to get your “unlimited size” allocation with kernel pool.

Your best bet for a large buffer (IMO) is to use MmAllocatePagesForMdl() to allocate a large number of non-paged physical pages for your buffer. You can then use IoBuildPartialMdl() and MmGetSystemAddressSpecifyCache() to map reasonably sized windows of that buffer into the kernel virtual address space. You won’t have easy contiguous access to the entire buffer, but it should be much closer to “unlimited size” than you’re going to get from the system pool.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, March 17, 2008 7:05 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] ExAllocatePoolWithTag won’t allocate more than 0x70ffff size of NonPagedPool of Data.

Hello there,

I am trying to allocate large data buffer using ExAllocatePoolWithTag on my kernel driver program and it did allocate until 0x70fffff on my machine. But the problem is beyond this size value, it won’t allocate anymore. Here’s what I did on my code:

PUCHAR DataBuf;

DataBuf = ExAllocatePoolWithTag(
NonPagedPool,
DataBufSize,
‘bBUF’);

if ( DataBuf == NULL )
{
return STATUS_INSUFFICIENT_RESOURCES;
}

DataBufSize can be set and when I try to set it more than 0x70fffff sizes, it will only return NULL on my DataBuf. My machine has a 1GB RAM and my OS is Windows XP SP2 freshly installed when no third party programs installed yet.

Any idea on how this code can allocate unlimited size based on my RAM resource available? I want my DataBuf to be allocated on a memory resident which should not be paged by the OS.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

Ok, thanks Peter.

I’ll try this one.

4

> Any idea on how this code can allocate unlimited size based on my RAM

resource available?

You have hit the kernel virtual address space limit for the pool.

To bypass it, you can use MmAllocatePagesForMdl to allocate physical memory
without giving it virtual addresses, and then use
IoBuildPartialMdl+MmGetSystemAddressForMdlSafe to map the part of this
physical memory chunk to the kernel virtual addresses.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

5

May I know what’s the pros and cons of using MmAllocateContiguousMemory and MmAllocatePagesForMdl? I tried MmAllocateContiguousMemory on the above code and now it works. But I just want to know which of the two is better to use in terms of performance and long term use. MmAllocateContiguousMemory looks no problem to me since I called the allocation on my AddDevice Routine.

6

MmAllocateContiguousMemory is pretty drastic if you don’t need physically contiguous memory. Since you were using pool I’m guessing that you don’t.

If you can work within the confines of MmAllocatePagedForMdl I think that will be better. Your driver will have to do more work and it will add the cost of mapping and unmapping views of your large buffer but you won’t be sucking up a huge piece of the 1-2GB kernel virtual address space.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Tuesday, March 18, 2008 7:02 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] ExAllocatePoolWithTag won’t allocate more than 0x70ffff size of NonPagedPool of Data.

May I know what’s the pros and cons of using MmAllocateContiguousMemory and MmAllocatePagesForMdl? I tried MmAllocateContiguousMemory on the above code and now it works. But I just want to know which of the two is better to use in terms of performance and long term use. MmAllocateContiguousMemory looks no problem to me since I called the allocation on my AddDevice Routine.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

> May I know what’s the pros and cons of using MmAllocateContiguousMemory

This call is intended only for DMA adapter object implementations to
implement ->AllocateCommonBuffer method.

This is not a generic allocation routine. ExAllocatePoolWithTag is.

and MmAllocatePagesForMdl?

Yes, this one is intended for large allocations which cannot be allocated by
ExAllocatePoolWithTag.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com