Hi,
I have an application where I map pagefile to memory using MapViewOfFile(). I send this pointer to an direct_io ioctl to get the data from the driver.
This call is being done very frequently, say 10 to 15 times a minute. every time, i do CreateFileMapping(), MapViewOfFile() and unmapviewoffile() and close the filehandle.
after some 10 hours, my application starts failing MapViewOfFile() command with errorcode 8( insufficient memory). I took a process dump. There is enough memory available.
Can anyone please let me know why this could be happening.
Thanks,
Vinay
xxxxx@gmail.com wrote:
Hi,
I have an application where I map pagefile to memory using MapViewOfFile(). I send this pointer to an direct_io ioctl to get the data from the driver.
This call is being done very frequently, say 10 to 15 times a minute. every time, i do CreateFileMapping(), MapViewOfFile() and unmapviewoffile() and close the filehandle.
after some 10 hours, my application starts failing MapViewOfFile() command with errorcode 8( insufficient memory). I took a process dump. There is enough memory available.
Can anyone please let me know why this could be happening.
It certainly could be memory fragmentation. Let’s say you’re trying to
map a 500MB chunk. To do that, there has to be 500MB of contiguous free
space available. It’s quite possible to have 500MB of memory free, but
with no single piece larger than 100MB. In that case, the mapping will
fail.
Why don’t you just create your mapping once and keep it around forever?
For that matter, why are you using a file mapping to get a memory buffer
for an ioctl?
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I dumped the process and looked at it. there is around 450MB contiguous virtual addres sspace available. I am trying to map 50MB to a pagefile.
The reason why I cannot preallocate the shared memory is because I will have more than 1 object of that class and I am creating the shared memory in that class object. I have seen it to be around 2/3 objects of that class.
Is there any better approach to pre allocate the shared memory when I have multiple objects and each needs to have a shared memory of its own?
Vinay
xxxxx@gmail.com wrote:
I dumped the process and looked at it. there is around 450MB contiguous virtual addres sspace available. I am trying to map 50MB to a pagefile.
The reason why I cannot preallocate the shared memory is because I will have more than 1 object of that class and I am creating the shared memory in that class object. I have seen it to be around 2/3 objects of that class.
Is there any better approach to pre allocate the shared memory when I have multiple objects and each needs to have a shared memory of its own?
You keep saying “shared”, but shared between who? If the sharing is
only between you and the driver, that doesn’t require a mapping. You
can just use malloc and free.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I am using the shared memory to get data from my driver.
This code path is being called around 10-15 times a minute. So, if I do a malloc, wont I hit the fragmentation issue ?
Vinay
Allocating and deallocating (or in your case mapping and unmapping) huge chunks of memory 10-15 times a minute is a terrible waste of performance IMO, and will lead to memory fragmentation no matter what you do.
The correct solution IMO is to create a pool of memory (perhaps a section, perhaps a simple VirtualAlloc) large enough to hold some fixed number of buffers. Say 5, so if each request is 50MB you would need to allocate a pool of 250MB. This is still pretty big, but the general algorithm stays the same no matter how many requests you make room for in your pool, so it doesn’t matter.
Then provide some synchronization around this pool (maybe a semaphore, for example) that controls access to the block of memory so that anybody trying to obtain a chunk of memory will block in the call until a chunk is available.
From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com [xxxxx@gmail.com]
Sent: Wednesday, July 23, 2008 11:42 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] MapViewOfFile failing with error code 8
I am using the shared memory to get data from my driver.
This code path is being called around 10-15 times a minute. So, if I do a malloc, wont I hit the fragmentation issue ?
Vinay
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
xxxxx@gmail.com wrote:
I am using the shared memory to get data from my driver.
Why? What do you believe you gain by using a memory mapping? If you
aren’t sharing the memory with another process, there is no point.
This code path is being called around 10-15 times a minute. So, if I do a malloc, wont I hit the fragmentation issue ?
The same issue exists.
Surely you must be able to comprehend the tremendous inefficiency
involved in allocating and freeing multiple megabytes every 4 seconds.
Isn’t that patently obvious? If you never need more than 4 or 5 at a
time, then just do your own pool of buffers. Create a simple linked
list. That way, you only allocate each buffer once, and you only
allocate as many as you need simultaneously. For example, ignoring
interlock issues:
list< unsigned char * > myList;
unsigned char * AllocateBuffer()
{
if( myList.empty () )
myList.push_back( new unsigned char[5000000] );
unsigned char * xxx = myList.front();
myList.pop_front();
return xxx;
}
void FreeBuffer( unsigned char * bfr )
{
myList.push_back( bfr );
}
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.