Hi everyone,
I’m relatively new to writing kernel mode drivers. I’ve done something some
time ago using DriverWorks (as I understand it was not a good choice) and
now I started to play with KMDF.
In my new driver I’ve faced the problem how to allocate some relative large
amount of memory - I need to allocate about 300 MB, it does not have to be
contiguous. The driver is for a physical device and will always be used on
computers with 2GB or more of physical memory so there should not be problem
with that amount.
I would like to have the memory not pageable. I don’t want to allocate the
memory in user mode. As I understand this amount is too big to allocate from
any of the pools under 32bit Windows.
I’ve searched this forum and other places and did not find any useful clues
on how to do this correctly so I’ve ended with experimenting:
to allocate the memory I do it this in EvtDeviceAdd:
- MmAllocatePagesForMdl() for allocating the required amount of memory. Then
I check if all memory was allocated using MmGetMdlByteCount().
- then MmGetSystemAddressForMdlSafe() to get pointer to that memory.
to free the memory I’m calling this in EvtDeviceContextCleanup:
- MmFreePagesFromMdl() for freeing the memory
- ExFreePool() for freeing the mdl.
This works okay, although I’m not sure whether it is correct way to do this
because after some loading and unloading of the driver it fails to load
again with error on MmGetSystemAddresForMdlSafe() - it returns me a NULL
pointer. After reboot the driver always loads correctly.
Can anyone tell me where could be problem? Am I releasing the memory not
correcly or why does the MmGetSystemAddressForMdlSafe() fail? Is there any
tool that will show me physical memory usage or some physical memory tracing
tool?
Martin
> MmFreePagesFromMdl() for freeing the memory
ExFreePool() for freeing the mdl.
ExFreePool() does NOT free MDL - what it frees is memory that you have allocated by ExAllocatePool().
MmGetSystemAddressForMdlSafe() is just a macro that, unless MDL has been already mapped to the kernel space, returns MmMapLockedPagesSpecifyCache() - the latter function maps physical pages that MDL describes into the virtual address space. What actually happens here is that you free *physical* pages that MDL describes, but leave the virtual range mapped. Once the amount of memory that you have mapped is HUGE, you eventually run out of available addressable space, and are unable to repeat the operation because of lack of virtual address space. Therefore, what you need here is a call to
MmUnmapLockedPages(), rather than to ExFreePool() …
Anton Bassov
Thanks for the answer, I’ll try that.
I think that I still have to call the ExFreePool() for freeing the MDL
structure, am I right?
Martin
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Sunday, July 27, 2008 3:29 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Allocating large memory
MmFreePagesFromMdl() for freeing the memory
ExFreePool() for freeing the mdl.
ExFreePool() does NOT free MDL - what it frees is memory that you have
allocated by ExAllocatePool().
MmGetSystemAddressForMdlSafe() is just a macro that, unless MDL has been
already mapped to the kernel space, returns MmMapLockedPagesSpecifyCache() -
the latter function maps physical pages that MDL describes into the virtual
address space. What actually happens here is that you free *physical* pages
that MDL describes, but leave the virtual range mapped. Once the amount of
memory that you have mapped is HUGE, you eventually run out of available
addressable space, and are unable to repeat the operation because of lack of
virtual address space. Therefore, what you need here is a call to
MmUnmapLockedPages(), rather than to ExFreePool() …
Anton Bassov
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
> I think that I still have to call the ExFreePool() for freeing the MDL structure, am I right?
As I told you already, ExFreePool() has nothing to do with MDLs whatsoever - driver-allocated MDLs are freed by IoFreeMdl(), while ExFreePool() frees memory allocated by calls to ExAllocatePool() …
Anton Bassov
Well, why is it written in MSDN then?
“The caller must use MmFreePagesFromMdl to release the memory pages that are
described by an MDL that was created by MmAllocatePagesForMdl. After calling
MmFreePagesFromMdl, the caller must also call ExFreePool to release the
memory that is allocated for the MDL structure itself.”
Martin
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Monday, July 28, 2008 1:29 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Allocating large memory
I think that I still have to call the ExFreePool() for freeing the MDL
structure, am I right?
As I told you already, ExFreePool() has nothing to do with MDLs whatsoever -
driver-allocated MDLs are freed by IoFreeMdl(), while ExFreePool() frees
memory allocated by calls to ExAllocatePool() …
Anton Bassov
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
> "The caller must use MmFreePagesFromMdl to release the memory pages that
are
described by an MDL that was created by MmAllocatePagesForMdl. After calling
MmFreePagesFromMdl, the caller must also call ExFreePool to release the
memory that is allocated for the MDL structure itself."
Looks like a documentation bug.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Yes, You should call MmFreePagesFromMdl and ExFreePool to release the memory by MmAllocatePagesForMdl.
It’s not. MmAllocatePagesForMdl() doesn’t use IoAllocateMdl() for MDL
allocation, but MmCreateMdl().
“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>> “The caller must use MmFreePagesFromMdl to release the memory pages that
>>are
>> described by an MDL that was created by MmAllocatePagesForMdl. After
>> calling
>> MmFreePagesFromMdl, the caller must also call ExFreePool to release the
>> memory that is allocated for the MDL structure itself.”
>
> Looks like a documentation bug.
>
> –
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>> “The caller must use MmFreePagesFromMdl to release the memory pages that
>>are
>> described by an MDL that was created by MmAllocatePagesForMdl. After
>> calling
>> MmFreePagesFromMdl, the caller must also call ExFreePool to release the
>> memory that is allocated for the MDL structure itself.”
>
> Looks like a documentation bug.
>
> –
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
The system just allocates a structure from the pool for the MDL and because
it does not know when you are done with it you need to free that memory
yourself. You call ExFreePool to free the MDL structure, not the memory that
it describes. For a VA Mdl this is normally this is done by IoFreeMdl.
//Daniel
> The system just allocates a structure from the pool for the MDL and because it does
not know when you are done with it you need to free that memory yourself.
This is totally illogical - it looks like just yet another MSDN bug. Every allocation function should have its reciprocal, and the latter should undo what the former does. If MDL gets allocated with a call to IoAllocateMdl(), then you should deallocate it with IoFreeMdl(). If both MDL and pages for it get allocated with a call to MmAllocatePagesForMd(), then MmFreePagesFromMdl() should free them both as well.
You call ExFreePool to free the MDL structure, not the memory that it describes.
Well, even if a call to ExAllocatePool() was made behind the scenes, it is not you who made it, and, hence, you should not be responsible for making a corresponding call to ExFreePool ()…
Anton Bassov
Not freeing the MDL in MmFreePagesFromMdl allows the MDL to be reused by the
system or a driver.
Then at least the names of IoAllocateMdl and IoFreeMdl leave no doubt it
that that they allocate and free an MDL structure. Because IoAllocateMdl
does not allocate any memory, there cannot be any confusion about the fact
that IoFreeMdl frees the MDL structure (and not any memory involved).
It is totally normal for any type of interface which allocates a structure
for you that it loads you with the responsability for freeing it especially
if that’s explicitly mentioned.
//Daniel
wrote in message news:xxxxx@ntdev…
>> The system just allocates a structure from the pool for the MDL and
>> because it does
>> not know when you are done with it you need to free that memory yourself.
>
> This is totally illogical - it looks like just yet another MSDN bug. Every
> allocation function should have its reciprocal, and the latter should undo
> what the former does. If MDL gets allocated with a call to
> IoAllocateMdl(), then you should deallocate it with IoFreeMdl(). If both
> MDL and pages for it get allocated with a call to MmAllocatePagesForMd(),
> then MmFreePagesFromMdl() should free them both as well.
>
>> You call ExFreePool to free the MDL structure, not the memory that it
>> describes.
>
> Well, even if a call to ExAllocatePool() was made behind the scenes, it is
> not you who made it, and, hence, you should not be responsible for making
> a corresponding call to ExFreePool ()…
>
> Anton Bassov
>
> Not freeing the MDL in MmFreePagesFromMdl allows the MDL to be reused by the system or a driver.
Reused for what??? After pages have been freed it describes nothing, and, unless you decide to provide your own extension to Mmxxx functions, you have no chance to associate it either with a set of newly-allocated pages or with buffer’s virtual address, because functions that do the above allocate fresh MDL anyway…
It is totally normal for any type of interface which allocates a structure for you that it loads
you with the responsability for freeing it especially if that’s explicitly mentioned.
Normally it gets done when a function that does not have its reciprocal allocates memory ( for example, when ZwQuerySystemInformation() returns requested data in a buffer that it allocates). In such case your call to a function that frees memory acts as a reciprocal to the original call. However, the situation that we discuss is somehow different…
responsAbility
Are we on French-speaking forum or what???
Anton Bassov
PLOINK:
The beam sags to the breaking point of becoming a non-technical
discussion. In an effort not to upset any MVPs I refrain from further
posting to this thread. The purpose of this message is to indicate that the
absence of any replies from here is not intended to be perceived by anyone
as an indication that I agree that you are right, funny or make any sense at
all.
//Daniel
wrote in message news:xxxxx@ntdev…
>> Not freeing the MDL in MmFreePagesFromMdl allows the MDL to be reused by
>> the system or a driver.
>
> Reused for what??? After pages have been freed it describes nothing, and,
> unless you decide to provide your own extension to Mmxxx functions, you
> have no chance to associate it either with a set of newly-allocated pages
> or with buffer’s virtual address, because functions that do the above
> allocate fresh MDL anyway…
>
>> It is totally normal for any type of interface which allocates a
>> structure for you that it loads
>> you with the responsability for freeing it especially if that’s
>> explicitly mentioned.
>
> Normally it gets done when a function that does not have its reciprocal
> allocates memory ( for example, when ZwQuerySystemInformation() returns
> requested data in a buffer that it allocates). In such case your call to a
> function that frees memory acts as a reciprocal to the original call.
> However, the situation that we discuss is somehow different…
>
>> responsAbility
>
> Are we on French-speaking forum or what???
>
> Anton Bassov
>
> The beam sags to the breaking point of becoming a non-technical discussion.
I am just showing off - I am showing my (virtually non-existent, despite having lived in French-speaking country for ages -shame on me), “knowledge” of French. Therefore, we can proceed with the technical part…
Anton Bassov