Hi guys,
I remember I done such things at my old VxD day but I’m not sure what
technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel? I’m
talking about big memory buffer.
What happened if the process that allocate those pages crash? Can I still
access those page.
Thanks,
Ronen Ochman
Have a look at MmProbeAndLockPages in the DDK docs. That’s probably what you
want…
I’m pretty sure the pages stay locked, whatever the app does, until the
driver unlocks them, which the driver SHOULD do when it’s finished using the
locked pages.
–
Mats
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Ronen Ochman
Sent: Monday, July 12, 2004 1:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not sure what
technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel? I’m
talking about big memory buffer.
What happened if the process that allocate those pages crash? Can I still
access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Memory allocation. MmProbeAndLockPages. On the process crash, MJ_CLEANUP path will be invoked, and this is the place to do unlock.
Or use METHOD_xx_DIRECT IOCTLs - in this case, the kernel will do MmProbeAndLockPages itself, and you will have the MDL at Irp->MdlAddress. In this case, the buffer is unlocked at IRP completion, and the crashed process will remain in some semi-dead state till all IRPs issued by it will be completed.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: Ronen Ochman
To: Windows System Software Devs Interest List
Sent: Monday, July 12, 2004 4:50 PM
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not sure what technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel? I’m talking about big memory buffer.
What happened if the process that allocate those pages crash? Can I still access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Memory allocation. No, the OS BSODs in this case.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
----- Original Message -----
From: xxxxx@3Dlabs.com
To: Windows System Software Devs Interest List
Sent: Monday, July 12, 2004 3:51 PM
Subject: RE: [ntdev] Memory allocation.
Have a look at MmProbeAndLockPages in the DDK docs. That's probably what you want...
I'm pretty sure the pages stay locked, whatever the app does, until the driver unlocks them, which the driver SHOULD do when it's finished using the locked pages.
--
Mats
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]On Behalf Of Ronen Ochman
Sent: Monday, July 12, 2004 1:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I'm not sure what technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel? I'm talking about big memory buffer.
What happened if the process that allocate those pages crash? Can I still access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Is METHOD_xx_DIRECT IOCTLs limit the buffer length (I’m talking about
allocating MBs)? What about performance?
When I work on 256mb machine I found I can’t allocate more then ~50mb kernel
memory.
I’m looking for way to increase this number, any suggestion? (tricks?)
Thanks,
Ronen
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Monday, July 12, 2004 2:09 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Memory allocation.
MmProbeAndLockPages. On the process crash, MJ_CLEANUP path will be
invoked, and this is the place to do unlock.
Or use METHOD_xx_DIRECT IOCTLs - in this case, the kernel will do
MmProbeAndLockPages itself, and you will have the MDL at Irp->MdlAddress. In
this case, the buffer is unlocked at IRP completion, and the crashed process
will remain in some semi-dead state till all IRPs issued by it will be
completed.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: Ronen mailto:xxxxx Ochman
To: Windows System Software Devs Interest mailto:xxxxx List
Sent: Monday, July 12, 2004 4:50 PM
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not sure what
technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel? I’m
talking about big memory buffer.
What happened if the process that allocate those pages crash? Can I still
access those page.
Thanks,
Ronen Ochman
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@checkpoint.com
To unsubscribe send a blank email to xxxxx@lists.osr.com</mailto:xxxxx></mailto:xxxxx>
Memory allocation.>Is METHOD_xx_DIRECT IOCTLs limit the buffer length (I’m
talking about allocating MBs)?
32MB or such. The thing is that the MDL tail is limited as 64KB, which is 8K
physical addresses (assuming 64bit Windows). 8K pages is 32MB.
In usual 32bit Windows, the limit is possibly 64MB, but I’m not sure of it.
What about performance?
Best performance for large (>> PAGE_SIZE) buffers.
When I work on 256mb machine I found I can’t allocate more then ~50mb kernel
memory.
I’m looking for way to increase this number, any suggestion? (tricks?)
I don’t think it is possible.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
There are actually a couple of options for you to lock pages in user
mode. The first (and easiest if you can be sure of the appropriate
permissions) is to use AWE extensions.
First, call AllocateUserPhysicalPage() to allocate the number of
physical pages you need. Then use VirtualAllocEx() with both the
MEM_PHYSICAL and MEM_RESERVE options to allocate the virtual addresses
that will point to these pages. Looking up these two functions in MSDN
will give you the necessary other documentation.
Doing this from kernel mode is much trickier, because in general you
must protect against rogue applications. Here are just a few of the
things you would have to do:
* Set the security descriptor on the device object such that only the
authorized user can send R/W ioctls.
* Handle the CREATE irp; you must track the file object that was used
here in a hash or list or similar.
* When the IOCTL to lock pages arrives, you must verify the file object
for the IOCTL is listed in your list of file objects from the CREATE
irps.
* Impose reasonable limits, such as only one client of N pages, or N
pages across the whole system.
* If the IOCTL passes the above, lock the pages on behalf of the user
application and track which pages were locked as part of the file object
information you cached from the CREATE irp. You need this to be a count
of the times a page is locked, not just a true/false type thing.
* Provide an unlock IOCTL that does similar handling to the above,
including the file object check.
* Finally, and most importantly, when a CLOSE irp arrives, automatically
unlock any pages of memory associated with the file object.
* Appropriate synchronization throughout the above.
Note that doing this yourself is just like using AWE, only it’s much
more likely to expose a security hole. Instead, I’d suggest using AWE
or at least a COM service which does this for clients (and cleans up in
the final release call).
Hope that helps,
.
-----Original Message-----
From: xxxxx@3Dlabs.com [mailto:xxxxx@3Dlabs.com]
Sent: Monday, July 12, 2004 4:52 AM
Subject: RE: Memory allocation.
Have a look at MmProbeAndLockPages in the DDK docs. That’s probably what
you want…
I’m pretty sure the pages stay locked, whatever the app does, until the
driver unlocks them, which the driver SHOULD do when it’s finished using
the locked pages.
–
Mats
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Ronen Ochman
Sent: Monday, July 12, 2004 1:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not
sure what technology exists at WDM.
How can I lock pages from user mode in order to use them at
kernel? I’m talking about big memory buffer.
What happened if the process that allocate those pages crash?
Can I still access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@3dlabs.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Thanks Henry,
The question is if I’m using AWE (or any other way) what happened for those
pages when the user-mode process die (crash)? Is windows unlock those pages
automatically, so any access from kernel mode will case BOSD?
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Henry Gabryjelski
Sent: Tuesday, July 13, 2004 6:22 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Memory allocation.
There are actually a couple of options for you to lock pages in user mode.
The first (and easiest if you can be sure of the appropriate
permissions) is to use AWE extensions.
First, call AllocateUserPhysicalPage() to allocate the number of physical
pages you need. Then use VirtualAllocEx() with both the MEM_PHYSICAL and
MEM_RESERVE options to allocate the virtual addresses that will point to
these pages. Looking up these two functions in MSDN will give you the
necessary other documentation.
Doing this from kernel mode is much trickier, because in general you must
protect against rogue applications. Here are just a few of the things you
would have to do:
* Set the security descriptor on the device object such that only the
authorized user can send R/W ioctls.
* Handle the CREATE irp; you must track the file object that was used here
in a hash or list or similar.
* When the IOCTL to lock pages arrives, you must verify the file object for
the IOCTL is listed in your list of file objects from the CREATE irps.
* Impose reasonable limits, such as only one client of N pages, or N pages
across the whole system.
* If the IOCTL passes the above, lock the pages on behalf of the user
application and track which pages were locked as part of the file object
information you cached from the CREATE irp. You need this to be a count of
the times a page is locked, not just a true/false type thing.
* Provide an unlock IOCTL that does similar handling to the above, including
the file object check.
* Finally, and most importantly, when a CLOSE irp arrives, automatically
unlock any pages of memory associated with the file object.
* Appropriate synchronization throughout the above.
Note that doing this yourself is just like using AWE, only it’s much more
likely to expose a security hole. Instead, I’d suggest using AWE or at
least a COM service which does this for clients (and cleans up in the final
release call).
Hope that helps,
.
-----Original Message-----
From: xxxxx@3Dlabs.com [mailto:xxxxx@3Dlabs.com]
Sent: Monday, July 12, 2004 4:52 AM
Subject: RE: Memory allocation.
Have a look at MmProbeAndLockPages in the DDK docs. That’s probably what you
want…
I’m pretty sure the pages stay locked, whatever the app does, until the
driver unlocks them, which the driver SHOULD do when it’s finished using the
locked pages.
–
Mats
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Ronen Ochman
Sent: Monday, July 12, 2004 1:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not sure
what technology exists at WDM.
How can I lock pages from user mode in order to use them at kernel?
I’m talking about big memory buffer.
What happened if the process that allocate those pages crash?
Can I still access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@3dlabs.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@checkpoint.com To
unsubscribe send a blank email to xxxxx@lists.osr.com
First, I’ll have to admit that I didn’t know the answer to this
particular question before talking with someone internally.
AWE allocated pages will be automatically released when your process
address space is cleaned up.
The simplest scenario is when you are using an IRP with these pages, and
the pages are only used by the driver when it has an IRP with those
pages as the buffer. In this scenario, the driver would need no special
processing or handling of the buffers (still need to probe and lock, and
you probably need either METHOD_NEITHER or a METHOD_BUFFERED with an
embedded pointer as per IOCTL_SCSI_PASS_THROUGH_DIRECT to get
performance).
The more complex scenario has the application send an irp which pends
the entire time that the buffer is in use. This would only be done if
the driver must continuously and without application involvement fill
the buffer. Eventually the process should cancel the irp (indicating
that the buffer won’t be used any longer).
In both the above cases, the cancel routine would need to ensure that
the drive is no longer accessing the memory prior to completing the irp.
The answers are getting more complex because it’s not clear what problem
you are trying to solve. The above may be the totally wrong way to
solve your particular problem, or there may be commonly-used methods to
do this already avaiable and well-tested. Can you share the more exact
problem you are trying to solve?
Thanks,
.
-----Original Message-----
From: Ronen Ochman [mailto:xxxxx@checkpoint.com]
Sent: Wednesday, July 14, 2004 12:05 AM
Subject: RE: Memory allocation.
Thanks Henry,
The question is if I’m using AWE (or any other way) what happened for
those
pages when the user-mode process die (crash)? Is windows unlock those
pages
automatically, so any access from kernel mode will case BOSD?
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Henry
Gabryjelski
Sent: Tuesday, July 13, 2004 6:22 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Memory allocation.
There are actually a couple of options for you to lock pages in user
mode.
The first (and easiest if you can be sure of the appropriate
permissions) is to use AWE extensions.
First, call AllocateUserPhysicalPage() to allocate the number of
physical
pages you need. Then use VirtualAllocEx() with both the MEM_PHYSICAL
and
MEM_RESERVE options to allocate the virtual addresses that will point to
these pages. Looking up these two functions in MSDN will give you the
necessary other documentation.
Doing this from kernel mode is much trickier, because in general you
must
protect against rogue applications. Here are just a few of the things
you
would have to do:
* Set the security descriptor on the device object such that only the
authorized user can send R/W ioctls.
* Handle the CREATE irp; you must track the file object that was used
here
in a hash or list or similar.
* When the IOCTL to lock pages arrives, you must verify the file object
for
the IOCTL is listed in your list of file objects from the CREATE irps.
* Impose reasonable limits, such as only one client of N pages, or N
pages
across the whole system.
* If the IOCTL passes the above, lock the pages on behalf of the user
application and track which pages were locked as part of the file object
information you cached from the CREATE irp. You need this to be a count
of
the times a page is locked, not just a true/false type thing.
* Provide an unlock IOCTL that does similar handling to the above,
including
the file object check.
* Finally, and most importantly, when a CLOSE irp arrives, automatically
unlock any pages of memory associated with the file object.
* Appropriate synchronization throughout the above.
Note that doing this yourself is just like using AWE, only it’s much
more
likely to expose a security hole. Instead, I’d suggest using AWE or at
least a COM service which does this for clients (and cleans up in the
final
release call).
Hope that helps,
.
-----Original Message-----
From: xxxxx@3Dlabs.com [mailto:xxxxx@3Dlabs.com]
Sent: Monday, July 12, 2004 4:52 AM
Subject: RE: Memory allocation.
Have a look at MmProbeAndLockPages in the DDK docs. That’s probably what
you
want…
I’m pretty sure the pages stay locked, whatever the app does, until the
driver unlocks them, which the driver SHOULD do when it’s finished using
the
locked pages.
–
Mats
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Ronen Ochman
Sent: Monday, July 12, 2004 1:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Memory allocation.
Hi guys,
I remember I done such things at my old VxD day but I’m not
sure
what technology exists at WDM.
How can I lock pages from user mode in order to use them at
kernel?
I’m talking about big memory buffer.
What happened if the process that allocate those pages crash?
Can I still access those page.
Thanks,
Ronen Ochman
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@3dlabs.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@checkpoint.com To
unsubscribe send a blank email to xxxxx@lists.osr.com
Ronen Ochman wrote:
Thanks Henry,
The question is if I’m using AWE (or any other way) what happened for those
pages when the user-mode process die (crash)? Is windows unlock those pages
automatically, so any access from kernel mode will case BOSD?
Memory will remain locked until it unlocked by your kernel mode driver.
So there is no problem to access it from kernel mode. You can catch the
CLOSE/CLEANUP and unlock the memory after your application is gone (
exites/crashed etc…)
Ilya Lifshits.
Software Team Leader
Jungo Software Technologies
Email: xxxxx@jungo.com
Web: http://www.jungo.com
Ronen Ochman wrote:
Thanks Henry,
The question is if I’m using AWE (or any other way) what happened for those
pages when the user-mode process die (crash)? Is windows unlock those pages
automatically, so any access from kernel mode will case BOSD?
Memory will remain locked until it unlocked by your kernel mode driver.
So there is no problem to access it from kernel mode. You can catch the
CLOSE/CLEANUP and unlock the memory after your application is gone (
exites/crashed etc…)
Ilya Lifshits.
Software Team Leader
Jungo Software Technologies
Email: xxxxx@jungo.com
Web: http://www.jungo.com