virtual -> physical address routine?

NTDEV
1

Hi,

What routine do I use to convert a kernel virtual addr into a physical
address?

THanks,
-PWM

2

MmGetPhysicalAddress, note be sure to have locked down the memory.

–
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Peter W. Morreale” wrote in message
news:xxxxx@ntdev…
>
> Hi,
>
> What routine do I use to convert a kernel virtual addr into a physical
> address?
>
> THanks,
> -PWM
>
>
>
> Information from ESET NOD32 Antivirus, version of virus
> signature database 4577 (20091105)
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>

Information from ESET NOD32 Antivirus, version of virus signature database 4577 (20091105)

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

3

On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:

MmGetPhysicalAddress, note be sure to have locked down the memory.

nod. And just to re-confirm, driver memory is locked by default, right?

-PWM

4

You mean like the memory occupied by the PE? Yes.

mm

5

Define driver memory

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Peter W. Morreale
Sent: Thursday, November 05, 2009 2:36 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] virtual -> physical address routine?

On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:

MmGetPhysicalAddress, note be sure to have locked down the memory.

nod. And just to re-confirm, driver memory is locked by default, right?

-PWM

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

> driver memory is locked by default, right?

In context of the above question there is no such thing as “driver memory”…

Driver image has multiple sections that may have different MM policies. Code and data in those sections that are not explicitly marked as pageable (or INIT) are indeed, guaranteed to be always present in RAM. Code and data of those sections that are marked as pageable mat get swapped out, and INIT gets discarded from RAM after driver gets initialized.

The only reason why Don mentioned locking is because if physical address returned by MmGetPhysicalAddress () backs up a pageable range it may get reused by MM for totally different purposes by the time you decide to make any practical use of it . Therefore, in order to ensure that you don’t corrupt someone else’s memory you have to tell MM not to reuse the target physical page while you are using it, and by locking the page down you meet this objective…

Anton Bassov

7

And… you wouldn’t be using this to perform DMA, would you?

Sorry, just checking. It’s a particular hot-button for me (see one of the recent articles in The NT Insider)…

Peter
OSR

8

Ahh I was about to ask the same question and give the reference to the NT Insider :-).

9

> What routine do I use to convert a kernel virtual addr into a physical

address?

The Windows philosophy is that your code should only need physical addresses for DMA, and thus should use DMA APIs.

But, if you have a really justificated need, use MmGetPhysicalAddress

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

10

> nod. And just to re-confirm, driver memory is locked by default, right?

Code and static data - yes, allocations - depends on ExAllocate parameters.

Device objects and extensions - yes, so are file objects.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

11

On Thu, 2009-11-05 at 22:43 +0000, Doron Holan wrote:

Define driver memory

d

Text & Stack. I understand about the Pools.

THx
-PWM

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Peter W. Morreale
Sent: Thursday, November 05, 2009 2:36 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] virtual -> physical address routine?

On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:
> MmGetPhysicalAddress, note be sure to have locked down the memory.
>
>

nod. And just to re-confirm, driver memory is locked by default, right?

-PWM

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

12

On Fri, 2009-11-06 at 07:14 +0300, Maxim S. Shatskih wrote:

> What routine do I use to convert a kernel virtual addr into a physical
> address?

The Windows philosophy is that your code should only need physical addresses for DMA, and thus should use DMA APIs.

But, if you have a really justificated need, use MmGetPhysicalAddress

Understood. Our driver supports a particular virtualization technology
in our hypervisor. We only need to know Window’s physical address of
pinned memory in order to complete the data transfer over our pci
bridge.

IIUC, by asking for Direct I/O in WdfDeviceInitSetIoType(), the data
buffers presented to the driver are pinned, in which case we only need
the physical address on the Windows side for each buffer.

This is a software-to-software driver, no actual hardware is involved.
By merely translating addresses, we save (at least) one memory copy.

Thanks,
-PWM

13

On Thu, 2009-11-05 at 18:11 -0500, xxxxx@hotmail.com wrote:

> driver memory is locked by default, right?

In context of the above question there is no such thing as “driver memory”…

Driver image has multiple sections that may have different MM policies. Code and data in those sections that are not explicitly marked as pageable (or INIT) are indeed, guaranteed to be always present in RAM. Code and data of those sections that are marked as pageable mat get swapped out, and INIT gets discarded from RAM after driver gets initialized.

Apologies. I should have been more specific. Having the ability to
page drivers (or even parts of them, save for explicit requests for
pagable memory) is a wholly new concept to me. We do not want our
driver to be pagable.

The only reason why Don mentioned locking is because if physical address returned by MmGetPhysicalAddress () backs up a pageable range it may get reused by MM for totally different purposes by the time you decide to make any practical use of it . Therefore, in order to ensure that you don’t corrupt someone else’s memory you have to tell MM not to reuse the target physical page while you are using it, and by locking the page down you meet this objective…

Understood. Again, asking for ‘direct i/o’ in *SetIoType() handles
that, correct?

Thanks,
-PWM

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

14

Yes, for READ AND WRITE. For IOCTL, the transfer type is defined by the METHOD argument on the CTL_CODE macro.

Peter
OSR

15

Stack can be paged out if you do a UserMode wait, so be careful. Also, i would imagine the data going over the hyper barrier is not small, remember you only have 12K of stack space total in the kernel

d

Sent from my phone with no t9, all spilling mistakes are not intentional.

-----Original Message-----
From: Peter W. Morreale
Sent: Friday, November 06, 2009 6:18 AM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] virtual -> physical address routine?

On Thu, 2009-11-05 at 22:43 +0000, Doron Holan wrote:
> Define driver memory
>
> d

Text & Stack. I understand about the Pools.

THx
-PWM

>
> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Peter W. Morreale
> Sent: Thursday, November 05, 2009 2:36 PM
> To: Windows System Software Devs Interest List
> Subject: Re: Re:[ntdev] virtual -> physical address routine?
>
> On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:
> > MmGetPhysicalAddress, note be sure to have locked down the memory.
> >
> >
>
> nod. And just to re-confirm, driver memory is locked by default, right?
>
> -PWM
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

16

On Fri, 2009-11-06 at 15:28 +0000, Doron Holan wrote:

Stack can be paged out if you do a UserMode wait, so be careful. Also, i would imagine the data going over the hyper barrier is not small, remember you only have 12K of stack space total in the kernel

d

Thank you for that tidbit. I will file that one away. Don’t suppose
there is any way to ensure the stack is also locked down?

As for the 12k stack, this is huge. Linux’s kernel stack is usually 8k

  • sizeof(struct task) = ~6-7k. Whatever will I do with all those extra
    lovely bytes? :slight_smile:

Thx
-PWM

Sent from my phone with no t9, all spilling mistakes are not intentional.

-----Original Message-----
From: Peter W. Morreale
> Sent: Friday, November 06, 2009 6:18 AM
> To: Windows System Software Devs Interest List
> Subject: RE: Re:[ntdev] virtual -> physical address routine?
>
>
> On Thu, 2009-11-05 at 22:43 +0000, Doron Holan wrote:
> > Define driver memory
> >
> > d
>
> Text & Stack. I understand about the Pools.
>
> THx
> -PWM
>
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Peter W. Morreale
> > Sent: Thursday, November 05, 2009 2:36 PM
> > To: Windows System Software Devs Interest List
> > Subject: Re: Re:[ntdev] virtual -> physical address routine?
> >
> > On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:
> > > MmGetPhysicalAddress, note be sure to have locked down the memory.
> > >
> > >
> >
> > nod. And just to re-confirm, driver memory is locked by default, right?
> >
> > -PWM
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

17

> Thank you for that tidbit. I will file that one away. Don’t suppose

there is any way to ensure the stack is also locked down?

Just never call KeWaitForSingleObject/KeWaitForMultipleObjects with the
WaitMode parameter set to UserMode and you’ll be fine.

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Peter W. Morreale” wrote in message
news:xxxxx@ntdev…
> On Fri, 2009-11-06 at 15:28 +0000, Doron Holan wrote:
>> Stack can be paged out if you do a UserMode wait, so be careful. Also, i
>> would imagine the data going over the hyper barrier is not small,
>> remember you only have 12K of stack space total in the kernel
>>
>> d
>
> Thank you for that tidbit. I will file that one away. Don’t suppose
> there is any way to ensure the stack is also locked down?
>
> As for the 12k stack, this is huge. Linux’s kernel stack is usually 8k
> - sizeof(struct task) = ~6-7k. Whatever will I do with all those extra
> lovely bytes? :slight_smile:
>
>
> Thx
> -PWM
>
>>
>> Sent from my phone with no t9, all spilling mistakes are not intentional.
>>
>> -----Original Message-----
>> From: Peter W. Morreale
>> Sent: Friday, November 06, 2009 6:18 AM
>> To: Windows System Software Devs Interest List
>> Subject: RE: Re:[ntdev] virtual -> physical address routine?
>>
>>
>> On Thu, 2009-11-05 at 22:43 +0000, Doron Holan wrote:
>> > Define driver memory
>> >
>> > d
>>
>> Text & Stack. I understand about the Pools.
>>
>> THx
>> -PWM
>>
>> >
>> > -----Original Message-----
>> > From: xxxxx@lists.osr.com
>> > [mailto:xxxxx@lists.osr.com] On Behalf Of Peter W.
>> > Morreale
>> > Sent: Thursday, November 05, 2009 2:36 PM
>> > To: Windows System Software Devs Interest List
>> > Subject: Re: Re:[ntdev] virtual -> physical address routine?
>> >
>> > On Thu, 2009-11-05 at 17:11 -0500, Don Burn wrote:
>> > > MmGetPhysicalAddress, note be sure to have locked down the memory.
>> > >
>> > >
>> >
>> > nod. And just to re-confirm, driver memory is locked by default,
>> > right?
>> >
>> > -PWM
>> >
>> >
>> > —
>> > NTDEV is sponsored by OSR
>> >
>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>> > http://www.osr.com/seminars
>> >
>> > To unsubscribe, visit the List Server section of OSR Online at
>> > http://www.osronline.com/page.cfm?name=ListServer
>> >
>> >
>> > —
>> > NTDEV is sponsored by OSR
>> >
>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>> > http://www.osr.com/seminars
>> >
>> > To unsubscribe, visit the List Server section of OSR Online at
>> > http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>

18

> in our hypervisor. We only need to know Window’s physical address of

pinned memory in order to complete the data transfer over our pci
bridge.

Am I correct that, from the guest’s point of view, this is DMA?

If not - then a) MmGetPhysicalAddress b) digging into MDL tail.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

19

> pagable memory) is a wholly new concept to me. We do not want our

driver to be pagable.

Why?

Imagine a function which calls some API which is < DISPATCH (IoCreateDevice, or registry, or ZwXxxFile, or such).

Then this function is also limited < DISPATCH, and, if this is not a pagefile IO path - then it should be pagable.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

20

> As for the 12k stack, this is huge. Linux’s kernel stack is usually 8k

  • sizeof(struct task)

Oh yes, this layout gives a random hard-to-find crash on stack overflow, while Windows gives clean double fault (0000007F/00000008).

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com