Thank you for answers.
I am curios about one thing. In intel arc. if Ring 0 can access rings 0 through 3. Why can’t a driver call directly win32 api functions?
> I am curios about one thing. In intel arc. if Ring 0 can access rings 0 through 3. Why can’t a driver call
directly win32 api functions?
They require another execution context then the driver’s.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
A couple reasons come to mind. First, the Windows APIs are not called as
functions; instead, ntdll.dll, which is a user-level DLL, converts them to a
sequence of loading-values-to-registers followed by a sysenter call. So the
“Win32 API” as such does not exist when you’re in the kernel. Even if you
somehow called the sysenter, the parameters would most likely be validated
as if they were user-mode addresses (look at sysenter, and realize it
probably can’t be called from kernel mode anyway).
There is a set of Nt and Zw functions that represent the lower-level
functions called from user space via sysenter (through a much more
convoluted path).
joe
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, January 14, 2009 4:02 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send msg to user mode?
Thank you for answers.
I am curios about one thing. In intel arc. if Ring 0 can access rings 0
through 3. Why can’t a driver call directly win32 api functions?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
Calls to the Win32 API are made through normal function exports into DLLs like kernel32.dll, advapi32.dll, etc… which may in turn call into the NT API, exported by NTDLL and consisting in large part of the system-call stubs to which you refer.
-p
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Joseph M. Newcomer
Sent: Wednesday, January 14, 2009 4:11 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to send msg to user mode?
A couple reasons come to mind. First, the Windows APIs are not called as
functions; instead, ntdll.dll, which is a user-level DLL, converts them to a
sequence of loading-values-to-registers followed by a sysenter call. So the
“Win32 API” as such does not exist when you’re in the kernel. Even if you
somehow called the sysenter, the parameters would most likely be validated
as if they were user-mode addresses (look at sysenter, and realize it
probably can’t be called from kernel mode anyway).
There is a set of Nt and Zw functions that represent the lower-level
functions called from user space via sysenter (through a much more
convoluted path).
joe
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, January 14, 2009 4:02 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send msg to user mode?
Thank you for answers.
I am curios about one thing. In intel arc. if Ring 0 can access rings 0
through 3. Why can’t a driver call directly win32 api functions?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Yes.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Wieland
Sent: Wednesday, January 14, 2009 7:19 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to send msg to user mode?
Calls to the Win32 API are made through normal function exports into DLLs
like kernel32.dll, advapi32.dll, etc… which may in turn call into the NT
API, exported by NTDLL and consisting in large part of the system-call stubs
to which you refer.
-p
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Joseph M. Newcomer
Sent: Wednesday, January 14, 2009 4:11 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to send msg to user mode?
A couple reasons come to mind. First, the Windows APIs are not called as
functions; instead, ntdll.dll, which is a user-level DLL, converts them to a
sequence of loading-values-to-registers followed by a sysenter call. So the
“Win32 API” as such does not exist when you’re in the kernel. Even if you
somehow called the sysenter, the parameters would most likely be validated
as if they were user-mode addresses (look at sysenter, and realize it
probably can’t be called from kernel mode anyway).
There is a set of Nt and Zw functions that represent the lower-level
functions called from user space via sysenter (through a much more
convoluted path).
joe
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, January 14, 2009 4:02 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send msg to user mode?
Thank you for answers.
I am curios about one thing. In intel arc. if Ring 0 can access rings 0
through 3. Why can’t a driver call directly win32 api functions?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
> Why can’t a driver call directly win32 api functions?
Technically there is nothing that prevents calling UM functions from the kernel mode under x86 - unlike other processors, it has no concept of “privileged mode”. Instead, privilege level is defined by the one of currently running code segment. Therefore, although privileged code cannot call unprivileged one, as long as call is made by a simple CALL instruction, win32 callee will operate as a kernel code
What happens next depends on what calls are made by win32 api. They are more than likely to call system services, so that SYSENTER instruction will be made. IIRC, this instruction explicitly requires unprivileged caller. Even if the target system is below XP so that a transition relies upon INT 0x2E (i.e. a transition to the the system service dispatcher will be made successfully) , problems still may arise when it comes to parameter validation…
Anton Bassov
Anton is, of course, entirely correct.
You might want to read a bit more about this concept, in an article from The NT Insider looong ago (1996): http://www.osronline.com/article.cfm?id=168
The point of the article was to describe execution context to driver writers, back at a time when this basic concept was poorly understood by most driver writers (this was only 3 years after the initial Windows NT release). So, I’m not encouraging you to write code that calls user-mode functions. But the article is a good illustration of the whole point of execution context, I think.
Peter
OSR
OK.
I understand that we may call a user mode dll’s function(if pages weren’t changed), but, as win32dlls, then if function tries to call a kernel mode routine(Sysenter/call gate) there will be a problem.
I read the article. In that article, it says:
“(…)which in turn calls MmMapLockedPages(…) to map the page table entries in the MDL into kernel virtual address space”
What is the meaning of “mapping” page table entries into kernel virtual address space?
I know that in intel arc. there are 4 GB virtual address space, 2 GB for kernel, 2 GB for user. And page table entries point these pages.
-Calling MmProbeAndLockPages: probes the specified virtual memory pages, makes them resident, and locks them in memory
-And then calling MmMapLockedPagesSpecifyCache routine: maps the physical pages that are described by an MDL.
What does “mapping” mean here? What is the process?
> But the article is a good illustration of the whole point of execution context,
In context of this discussion, (no pun intended) the most important part of execution context is, apparently, not even context in itself (after all, the callee does not care what is below it on the stack, so that the fact that stacks are different is not a problem in itself) but how the code arrives to it when it wants to access it in order to obtain info. This is done via FS register, which is different for KM and UM code. UM FS points to PEB (or TEB - I don’t remember which of them) , but the KM one points to Processor Control Region that is described by a totally different structure. Therefore, if win32 function tries to access memory block pointed to by FS register, it is going to interpret Processor Control Region as PEB, and do it while running in privileged mode. BANG!!!
Anton Bassov
xxxxx@gmail.com wrote:
“(…)which in turn calls MmMapLockedPages(…) to map the page table entries in the MDL into kernel virtual address space”
What is the meaning of “mapping” page table entries into kernel virtual address space?
It means creating page tables in the kernel’s virtual space (>=
0x80000000 by default) that point to the same physical addresses as the
page tables that already exist in user space.
I know that in intel arc. there are 4 GB virtual address space, 2 GB for kernel, 2 GB for user. And page table entries point these pages.
I would correct one part of that. The 2GB/2GB split is not an “Intel
architecture” thing; it is merely a Windows design decision. The Intel
x86 architecture allows arbitrary pages to be marked as user or kernel.
The way that is managed is strictly up to the operating system.
-Calling MmProbeAndLockPages: probes the specified virtual memory pages, makes them resident, and locks them in memory
-And then calling MmMapLockedPagesSpecifyCache routine: maps the physical pages that are described by an MDL.What does “mapping” mean here? What is the process?
It means creating a page table entry for each page, which assigns a new
virtual address to that physical address.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Hmm… OK. Thank you very much. We need this because memory manager may change the page table entries for UM pages. So we *must* make it safe. Right?
But in DMA operation, MmGetMdlVirtualAddress is used. And it returns StartVa, right?
And we pass it to MapTransfer function. Is startva in virtual addres space? Why isn’t MappedSystemVa used?
See below…
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, January 15, 2009 3:57 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to send msg to user mode?
OK.
I understand that we may call a user mode dll’s function(if pages weren’t
changed), but, as win32dlls, then if function tries to call a kernel mode
routine(Sysenter/call gate) there will be a problem.
I read the article. In that article, it says:
“(…)which in turn calls MmMapLockedPages(…) to map the page table
entries in the MDL into kernel virtual address space”
What is the meaning of “mapping” page table entries into kernel virtual
address space?
******************
*
It means that you are given a memory address which is in the address space
of kernel memory which you can use to access the pages. Normally, pages are
mapped using user-mode addresses, and each process has its own private
mapping from virtual addresses to physical addresses. If you have two
instructions
mov eax, value
mov value, eax
(which admittedly is a bit silly, since it loads the value from location
‘value’ into eax, and puts it right back where it came from), it is entirely
possible that the *physical* page in memory that represents ‘value’ could
*change* between the two instructions. It works like this:
Just as your program finishes the mov eax, value instruction, a
high-priority and very, very large process causes your thread to be
suspended. While that high-priority thread is running in the other process,
it causes a lot of page faults and pages move in and out of memory to cope
with this. One of the pages that is pushed out to the disk to make room for
the high-priority thread is your page that holds ‘value’. Later, when your
thread resumes execution, it takes a page fault, and the page is retrieved
from disk, and put *somewhere else* in physical memory. So the actual
physical page has changed, but your program never knows this, because the
virtual memory map has taken this into account.
The kernel has its own memory map (for more details see below). When a page
is ‘locked’, it is marked as ‘not pageable’ and the paging logic is not
permitted to move it; it is locked into that physical address until it is
unlocked. When you call MmMapLockedPages, the set of locked pages
represented by the MDL are entered into the *kernel* page map, and thus
given (contiguous) addresses in kernel memory space. So now there are *two*
page map entries for each page: one page map entry in the kernel map, and
one page map entry in the user map. [Note that in general there can be MANY
page maps that reference these pages, such as if the pages are shared
between processes, but that’s a generalization]
*
*******************
I know that in intel arc. there are 4 GB virtual address space, 2 GB for
kernel, 2 GB for user. And page table entries point these pages.
*******************
*
No, the 2GB is not intrinsic to the x86; if you boot with the 3GB user
option, the user space can be 3GB and the kernel space is 1GB. In Win64,
the kernel space starts at 8TB, and user mode can be 4GB per process
(providing you linked with /LARGEADDRESSAWARE). The 2/2 split is completely
arbitrary (when I teach, I mime tossing a coin, examining it, and saying
“Besides, who could need more than 2GB of memory to write a program?” [IIS,
Oracle, SQL server, among others…]
*
*******************
-Calling MmProbeAndLockPages: probes the specified virtual memory pages,
makes them resident, and locks them in memory -And then calling
MmMapLockedPagesSpecifyCache routine: maps the physical pages that are
described by an MDL.
What does “mapping” mean here? What is the process?
********************
*
It’s always a table lookup. Every time you present an “address” to the
hardware, it looks it up, by using the high-order 20 bits (in Win32) to
select a table entry which holds the high-order 20-to-24 bits of a physical
memory address of a 4096-byte page. This physical address is combined with
the low-order 12 bits of the address and the result is a 32-to-36-bit
physical address which is placed on the address bus of the hardware (x86 can
support up to a 36-bit physical address, allowing 64GB of memory, but only
on some models of CPUs and motherboard chips; lower-end chips generate only
a 32-bit address, but I digress).
Note that this means that to represent 2GB of user space, you would need a
4MB table per process (if I’ve not dropped a decimal point here…) if it
were simple indexing. This is impractical in general, so there is a complex
tree-structured organization of page tables that reduces the amount of space
required to represent a page table in memory, so that if you are not using
massive chunks of memory, you don’t need to have any page table entries to
represent them. In addition, because this would mean making two memory
accesses for each access (one to get the table entry and one to load or
store the data), your program would suffer severe performance degradation
when running virtual, so a special cache, called the Translation Lookaside
Buffer (TLB) keeps track of the recent page translations so it can do the
translation without doing a table lookup to memory each time. You can read
about all of this in any of the Intel manuals, but it is a lot of messy
detail, and typically I just say “It’s all magic”. All you really need to
know is
Virtual address –> magic –> physical address
And to invoke the magic, you wave your wand and say “MmMapLockedPages” in
your best Harry Potter imitation and it creates the magic required to make
that mapping work.
*
********************
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
xxxxx@gmail.com wrote:
Hmm… OK. Thank you very much. We need this because memory manager may change the page table entries for UM pages. So we *must* make it safe. Right?
Well, the issue is that the user-mode page tables are completely
replaced every time there is a process switch. If there’s a chance the
driver will need to access this memory at other times (during an ISR,
during a DPC, during a completion routine, during a timer routine,
etc.), then you must make sure the memory pages are locked (so they
can’t be freed) and mapped into kernel space, which does NOT change on a
process switch.
But in DMA operation, MmGetMdlVirtualAddress is used.
Only if the MDL is known to map to a kernel address. Otherwise, you’d
need to use MmGetSystemAddressForMdlSafe, which will map the MDL into
kernel space if it is not already there. Remember that, in a DMA
operation, you want the PHYSICAL addresses, which are embedded inside
the MDL.
And it returns StartVa, right?
Yes, which can be in either user space or kernel space.
And we pass it to MapTransfer function. Is startva in virtual addres space? Why isn’t MappedSystemVa used?
In general, you should use the APIs instead of messing with the members
inside the MDL directly. MappedSystemVa will be garbage unless
MmGetSystemAddressForMdlSafe has been called.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> What is the meaning of “mapping” page table entries into kernel virtual address space?
Pages are mapped, not page table entries.
“Mapping” means - creating a system PTE range pointing to these pages, so that the pages are visible via some kernel-space addresses.
It is separated to steps of - allocate a system PTE range, set these PTEs. In XP and later, there are decomposed APIs for each of these steps.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
> Hmm… OK. Thank you very much. We need this because memory manager may change the page
table entries for UM pages. So we *must* make it safe. Right?
Yes.
But in DMA operation, MmGetMdlVirtualAddress is used. And it returns StartVa, right?
Actually, offset is used in DMA APIs, but due to some strange (mis)design the offset must be specified in the form of ( (PUCHAR)MmGetMdlVirtualAddress(…) + Offset ).
And we pass it to MapTransfer function. Is startva in virtual addres space?
This is just a minor implementation detail of MapTransfer and similar DMA APIs. It would be much more logical for them to have Offset parameter instead of base address, but the life is as it is, and you must use ( (PUCHAR)MmGetMdlVirtualAddress(…) + Offset ) instead of Offset.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com