How to get physical pages from user space for DMA transaction.

>we are speaking about the large pages/physically contiguous buffers here

Hmmm? What am I missing?

The user calls VirtualAllocEx specifying reserve, commit, and large pages. Then passes up the UVA to the driver in some sort of Direct I/O request.

Isn’t that all there is to it? No mystic privs required in that case, are there?

Peter
OSR
@OSRDrivers

> Isn’t that all there is to it? No mystic privs required in that case, are there?

https://docs.microsoft.com/en-us/windows/desktop/memory/large-page-support

As you can see, before you can call VirtualAlloc() for getting a large page you have to enable SeLockMemoryPrivilege with AdjustTokenPrivileges() , and before you are in a position to do it you have to ensure that the account has been actually assigned this privilege - you cannot add new privileges by means of AdjustTokenPrivileges(), can you. IIRC, this part can be done only via the management console and not programatically…

Anton Bassov

Well, I learned something today.

Despite the docs you pointed to being very clear, a blog post by Raymond Chen (I mean, what does HE know, right?), and numerous articles on SO… I couldn’t believe it. So, I worked super hard and wrote my own program (OK, not really, I copied 90% of it from somebody somewhere):

int rc, min;

min = GetLargePageMinimum();

void *p = VirtualAlloc(NULL,
min,
MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES,
PAGE_READWRITE);

rc = GetLastError();

printf(“Pointer = %p, Error = %d.\n”, p, rc);

You know what? Error = 1314 – “A required privilege is not held”… I’ll be darned.

Gotta keep those pesky user-mode programmers under control, I guess! Can’t have them allocating LARGE PAGES. OMG!

Thanks Anton. I hereby grant you a free pass to make one rude, crude, arrogant, and socially irresponsible post here on NTDEV without being banned for life.

:wink:

Peter
OSR
@OSRDrivers

Actually, I believe I should earn some "good karma"on NTDEV before even thinking about switching to the “ironical mode”, taking into consideration the troll-management capabilities of the upcoming platform. Therefore, for the time being I will try my best to make some posts that fall into the “useful” category in order to avoid “The Hanging Judge’s” potential wrath…

Anton Bassov

There is an important reason for this restriction: large pages are not pageable. Think about it. You have some page file(s) full of 4k chunks with lots of data in them. Assume for efficiency that you manage the data in these files with some kind of bitmap ? like say the Windows team has done ? and then say now a want a 2 MB chunk to page out this large page I have been working on. Clearly it doesn?t work.

The ?obvious? solution is to say that large pages are chiefly used in limited situations such as mapping kernel32.dll or by ?advanced? users who have lots of RAM and full control over there systems and therefore it is okay to just make then not pageable. Once you have arrived at this conclusion it is not hard to see how the need for the lock pages in memory privilege becomes the next requirement.

IIRC Anton is wrong however in that this privilege is granted (but not enabled) for administrators by default. This has changed from Windows version to version and it has been a long time since I worked outside of a GPO controlled environment so I might have this mistaken

Any properly written code that tries to use large pages as a performance optimization can fail back to an successfully use normal pages. That?s what they are, a performance optimization for reducing the TLB lookup cost. Any other use is nonsense.

Sent from Mailhttps: for Windows 10

________________________________
From: xxxxx@lists.osr.com on behalf of xxxxx@osr.com
Sent: Friday, August 31, 2018 3:37:32 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to get physical pages from user space for DMA transaction.

Well, I learned something today.

Despite the docs you pointed to being very clear, a blog post by Raymond Chen (I mean, what does HE know, right?), and numerous articles on SO… I couldn’t believe it. So, I worked super hard and wrote my own program (OK, not really, I copied 90% of it from somebody somewhere):

int rc, min;

min = GetLargePageMinimum();

void *p = VirtualAlloc(NULL,
min,
MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES,
PAGE_READWRITE);

rc = GetLastError();

printf(“Pointer = %p, Error = %d.\n”, p, rc);

You know what? Error = 1314 – “A required privilege is not held”… I’ll be darned.

Gotta keep those pesky user-mode programmers under control, I guess! Can’t have them allocating LARGE PAGES. OMG!

Thanks Anton. I hereby grant you a free pass to make one rude, crude, arrogant, and socially irresponsible post here on NTDEV without being banned for life.

:wink:

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:></https:>

> IIRC Anton is wrong however in that this privilege is granted (but not enabled) for

administrators by default.

Please read my post more carefully. Look what I had said

If it was granted by default there would be no problem whatsoever - the only thing that you would be required to do in such case is to enable this privilege in a token, which, unlike adding privileges to the account, may be done programatically. In fact, there would be no need to even mention it, in the first place. However, the fact that it requires user interaction with a console adds sort of an extra “complication”. Let’s face it - telling end users that they have to configure the OS in a certain way before your piece of hardware can be utilised does not really seem to add any extra selling points to your product, don’t you think…

This has changed from Windows version to version and it has been a long time
>since I worked outside of a GPO controlled environment so I might have this mistaken

Fair enough -assuming that this feature may change from one OS version to another, I have to admit that my practical experience with Windows as a user is VERY outdated. Unless we count booting up a new machine and taking whatever steps are necessary under the given OS version
before “defenestration process” can be successfully launched, as a “user experience”, the last Windows version that I have practical experience with is XP.

The only thing that I don’t understand is WHY it should be changing under different OS versions, in the first place. After all, the principle of “least privilege by default” seems to be pretty universal, and the ability of user apps to lock physical pages in RAM does not seem to be of crucial importance in 95+% of cases, does it…

Anton Bassov

Went threw some code I wrote many years ago for testing LARGE_PAGES. Here
is what i had to do:

static LPVOID Alloc(DWORD dwSize)
{
#ifdef _WIN32
DWORD type = MEM_COMMIT | MEM_RESERVE;
#if defined (_USE_LARGE_PAGES)
DWORD isize = dwSize;
HANDLE hToken;
TOKEN_PRIVILEGES tp;

// open process token
if (OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES |
TOKEN_QUERY, &hToken))
{
// get the luid
if (LookupPrivilegeValueA(NULL, “SeLockMemoryPrivilege”,
&tp.Privileges[0].Luid))
{
BOOL status;
DWORD error;
tp.PrivilegeCount = 1;
tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED;
// enable privilege
status = AdjustTokenPrivileges(hToken, FALSE, &tp, 0,
(PTOKEN_PRIVILEGES)NULL, 0);
// It is possible for AdjustTokenPrivileges to return TRUE and still not
succeed.
// So always check for the last error value.
error = GetLastError();
if (status && (error == ERROR_SUCCESS))
{
HMODULE hModule;
PGLPM pGLPM;

pGLPM = (PGLPM)NULL;
hModule = GetModuleHandleA( “Kernel32.dll” );
if ( hModule )
pGLPM = (PGLPM)GetProcAddress( hModule, “GetLargePageMinimum” );

// Get environemnt specific large page size
//SIZE_T minsize = GetLargePageMinimum();
SIZE_T minsize = 0;
if ( (PGLPM)NULL != pGLPM )
minsize = pGLPM();

if ( 0 == minsize )
{
// If the processor does not support large pages, the return value is zero.
// Or GetLargePageMinimum is not exist in Kernel32.dll
minsize = 2 * 1024 * 1024; // The minimum large page size varies, but it is
typically 2 MB or greater.
}

if (isize >= minsize)
{ // If we get this far, we know that we can allocate large pages
SIZE_T blocks = isize / minsize;
// Allocation size must be multiple of large page size
if (isize % minsize)
blocks++;
isize = blocks * minsize;
type |= MEM_LARGE_PAGES;
} // Too small; no need for large pages
#if defined (_FORCE_LARGE_PAGES)
else
{
isize = minsize;
type |= MEM_LARGE_PAGES;
}
#endif
} // Error setting privileges
} // Error on privilege lookup
} // Error opening token

// If we failed to enable large page security above, the function will
allocate
// as normal; so no failure in this case
PVOID m = VirtualAlloc(NULL, isize, type, PAGE_READWRITE);

if (hToken != INVALID_HANDLE_VALUE)
{
tp.Privileges[0].Attributes = 0;
tp.PrivilegeCount = 1;
// disable privilege
AdjustTokenPrivileges(hToken, FALSE, &tp, 0, (PTOKEN_PRIVILEGES)NULL, 0);
// close the handle
CloseHandle(hToken);
}

if (!m && (type & MEM_LARGE_PAGES))
{ // Large page allocation failed, revert back to normal allocation
type &= ~MEM_LARGE_PAGES;
m = VirtualAlloc(NULL, dwSize, type, PAGE_READWRITE);
}
else
{
}

return m;
#else
return VirtualAlloc(NULL, dwSize, type, PAGE_READWRITE);
#endif
#else
void*p = NULL;
if(0 == posix_memalign(&p, 0x1000, dwSize) )
return p;
return NULL;
#endif

}

On Mon, Sep 3, 2018 at 7:18 AM xxxxx@hotmail.com
wrote:

> > IIRC Anton is wrong however in that this privilege is granted (but not
> enabled) for
> > administrators by default.
>
>
> Please read my post more carefully. Look what I had said
>
>


>
>
>
> If it was granted by default there would be no problem whatsoever - the
> only thing that you would be required to do in such case is to enable this
> privilege in a token, which, unlike adding privileges to the account, may
> be done programatically. In fact, there would be no need to even mention
> it, in the first place. However, the fact that it requires user interaction
> with a console adds sort of an extra “complication”. Let’s face it -
> telling end users that they have to configure the OS in a certain way
> before your piece of hardware can be utilised does not really seem to add
> any extra selling points to your product, don’t you think…
>
>
>
> > This has changed from Windows version to version and it has been a long
> time
> >since I worked outside of a GPO controlled environment so I might have
> this mistaken
>
>
> Fair enough -assuming that this feature may change from one OS version to
> another, I have to admit that my practical experience with Windows as a
> user is VERY outdated. Unless we count booting up a new machine and taking
> whatever steps are necessary under the given OS version
> before “defenestration process” can be successfully launched, as a “user
> experience”, the last Windows version that I have practical experience
> with is XP.
>
>
>
> The only thing that I don’t understand is WHY it should be changing under
> different OS versions, in the first place. After all, the principle of
> “least privilege by default” seems to be pretty universal, and the ability
> of user apps to lock physical pages in RAM does not seem to be of crucial
> importance in 95+% of cases, does it…
>
>
> Anton Bassov
>
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
>


Jamey Kirby
Disrupting the establishment since 1964

This is a personal email account and as such, emails are not subject to
archiving. Nothing else really matters.
</http:>

Hi All,

Thank you for your precious response. I have concern about security as well.

I need one more help on same that can I allocate more than 4M at the time of

WdfCommonBufferCreate(DmaEnabler,WriteCommonBufferSize = 20M, WDF_NO_OBJECT_ATTRIBUTES, &DevExt->WriteCommonBuffer);

if yes than Please tell me requirement.because while allocating this I am not able to install driver.

Thanks,
Kishan Patel

Hmmmmm…

How much memory do you have on the system? You’re (effectively) trying to allocate 20MB of contiguous memory… which can be difficult once the system is up and running.

Peter
OSR
@OSRDrivers

Hi Peter,

We are using 4gb RAM. Is there any way that I can allocate at least around 8MB Buffer?

Thanks,
Kishan Patel

Well. Please read mine carefully too ???

SeLockMemoryPrivilege can be assigned to a user account either explicitly or via group membership either via local security policy of group policy from an AD. In the same way that absolutely any other privilege can be assigned. Privileges and ACLs are the two fundamental security concepts in Windows (and most other systems). What has been assigned by default in the local security policy and the default AD GPOs has changed significantly over the years

ACLs are for us commoners who for example in the soviet system have achieved level 7 systems programmer and are therefore entitled to a cat as a domestic animal. Do you prefer a cat or a dog?

Privileges are assigned to those who can override this kind of ordinary restraint. Locking pages in memory; backing up sensitive files; these are the chief uses of this kind of autgo

Sent from Mailhttps: for Windows 10

________________________________
From: xxxxx@lists.osr.com on behalf of xxxxx@hotmail.com
Sent: Monday, September 3, 2018 7:17:54 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to get physical pages from user space for DMA transaction.

> IIRC Anton is wrong however in that this privilege is granted (but not enabled) for
> administrators by default.

Please read my post more carefully. Look what I had said



If it was granted by default there would be no problem whatsoever - the only thing that you would be required to do in such case is to enable this privilege in a token, which, unlike adding privileges to the account, may be done programatically. In fact, there would be no need to even mention it, in the first place. However, the fact that it requires user interaction with a console adds sort of an extra “complication”. Let’s face it - telling end users that they have to configure the OS in a certain way before your piece of hardware can be utilised does not really seem to add any extra selling points to your product, don’t you think…

> This has changed from Windows version to version and it has been a long time
>since I worked outside of a GPO controlled environment so I might have this mistaken

Fair enough -assuming that this feature may change from one OS version to another, I have to admit that my practical experience with Windows as a user is VERY outdated. Unless we count booting up a new machine and taking whatever steps are necessary under the given OS version
before “defenestration process” can be successfully launched, as a “user experience”, the last Windows version that I have practical experience with is XP.

The only thing that I don’t understand is WHY it should be changing under different OS versions, in the first place. After all, the principle of “least privilege by default” seems to be pretty universal, and the ability of user apps to lock physical pages in RAM does not seem to be of crucial importance in 95+% of cases, does it…

Anton Bassov


NTDEV is sponsored by OSR

Visit the list online at: https:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at https:

To unsubscribe, visit the List Server section of OSR Online at https:</https:></https:></https:></https:>

Hi Marion,

Can you help me to get steps how I can lock user buffer allocated with malloc and pass that pages for DMA operation.

Thanks,
Kishan Patel

On Sep 5, 2018, at 5:02 AM, xxxxx@gmail.com wrote:
>
> We are using 4gb RAM. Is there any way that I can allocate at least around 8MB Buffer?

There are no guarantees. If you do this at boot time, you should be able to get your full 20MB allocation. After the system has been running for a while, physical memory gets quite fragmented. When you ask for a block, the system does try to rearrange things to create the region you want, but it might not work.

Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Hi Tim,

Is there any other way that we can make sure at least 8M buffer should be allocated? Or Either we can lock page from user space and that page can be used for DMA transaction?

Thanks,
Kishan Patel

On Sep 5, 2018, at 10:48 PM, xxxxx@gmail.com wrote:
>
> Is there any other way that we can make sure at least 8M buffer should be allocated? Or Either we can lock page from user space and that page can be used for DMA transaction?

Well, hang on a minute. The only reason you’re having trouble grabbing 20MB is that you are asking for a common buffer – memory that has continuous physical addresses. You can’t get that from user mode at all. If you have scatter/gather hardware, so that all you need is 20MB of memory, then you don’t need a common buffer. Just use ExAllocatePool. That should always succeed.

Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

> Just use ExAllocatePool.

Or, my favorite: MmAllocatePagesForMdlEx

(For some reason, seeing a call to ExAllocatePoolWithTag for a 20MB chunk always makes me feel weird… It’s kinda like seeing a 20MB call to malloc() to me…)

Peter
OSR
@OSRDrivers

Hi Tim,

–> Well, hang on a minute. The only reason you’re having trouble grabbing 20MB is
that you are asking for a common buffer – memory that has continuous physical
addresses. You can’t get that from user mode at all.

I am allocating common buffer on driver itself and I am using same common buffer’s logical address for DMA transaction. Now I want to increase size of allocation more than 8MB so on a single transaction from user to kernel I can send data which is more than 8MB.


–> If you have scatter/gather hardware, so that all you need is 20MB of memory, then you don’t
need a common buffer. Just use ExAllocatePool. That should always succeed.
???

My hardware don’t support SG.
https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an690.pdf

So here what I feel is I have two option to increase throughput.

  1. With same common buffer if I succeed to get large buffer so I can minimize iteration from user space to kernel space and I can send large buffer from user to kernel for operation.

  2. If i can lock user buffer pages for DMA operation so it will save my time of copying data to/from common buffer. I can use that address directly from user space.
    I am curiously wanted to know how we can lock pages from user space for DMA operation as JUNGO is doing. I know we had discussion that it’s HUGE security hole. But I wanted to know this method. Can you please help me on that?

Please correct me if i am wrong.

Thanks,
Kishan Patel

Hi Peter,

Just use ExAllocatePool.

Or, my favorite: MmAllocatePagesForMdlEx

(For some reason, seeing a call to ExAllocatePoolWithTag for a 20MB chunk always
makes me feel weird… It’s kinda like seeing a 20MB call to malloc() to me…)

Can i use this memory for DMA transaction?

Thanks,
Kishan Patel

>Can i use this memory for DMA transaction?

No.

How about we take a step back, and you tell us what you?re trying to accomplish.

Lets start with ?Does your device support scatter/gather??

Peter
OSR
@OSRDrivers

>>Can i use this memory for DMA transaction?

No.

How about we take a step back, and you tell us what you?re trying to accomplish.

Lets start with ?Does your device support scatter/gather??

–> No Peter , My device does not support scatter/gather. I am retrieving logical address using common buffer , Processing that logical address and writing that address to CRA register of my DMA. and passing descriptor for DMA operation.

Kishan Patel