Hello. I need to use a heap allocator, which is able to allocate blocks of non-fixed length. With such functionality I want to share memory between my driver and user-mode application, because a lot of data should be transfered and I don’t want to call often DeviceIoControl and copy buffers manually.
So I tried ZwAllocateVirtualMemory, RtlCreateHeap in my DriverEntry. But after DriverEntry pass, I get a BSOD in process services.exe (IRQL_NOT_LESS_OR_EQUAL) in context of process “services.exe”.
Code is below:
PVOID HpCreateHeap(
OUT PVOID *pHeapAddr,
IN SIZE_T HeapSize)
{
HANDLE RetVal = NULL;
RTL_HEAP_PARAMETERS heapParams;
NTSTATUS Status;
DWORD RegionSize = HeapSize;
RtlZeroMemory(&heapParams, sizeof(heapParams));
heapParams.Length = sizeof(heapParams);
*pHeapAddr = NULL;
Status = ZwAllocateVirtualMemory((HANDLE)-1, pHeapAddr,0, &RegionSize, MEM_COMMIT,PAGE_READWRITE);
if (!NT_SUCCESS(Status)) return NULL;
RetVal = RtlCreateHeap(
0,
pHeapAddr,
HeapSize,
HeapSize,
NULL,
0
);
return RetVal;
}
What that BSOD caused? So what should I do to prevent that?
These functions are undocumented and require to operate in the context of
the requesting process. Even if you get them working you either have to
have some complex mechanisms for notification of data arrival or go to the
kernel to get notified which removes the justification on this.
How many 100’s of MB/sec are you trying to do, that makes you think you
need this? Be aware that if you do need that bandwidth the heap functions
cannot handle it, and the context switching to have things in the right
process can kill you. I suspect if you try the standard IOCTL method and
write clean code, you will get the bandwidth you want unless you are
streaming video or similar applications.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
wrote in message news:xxxxx@ntdev…
> Hello. I need to use a heap allocator, which is able to allocate blocks
> of non-fixed length. With such functionality I want to share memory
> between my driver and user-mode application, because a lot of data should
> be transfered and I don’t want to call often DeviceIoControl and copy
> buffers manually.
>
> So I tried ZwAllocateVirtualMemory, RtlCreateHeap in my DriverEntry. But
> after DriverEntry pass, I get a BSOD in process services.exe
> (IRQL_NOT_LESS_OR_EQUAL) in context of process “services.exe”.
> Code is below:
>
> PVOID HpCreateHeap(
> OUT PVOID *pHeapAddr,
> IN SIZE_T HeapSize)
> {
>
> HANDLE RetVal = NULL;
> RTL_HEAP_PARAMETERS heapParams;
> NTSTATUS Status;
> DWORD RegionSize = HeapSize;
>
> RtlZeroMemory(&heapParams, sizeof(heapParams));
> heapParams.Length = sizeof(heapParams);
>
> *pHeapAddr = NULL;
> Status = ZwAllocateVirtualMemory((HANDLE)-1, pHeapAddr,0, &RegionSize,
> MEM_COMMIT,PAGE_READWRITE);
> if (!NT_SUCCESS(Status)) return NULL;
>
> RetVal = RtlCreateHeap(
> 0,
> pHeapAddr,
> HeapSize,
> HeapSize,
> NULL,
> 0
> );
>
> return RetVal;
> }
>
> What that BSOD caused? So what should I do to prevent that?
>
Thank you, Don for your valuable notices. But both those functions are documented in Windows DDK. So, it is not clear for me, when they can be used…
Sorry, you are correct I forgot they got slapped in doc’s with the WDK.
The bottom line is that using them is hard, and will complicate your code
significantly for no value unless you are doing extremely fast data
transfers, and are willing to spend a lot of time on getting the user to
kernel synchronization scheme correct.
If you really want to do this stuff, then look at some of the articles of
sharing memory from the kernel to user space. It is out of date, but you
might look at the MemoryMap.zip located at
http://www.eclectic-eng.com/download.htm.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
wrote in message news:xxxxx@ntdev…
> Thank you, Don for your valuable notices. But both those functions are
> documented in Windows DDK. So, it is not clear for me, when they can be
> used…
>
>
That example can help to solve problem how to use same memory area in driver and application. But when I have a lot of structures of variable length, I want to allocate it from one big memory block (which I previously mapped into application’s space), i.e. I need own heap allocator for blocks of variable length. I supposed, that RtlCreateHeap will help in such situation. But there is a lot of problems with it. So is there mechanism like RtlCreateHeap ready for use? Or it must be written by oneself ? If second, is there such libraries, sources, examples with such heap realization?
You do realize that if you are talking about a lot of variable length
blocks with small allocations (at least less than 1MB) that you are
creating:
-
A major potential for a security hole, anytime you share memory
between an app and a driver you are opening security, so far this does not
sound like you have a lot of reasons to.
-
A synchronization nightmare, you have to synchronize the kernel
and the application, preferably without a lot of kernel calls (or else why
are you sharing the memory in the first place).
-
You have to extremely careful to handle application failure, since
with the heap approach you can easily crash the system when the application
dies unexpectedly.
I would recomend you rethink your design. This is unlikely to be a stable
driver.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
wrote in message news:xxxxx@ntdev…
> That example can help to solve problem how to use same memory area in
> driver and application. But when I have a lot of structures of variable
> length, I want to allocate it from one big memory block (which I
> previously mapped into application’s space), i.e. I need own heap
> allocator for blocks of variable length. I supposed, that RtlCreateHeap
> will help in such situation. But there is a lot of problems with it. So
> is there mechanism like RtlCreateHeap ready for use? Or it must be
> written by oneself ? If second, is there such libraries, sources,
> examples with such heap realization?
>
Don Burn wrote:
These functions are undocumented and require to operate in the context of
the requesting process. Even if you get them working you either have to
have some complex mechanisms for notification of data arrival or go to the
kernel to get notified which removes the justification on this.
How many 100’s of MB/sec are you trying to do, that makes you think you
need this? Be aware that if you do need that bandwidth the heap functions
cannot handle it, and the context switching to have things in the right
process can kill you. I suspect if you try the standard IOCTL method and
write clean code, you will get the bandwidth you want unless you are
streaming video or similar applications.
Even streaming video doesn’t need these kinds of tricks. A full D1
video stream is only 27 MB/sec. We recently did a video capture driver
that needed to use DMA common buffers, which means I had to copy every
frame to the user buffer. With two full streams running simultaneously,
the CPU load was less than 10%.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Don Burn wrote:
>
>> How many 100’s of MB/sec are you trying to do, that makes you think you
>> need this? Be aware that if you do need that bandwidth the heap
>> functions
>> cannot handle it, and the context switching to have things in the right
>> process can kill you. I suspect if you try the standard IOCTL method
>> and
>> write clean code, you will get the bandwidth you want unless you are
>> streaming video or similar applications.
>
> Even streaming video doesn’t need these kinds of tricks. A full D1
> video stream is only 27 MB/sec. We recently did a video capture driver
> that needed to use DMA common buffers, which means I had to copy every
> frame to the user buffer. With two full streams running simultaneously,
> the CPU load was less than 10%.
>
Tim, I meant HD TV no-compression, I was doing this in 1998 when getting a
150MB/sec was a little hard.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
RtlCreateHeap is the same as Win32’s HeapCreate, I even expect
kernel32!HeapCreate to be a forwarder to ntdll!RtlCreateHeap, like
kernel32!HeapAlloc is ntdll!RtlAllocateHeap.
ZwAllocateVirtualMemory is the core of VirtualAlloc, effectively
VirtualAlloc = ZwAllocateVirtualMemory + error code conversion from NTSTATUS to
Win32 error.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> Hello. I need to use a heap allocator, which is able to allocate blocks of
non-fixed length. With such functionality I want to share memory between my
driver and user-mode application, because a lot of data should be transfered
and I don’t want to call often DeviceIoControl and copy buffers manually.
>
> So I tried ZwAllocateVirtualMemory, RtlCreateHeap in my DriverEntry. But
after DriverEntry pass, I get a BSOD in process services.exe
(IRQL_NOT_LESS_OR_EQUAL) in context of process “services.exe”.
> Code is below:
>
> PVOID HpCreateHeap(
> OUT PVOID *pHeapAddr,
> IN SIZE_T HeapSize)
> {
>
> HANDLE RetVal = NULL;
> RTL_HEAP_PARAMETERS heapParams;
> NTSTATUS Status;
> DWORD RegionSize = HeapSize;
>
> RtlZeroMemory(&heapParams, sizeof(heapParams));
> heapParams.Length = sizeof(heapParams);
>
> *pHeapAddr = NULL;
> Status = ZwAllocateVirtualMemory((HANDLE)-1, pHeapAddr,0, &RegionSize,
MEM_COMMIT,PAGE_READWRITE);
> if (!NT_SUCCESS(Status)) return NULL;
>
> RetVal = RtlCreateHeap(
> 0,
> pHeapAddr,
> HeapSize,
> HeapSize,
> NULL,
> 0
> );
>
> return RetVal;
> }
>
> What that BSOD caused? So what should I do to prevent that?
>
>driver and application. But when I have a lot of structures of variable
length, I
want to allocate it from one big memory block (which I previously mapped into
application’s space), i.e. I need own heap allocator for blocks of variable
length. I
No need. Use the usual HeapCreate and HeapAlloc.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com