I would like ot have a buffer send to driver from user space. I am trying
to avoid copy memory so I would like to use METHOD_DIRECT for IO/CTL.
My question is :
Jim
The mdl is probed and locked,so you can queue it and use it in any context. The buffer remains valid for as long as the irp is valid, IE until you complete the irp. How big of a buffer are you concerned with?
d
Bent from my phone
From: Jimmailto:xxxxx
Sent: ?6/?9/?2013 10:36 AM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: [ntdev] MDL related question.
I would like ot have a buffer send to driver from user space. I am trying to avoid copy memory so I would like to use METHOD_DIRECT for IO/CTL.
My question is :
- Assuming I allocated in user-space a buffer and use METHOD_DIRECT , am I safe to use the MDL on my driver and queue it ? what if the process crashed ? the MDL still reserved for my driver ? I am not quite sure if the content of the MDL is safe (the content of the phyiscal pages) when the buffers on users modes are exactly freed.
Jim
— NTDEV is sponsored by OSR OSR is HIRING!! See http://www.osr.com/careers For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>
It is not a big buffer, but may have several buffers.
Problem is that the buffer I am reffering to is going to arrive to an
AVStream driver probably by a KS_METHOD with type KSMETHOD_TYPE_SOURCE. I
would like to avoid copy the buffer.
Not quite sure how KS_METHOD work irp-wise. does it overlapped ? If not,
can I keep the mdl probed/locked after KS_METHOD return ?
Thanks,
Jim
On Sun, Jun 9, 2013 at 8:57 PM, Doron Holan wrote:
> The mdl is probed and locked,so you can queue it and use it in any
> context. The buffer remains valid for as long as the irp is valid, IE until
> you complete the irp. How big of a buffer are you concerned with?
>
> d
>
> Bent from my phone
> ------------------------------
> From: Jim
> Sent: 6/9/2013 10:36 AM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] MDL related question.
>
> I would like ot have a buffer send to driver from user space. I am
> trying to avoid copy memory so I would like to use METHOD_DIRECT for IO/CTL.
>
> My question is :
> - Assuming I allocated in user-space a buffer and use METHOD_DIRECT , am I
> safe to use the MDL on my driver and queue it ? what if the process crashed
> ? the MDL still reserved for my driver ? I am not quite sure if the content
> of the MDL is safe (the content of the phyiscal pages) when the buffers on
> users modes are exactly freed.
>
> Jim
>
> — NTDEV is sponsored by OSR OSR is HIRING!! See
> http://www.osr.com/careers For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
Jim wrote:
It is not a big buffer, but may have several buffers.
Problem is that the buffer I am reffering to is going to arrive to an
AVStream driver probably by a KS_METHOD with type
KSMETHOD_TYPE_SOURCE. I would like to avoid copy the buffer.
You are guilty of premature optimization. Today’s processors copy data
really, really fast. First make it work, then check the performance to
see if you NEED to optimize.
If you are sending a lot of data, why are you using a KSMETHOD instead
of streaming it through a normal KS pin?
Not quite sure how KS_METHOD work irp-wise. does it overlapped ? If
not, can I keep the mdl probed/locked after KS_METHOD return?
Yes, the KS IRPs are always sent asynchronous. If you aren’t ready to
complete it, just have the handler return STATUS_PENDING. The AVStream
library will do the rest. When you’re ready to complete it, call
KsCompletePendingRequest.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
>>The mdl is probed and locked,so you can queue it and use it in any context.
Have a question here, shouId we map to SystemVA as well i.e. MmGetSystemAddressForMdlSafe() sill?
–WDK—
#if (NTDDI_VERSION >= NTDDI_WIN2K)
IRQL_requires_max(DISPATCH_LEVEL)
Post_writable_byte_size(Mdl->ByteCount)
At(Mdl->MappedSystemVa,
Post_writable_byte_size(Mdl->ByteCount)) // Esp:829
Check_return
Success(return != NULL)
FORCEINLINE
PVOID
MmGetSystemAddressForMdlSafe (
Inout PMDL Mdl,
In ULONG Priority // MM_PAGE_PRIORITY logically OR’d with MdlMapping*
)
//++
//
// Routine Description:
//
// This routine returns the mapped address of an MDL. If the
// Mdl is not already mapped or a system address, it is mapped.
//
// Arguments:
//
// MemoryDescriptorList - Pointer to the MDL to map.
//
// Priority - Supplies an indication as to how important it is that this
// request succeed under low available PTE conditions.
//
// Return Value:
//
// Returns the base address where the pages are mapped. The base address
// has the same offset as the virtual address in the MDL.
//
// Unlike MmGetSystemAddressForMdl, Safe guarantees that it will always
// return NULL on failure instead of bugchecking the system.
//
// This routine is not usable by WDM 1.0 drivers as 1.0 did not include
// MmMapLockedPagesSpecifyCache. The solution for WDM 1.0 drivers is to
// provide synchronization and set/reset the MDL_MAPPING_CAN_FAIL bit.
//
//–
{
if (Mdl->MdlFlags & (MDL_MAPPED_TO_SYSTEM_VA | MDL_SOURCE_IS_NONPAGED_POOL)) {
return Mdl->MappedSystemVa;
} else {
return MmMapLockedPagesSpecifyCache(Mdl, KernelMode, MmCached,
NULL, FALSE, Priority);
}
}
#endif
xxxxx@yahoo.com wrote:
>> The mdl is probed and locked,so you can queue it and use it in any context.
Have a question here, shouId we map to SystemVA as well i.e. MmGetSystemAddressForMdlSafe() sill?
Well, that depends on whether you need a system VA. Much of the time,
when you get an MDL, it has already been mapped to kernel space, so the
system VA is already present.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> xxxxx@yahoo.com wrote:
>>> The mdl is probed and locked,so you can queue it and use it in any
>>> context.
> Have a question here, shouId we map to SystemVA as well i.e.
> MmGetSystemAddressForMdlSafe() sill?Well, that depends on whether you need a system VA. Much of the time,
when you get an MDL, it has already been mapped to kernel space, so the
system VA is already present.
It wouldn’t be mapped to memory unless some higher level driver has done
an MmGetSystemAddressForMdlSafe or one of its equivalent aliases.
joe
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
NTDEV is sponsored by OSR
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>>>The mdl is probed and locked,so you can queue it and use it in any
>> context.
Have a question here, shouId we map to SystemVA as well i.e.
MmGetSystemAddressForMdlSafe() sill?
You only need to map it to a system VA if your driver needs to read the
contents of the buffers. If you are going to do DMA, you would not need
to do this; in fact, it wastes a critical resource (kernel address space)
and contributes to fragmenting the kernel address space. Also note that
if your buffer is large, and a contiguous block of kernel addresses cannot
be found to represent it, the MmGSAFMS will return NULL, and you would
feel compelled to fail the operation, but if you don’t need a system
address, the fact that you can’t map it is irrelevant, so you are not only
wasting resources, you are potentially failing an operation that should
succeed.
joe
–WDK—
#if (NTDDI_VERSION >= NTDDI_WIN2K)
IRQL_requires_max(DISPATCH_LEVEL)
Post_writable_byte_size(Mdl->ByteCount)
At(Mdl->MappedSystemVa,
Post_writable_byte_size(Mdl->ByteCount)) // Esp:829
Check_return
Success(return != NULL)
FORCEINLINE
PVOID
MmGetSystemAddressForMdlSafe (
Inout PMDL Mdl,
In ULONG Priority // MM_PAGE_PRIORITY logically OR’d with
MdlMapping*
)
//++
//
// Routine Description:
//
// This routine returns the mapped address of an MDL. If the
// Mdl is not already mapped or a system address, it is mapped.
//
// Arguments:
//
// MemoryDescriptorList - Pointer to the MDL to map.
//
// Priority - Supplies an indication as to how important it is that
this
// request succeed under low available PTE conditions.
//
// Return Value:
//
// Returns the base address where the pages are mapped. The base
address
// has the same offset as the virtual address in the MDL.
//
// Unlike MmGetSystemAddressForMdl, Safe guarantees that it will
always
// return NULL on failure instead of bugchecking the system.
//
// This routine is not usable by WDM 1.0 drivers as 1.0 did not
include
// MmMapLockedPagesSpecifyCache. The solution for WDM 1.0 drivers is
to
// provide synchronization and set/reset the MDL_MAPPING_CAN_FAIL bit.
//
//–
{
if (Mdl->MdlFlags & (MDL_MAPPED_TO_SYSTEM_VA |
MDL_SOURCE_IS_NONPAGED_POOL)) {
return Mdl->MappedSystemVa;
} else {
return MmMapLockedPagesSpecifyCache(Mdl, KernelMode, MmCached,
NULL, FALSE, Priority);
}
}
#endif
NTDEV is sponsored by OSR
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Sone years ago, I was involved in a DSP project. The data was 44.1KHz
stereo audio. We wanted to do an FFT. The goal was to ultimately change
the base representation from 16-bit integers to floats to expedite such
computations. As proof-of-concept, I converted a significant-length
buffer (about two seconds of samples) from integer to float, called the
FFT, got the array of float result, normalized it for plotting, and
converted it back to integer. All of this was so fast that I coud easily
meet the realtime window constraints to keep te FFT plot updated. This
was more than a “simple” copy,and it took no noticeable time processing
realtime audio. We never did change the base representation, because that
would have taken weeks of coding and testing, and my proof-of-concept took
about ten hours. So I, too, question attempts at premature “optimization”
based on “gut feeling” or “best practice when programming a PDP-11”. And
to te OP, “big” is a useless description, and “several” is likewise a
vague description. You cannot make design decisions based on such
nonsensical terms. “Buffers will not exceed 64K and there may be as many
as 20” could tell us a lot. Such as “you don’t need to worry about
optimizing this”. If you said “Buffers will be 50MB and there can be as
many as 500” requires a different set of decisions.
joe
Jim wrote:
> It is not a big buffer, but may have several buffers.
>
> Problem is that the buffer I am reffering to is going to arrive to an
> AVStream driver probably by a KS_METHOD with type
> KSMETHOD_TYPE_SOURCE. I would like to avoid copy the buffer.You are guilty of premature optimization. Today’s processors copy data
really, really fast. First make it work, then check the performance to
see if you NEED to optimize.If you are sending a lot of data, why are you using a KSMETHOD instead
of streaming it through a normal KS pin?> Not quite sure how KS_METHOD work irp-wise. does it overlapped ? If
> not, can I keep the mdl probed/locked after KS_METHOD return?Yes, the KS IRPs are always sent asynchronous. If you aren’t ready to
complete it, just have the handler return STATUS_PENDING. The AVStream
library will do the rest. When you’re ready to complete it, call
KsCompletePendingRequest.–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
NTDEV is sponsored by OSR
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Once the MDL is probed and locked you can map it with
MmGetSystemAddressForMdlSafe from ANY process context. However, as the rest
of the responses indicated mapping the MDL is entirely optional.
-scott
OSR
wrote in message news:xxxxx@ntdev…
>The mdl is probed and locked,so you can queue it and use it in any
>context.
Have a question here, shouId we map to SystemVA as well i.e.
MmGetSystemAddressForMdlSafe() sill?
–WDK—
#if (NTDDI_VERSION >= NTDDI_WIN2K)
IRQL_requires_max(DISPATCH_LEVEL)
Post_writable_byte_size(Mdl->ByteCount)
At(Mdl->MappedSystemVa,
Post_writable_byte_size(Mdl->ByteCount)) // Esp:829
Check_return
Success(return != NULL)
FORCEINLINE
PVOID
MmGetSystemAddressForMdlSafe (
Inout PMDL Mdl,
In ULONG Priority // MM_PAGE_PRIORITY logically OR’d with
MdlMapping*
)
//++
//
// Routine Description:
//
// This routine returns the mapped address of an MDL. If the
// Mdl is not already mapped or a system address, it is mapped.
//
// Arguments:
//
// MemoryDescriptorList - Pointer to the MDL to map.
//
// Priority - Supplies an indication as to how important it is that this
// request succeed under low available PTE conditions.
//
// Return Value:
//
// Returns the base address where the pages are mapped. The base
address
// has the same offset as the virtual address in the MDL.
//
// Unlike MmGetSystemAddressForMdl, Safe guarantees that it will always
// return NULL on failure instead of bugchecking the system.
//
// This routine is not usable by WDM 1.0 drivers as 1.0 did not include
// MmMapLockedPagesSpecifyCache. The solution for WDM 1.0 drivers is to
// provide synchronization and set/reset the MDL_MAPPING_CAN_FAIL bit.
//
//–
{
if (Mdl->MdlFlags & (MDL_MAPPED_TO_SYSTEM_VA |
MDL_SOURCE_IS_NONPAGED_POOL)) {
return Mdl->MappedSystemVa;
} else {
return MmMapLockedPagesSpecifyCache(Mdl, KernelMode, MmCached,
NULL, FALSE, Priority);
}
}
#endif
I wonder how multiprocessor-safe is the code shown above. What if calls to MmMapLockedPages will race?
The code shown is the the MmGetSystemAddressForMdlSafe macro and you’re
correct, it’s not thread safe. From the docs:
Only one thread at a time can safely call MmGetSystemAddressForMdlSafe for a
particular MDL because this routine assumes that the calling thread owns the
MDL. However, MmGetSystemAddressForMdlSafe can be called more than one time
for the same MDL either by making all calls from the same thread or, if the
calls are from multiple threads, by explicitly synchronizing the calls.
http://msdn.microsoft.com/en-us/library/windows/hardware/ff554559(v=vs.85).aspx
-scott
OSR
wrote in message news:xxxxx@ntdev…
I wonder how multiprocessor-safe is the code shown above. What if calls to
MmMapLockedPages will race?
So the right approach would be to call MmGetSystemAddressForMdlSafe once in the dispatch routine (before the IRP is queued or otherwise handed to the asynchronous handler). Then it won’t require any further synchronization.
> So the right approach would be to call MmGetSystemAddressForMdlSafe once
in the dispatch routine (before the IRP is queued or otherwise handed to
the asynchronous handler). Then it won’t require any further
synchronization.
Two considerations:
(1) it should not be called if the address is not needed, e.g. the device
does DMA
(2) It can be called in any context, such as when the IRP is dequeued, as
long as it is not called in another thread with potential concurrency
If you do it in the top-level dispatch routine, then every pending IRP
will consume a block of address space, but if you only do it when the IRP
is dequeued, then only the active IRP(s) are consuming a scarce resource.
joe
NTDEV is sponsored by OSR
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer