> The long-lived IRP folks have referred to using Direct I/O *is* effectively creating a
shared memory buffer.
The problem is that this buffer cannot outlive its IRP. If you complete an outstanding IRP every time its buffer gets filled you are effectively going to lose all the benefits of shared memory. If you allow “multi-use” of this buffer without completing its corresponding IRP, then you will have to find some means of synchronising an access to it, i.e. to do something that these folks so fiercely oppose to (according to some of them, it may take YEARS!!! to implement this part) .
Therefore, this option seems to leave these folks in a sort of limbo.
According to some other folks from this camp, sharing memory does not necessarily improve the performance, because they see some ways of writing to the userland buffer in arbitrary context (and even in atomic one) without locking its descriptor MDL in RAM.
In other words, the folks from the “SHARING MEMORY IS STUPID” camp don’t seem to sound particularly coherent, do they…
There is no such issue for whose who have worked with shared buffers and know how to work with them.
Bugcheck 76 arises when you lock a region of pages allocated by the user or by the driver on behalf of the user (ZwAllocateVirtualMemory) and fail to unlock them before the request completes or the process exits. There may be also other situations but I’ve only tested the ZwAllocateVirtualMemory case.
But, for sure, you don’t need neither EvtFileCleanup nor EvtFileClose with shared buffers.
It seems that some say no to shared memory because they just never ever tested it.
Sending audio data from kernel to user is called recording which looks like a read operation, right or wrong ?
Again, playing is done the other way around, no ? A user application sends audio data to an audio driver (kernel) which in turn sends it to an audio device. Playing audio is typically a write operation to me.
Suppose, for instance, that a file on disk contains raw audio data. Than to play this audio content, an application would just open the file along with your device, read the file content and write it to the device.
That’s how they do in UNIKS. Playback is done like this:
MmProbeAndLockPages increments EPROCESS->NumberOfLockedPages if a locked range belongs to a user space and MmUnlockPages decrements it using MDL->Process to access EPROCESS->NumberOfLockedPages.
So when a process terminates with nonzero NumberOfLockedPages the system should bugcheck. But there is a possibility that this bugcheck is enforced only for a debug kernel build OR when a DriverVerifier is active.
Remember that DPC does not only implies “resident memory”. The buffer should also be accessible in an arbitrary context. The mapping should hence be a kernel mapping. So you end up with either direct I/O or buffered I/O.
It is much safer to let the I/O manager do the job.
When pages are locked they can always be mapped in a system space shared by all processes as long as you have enough system PTEs( an array of PTEs responsible for virtual to physical mapping for a system portion of address space ) and a consecutive virtual address in the system space. This is done by a call to MmGetSystemAddressForMdlSafe ( or an underlying MmMapLockedPagesSpecifyCache ).
So you end up with either direct I/O or buffered I/O.
Shared buffer can be called direct I/O as it is essentially the same from the memory management point of view. But it can’t be called “buffered” as in Window parlance “buffered” is an I/O via an intermediate system buffer allocated from a non paged pool and the final transfer from this buffer to an application address range happens in an application process context with a special APC scheduled by IoCompleteRequest.
It is much safer to let the I/O manager do the job.
I don’t see how it can be unsafe to lock and map pages. This has been done for the last 25 years by myriads of developers.
>I don’t see how it can be unsafe to lock and map pages. This has been done for the last 25 years by myriads of developers.
Yes, as far as the job is done by the I/O manager. When you rely on EvtFileCleanup, you are not on a safe path.
I have made the test with a process that creates a child process that inherits handles. If the child exits after the parent, EvtFileCleanup runs in the context of the child. I used a kernel handle to a secondary device (control device) opened in the context of the main device’s EvtFileCreate to gain a last chance code path in the context of the parent when it exits. It works, and a kernel mode handle is safer than a user mode handle, but it is just a trick.
>That’s what the “long lived IRP” approach is, exactly.
In our classes we calling it the Big Honkin Hangin IRP (BHHI)… Same thing.
What about synchronising an access to this buffer? As we have learned on this thread, synchronisation schemes that are build around KEVENT (i.e. the lowest possible denominator in the Windows dispatcher) are supposedly “inefficient”. Certainly, if you decide that you can afford some extra overhead you can use asynch IOCTLs instead of events for notification about data arrivals, effectively simulating sigaction() mechanism in Windows, but, IIRC, according to Mr. Johnson( a.k.a “Skywing” ) “there is no advantage of using anything other than system-provided mechanisms” (why asynch IRP completion, a.k.a the “inverted call”, does not fall into this class, simply escapes me - you better ask Mr.Johnson about it).
In other words, our “Propaganda and Marketing Department” desperately needs some solutions - you’ve got to give them some notification/synchronisation mechanism that they can promote.
I guess some “revelations” are due pretty shortly-- I would not be too surprised to see statements like “BHHI is the only blessed way of doing things”, “Those idiots who use the inverted calls …etc” ( most likely expressed in capitals,of course), stories about the multi-dollar lawsuits resulting from the use of the inverted calls and all other “exciting” stuff, from our usual suspects, in not-so-distant future…
At the risk of getting yelled at, you could use a shared section object.
I’ve done it and it is quite fast. (ducking)
On Sat, Sep 3, 2016, 1:25 PM wrote:
> >That’s what the “long lived IRP” approach is, exactly. > > > In our classes we calling it the Big Honkin Hangin IRP (BHHI)… Same > thing. > > > > > > What about synchronising an access to this buffer? As we have learned on > this thread, synchronisation schemes that are build around KEVENT (i.e. the > lowest possible denominator > in the Windows dispatcher) are supposedly “inefficient”. Certainly, if > you decide that you can afford some extra overhead you can use asynch > IOCTLs instead of events for notification about data arrivals, effectively > simulating sigaction() mechanism in Windows, but, IIRC, according to > Mr. Johnson( a.k.a “Skywing” ) “there is no advantage of using anything > other than system-provided mechanisms” (why asynch IRP completion, a.k.a > the “inverted call”, does not fall into this class, simply escapes me - you > better ask Mr.Johnson about it). > > > In other words, our “Propaganda and Marketing Department” desperately > needs some solutions - you’ve got to give them some > notification/synchronisation mechanism that they can promote. > > > > I guess some “revelations” are due pretty shortly-- I would not be too > surprised to see statements like “BHHI is the only blessed way of doing > things”, “Those idiots who use the inverted calls …etc” ( most likely > expressed in capitals,of course), stories about the multi-dollar lawsuits > resulting from the use of the inverted calls and all other “exciting” > stuff, from our usual suspects, in not-so-distant future… > > > > > > Anton Bassov > > — > NTDEV is sponsored by OSR > > Visit the list online at: < > http://www.osronline.com/showlists.cfm?list=ntdev> > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
> When you rely on EvtFileCleanup, you are not on a safe path.
As I mentioned above one of the correct implementations is pending any IRP with a CancelRoutine, this allows to detect an abnormal process termination inside this CancelRoutine. Cleanup only solution might be used if a process runs with SYSTEM privileges as in that case an attacker that duplicates handles has to have SYSTEM privileges and if it has such privileges it can crash the system anyway.
I have made the test with a process that creates a child process that inherits handles.
You just prove a well known fact about handles duplication.
It works, and a kernel mode handle is safer than a user mode handle, but it is just a trick.
Kernel handles belong to a special kernel handles table and they are not connected with the process in which context they were opened. That is why they are never closed on any process termination.
>Kernel handles belong to a special kernel handles table and they are not connected with the process in which context they were opened. That is why they are never closed on any process termination.
Oh sorry about this, you are rigtht. This is a user handle.