You might want to try reading THE VERY FIRST POST in the forum. You know, the one that starts Please Read:”
Peter
Tim,
I store the below in my lookasidelist structure. yes it includes pointers.
-
pointer to the allocated buffer
-
pointer to the freed buffer
-
pooltag
-
pooltype
Doron,
I suppose returning the memory to lookasidelist means freeing the memory using ExFreeToLookasideListEx. if lookaside list owns the lifetime of memory then how does other driver access the same memory when my driver has not called Deletelookasidelist. Am I missing something here? -
the analysis doc says the other driver tries to insert the node into the lookasidelist when crash is seen. Also the node where it tries to access has references in my driver.
Few basic questions as I am a bit confused here.I googled…But did not get the answer.
- can 2 drivers share lookaside list?
- If yes what type of drivers share lookasidelist. Any examples?
- If yes how can that be done.
Can two drivers share a look aside list? Yes. But should they? I think that is a bad idea as you run into lifetime issues of the list itself and probably run into DV having problems tracking state when enabled. There are no examples of two drivers sharing a lookaside. The bigger question is why are you designing two drivers to share a lookaside? Why can’t each driver have their own lookaside ? Yes, calling ExFreeToLookasideListEx means the lookaside now owns the memory. It is an internal private decision if the list holds onto the allocation to reuse it later or frees it. That means that while the memory is freed to the lookaside, if it is kept for reuse and a driver incorrectly still references that memory, it can corrupt lookaside state. IIRC one thing that DV can do is force the lookaside to immediately free allocations when ExFreeToLookasideListEx is called. Deletelookasidelist does two things. 1) it removes the lookaside from the kernels list of lookasides. 2) it frees all allocations (waiting for reuse) still in the list. Once you call Deletelookasidelist you can’t use the lookaside anymore. You keep asking how the two drivers can share the memory. Sharing memory has nothing to do with how it’s allocated, so you may have underlying design problems in how these two drivers interact and don’t have a clear way to handle lifetime of allocations to know when they can be freed.
@Doron Can two drivers share a look aside list? Yes. …
Eh?? I had always considered lookaside lists [https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/using-lookaside-lists] as a way to preallocate a bunch of of buffers, always available in case I need some memory and the system is memory constrained without going through the hassle of making my own slab allocator. The example in the literature shows just that, when you need an SRB then there’s always going to be memory available … in fact that’s what I’ll use for some of my drivers that need to be constantly DMA’ing stuff off of hardware of some sort, because it’s always going to be there …
I’m also aware that driver allocated buffers can be pushed up and down the IRP stack so as long as the other driver(s) in the stack knows not to free the memory (which they shouldn’t anyway) then using a lookaside list for that will also work …
I had always considered, however, that the lookaside lists are driver-scope and not system-scope, that I can’t just shuffle them around like some kind of kernel level global atom table [https://docs.microsoft.com/en-us/windows/win32/dataxchg/about-atom-tables] … but are you saying that is not true, that it is a global lookaside list pool and I could use the entries like some kind of kernel level atom table, shuffling them around from driver to peer driver?
From https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/using-lookaside-lists , note the last sentence The operating system maintains state about all paged and nonpaged lookaside lists that are currently being used, dynamically tracking the demand for allocations and deallocations of entries in all lists, and available system pool for new entries. When demand for allocations is high, the operating system increases the number of entries it holds in each lookaside list. When demand falls again, it frees surplus lookaside entries back to system pool. Can a lookaside list be shared between two drivers? TECHNICALLY , yes. Is a good architectural choice? ABSOLUTELY no. I never said there was a global lookaside pool. When you initialize a lookaside list, your driver local list is inserted into private list in the kernel that tracks all active lookaside lists. Deleting the lookaside removed it from the kernels tracking list. This thread is confusing sharing memory with how it’s allocated. They are two separate topics. When sharing memory between two drivers, the method in which the buffer was allocated should be opaque. It should not matter if the buffer came from a lookaside, is a part of the device extension, etc.
Got it … good to get some clarification on that, glad that one more thing hadn’t changed while I was sleeping last night … 
Kernel memory is all the same, system-wide. The operating system has no concept of driver “ownership” of memory. When you create a lookaside list in a driver, it’s just a pointer to some memory. Your driver can hand that pointer to anyone you want to, and anyone who has the pointer can allocate lookaside buffers from that list and free them back again. It’s just memory. The PROBLEM comes when you need to shut things down, because YOU have to provide the syncronization. No one can delete the lookaside list until all the drivers are through using it. The system provides no mechanism for managing that.
Stepping back, it’s not at all clear to me that’s what you’re doing. ARE you actually sharing a lookaside list between two drivers? Your statements are contradictory. ARE you seeing this crash when you try to delete the while lookaside list? Why are you trying to delete it? Are you coordinating deleting the list with the other driver? Are you sure all of the buffers have been returned to the list before deleting it?
Or is your problem unrelated to “deleting the whole list”, and simply just some screwup with allocating and freeing buffers from the list?
When demand for allocations is high, the operating system increases the number of entries it holds in each lookaside list. When demand falls again, it frees surplus lookaside entries back to system pool.
And when demand for pool increases, it frees entries for your look-aside list arbitrarily. Thereby penalizing your driver, and canceling one good reason to use lookaside lists in the first place — which is to ensure you have memory preallocated.
But Doron knew I was going to say that, because it’s yet one more of my hot buttons.
Peter
2 Likes
@“Peter_Viscarola_(OSR)” said:
When demand for allocations is high, the operating system increases the number of entries it holds in each lookaside list. When demand falls again, it frees surplus lookaside entries back to system pool.
And when demand for pool increases, it frees entries for your look-aside list arbitrarily. Thereby penalizing your driver, and canceling one good reason to use lookaside lists in the first place — which is to ensure you have memory preallocated.
But Doron knew I was going to say that, because it’s yet one more of my hot buttons.
Peter
Eh? How does the OS determine what entry is “surplus” and gets freed? I had always considered it would be there until I told the OS to make it go away … should I write something to each buffer to mark it, like a dog on a fencepost?
All entries freed and held in the lookaside are candidates for the kernel to reclaim and truly free. Remember that private list of lookasides in the kernel? Under memory pressure, the kernel can walk that list and for each lookaside, scavenge and free entries it holds.
Ah, that makes sense … so if I think that I might sometime need up to 64 blocks of 64K each then I would create a lookaside list [ExInitializeLookasideListEx] and then call [ExAllocateFromLookasideListEx] 64 times for 64K allocations, then once that’s done as long as I don’t “free” them but instead keep my own dirty/ clean state for those buffers then I’m OK … when the device/ context unloads then I can call [ExFreeToLookasideListEx] 64 times, then [ExDeleteLookasideListEx] to finish up …
You don’t need a lookaside for this pattern. The allocator in the lookaside is no different than allocating from pool in the driver. Just allocate your 64 buffers and then free them. The intent of the lookaside is to cooperate with the kernel when it is under memory pressure and return allocations, if you never want to return them don’t use a lookaside (aka Peters issue with the lookaside pattern).
2 Likes
I think I got confused totally here.
I am not sharing any lookaside list with any driver. The question was out of curiosity. Also I got confused with the system-wide set of active lookaside lists where every driver which initialize the lookasidelist adds an entry to its tail.
My dump analysis says that
When there is a request to delete the lookaside list, the entries in the system wide list are adjusted using its prev and next pointers.
The other driver tries to delete the lookaside list created by it using nt!ExDeleteNPagedLookasideList but when adjusting the existing entries in the system wide list, it found an invalid entry resulted in the crash.
the system wide lookaside list entered an invalid state with incorrect entries due to my driver not calling the appropriate Delete function (ExDeleteLookasideListEx).
I got confused with the system wide lookaside list with my lookaside list by itself created by the driver.
So the issue is now resolved and the morale of the story is that you must delete the lookaside list before freeing the underlying memory?
Aha, so you are saying that driver #1 deletes a lookaside list, then driver #2 deletes a completely separate lookaside list, but at that time the data structures are screwed up?
((sigh)) I think I’m going to stick with my startup slab allocator … get a big lump at boot when the system memory isn’t too fragmented, pin and lock the lump then slice it up and use it as needed … LookAsideLists appears to be like a Red Velvet cake, nice to look at but oh so much trouble when you start taking bites …
1 Like
going to stick with my startup slab allocator
That’s what we do.
Lookaside lists have their place. Use them if what you want is to keep the OS running with every driver having an even chance of accessing scarce resources. That might make sense for a general purpose driver on a general purpose server or workstation machine. But in a system with a dedicated, specific, function? Nope.
It’s a laudable and highly egalitarian goal: To each according to their need, and nobody gets to accumulate limited resources to the detriment of others.
The problem, of course, is that it’s a fantasy. Every driver isn’t equally valued. Your disk driver or your file system are simply more valuable, and thus deserve more of the scarce resource, than your Bluetooth controller or your sound driver.
In the real world, in the 21st Century, where memory is cheap, the real answer is “Don’t make memory a scarce resource, and then this isn’t an issue.” We thus avoid the entire debate. I put 96GB of memory on a test machine recently. ECC memory at that…
Peter
1 Like
So what I see in dump is,
when another driver tries to delete its looksasidelist node in the system wide lookaside list using ExDeleteNPagedLookasideList it tries to check its Flink and Blink entry. when we dump Flink , it has 0 values as seen below.
what does this 0 value mean in its flink and blink entries in systemwide lookasidelist?
kd> dx -id 0,0,ffffd4869ed98200 -r1 ((nt!_LIST_ENTRY *)0xffffd4869e90a860)
((xxxxx!_LIST_ENTRY *)0xffffd4869e90a860) : 0xffffd4869e90a860 [Type: _LIST_ENTRY *]
[+0x000] Flink : 0x0 [Type: _LIST_ENTRY *]
[+0x008] Blink : 0x0 [Type: _LIST_ENTRY *]
Does this mean whoever linked it before didn’t call ExDeleteNPagedLookasideList to remove the entries from the systemwide list.
I see this in my dump
(_LIST_ENTRY*) 0xffffc1048dda68e0 : 0xffffc1048dda68e0 [Type: _LIST_ENTRY *]
[+0x000] Flink : 0xffffc1048fc40640 [Type: _LIST_ENTRY *]
[+0x008] Blink : 0xffffc1048dda6860 [Type: _LIST_ENTRY *]
Here 0xffffc1048dda68e0 is the list entry created by my driver.
After some time I see 0 in the same address.
(_LIST_ENTRY*) 0xffffc1048dda68e0 : 0xffffc1048dda68e0 [Type: _LIST_ENTRY *]
[+0x000] Flink : 0x0 [Type: _LIST_ENTRY *]
[+0x008] Blink : 0x0 [Type: _LIST_ENTRY *]
call stack says the below when a memory access breakpoint is put.
I believe this flink and blink pointers don’t point the memory location allocated by the driver thru Exallocatefromlookasidelist. It points the next node and previous node in that system wide lookaside list. Source of this corruption is because of the below calls
00 nt!memset
01 nt!IopXxxControlFile
02 nt!NtDeviceIoControlFile
03 nt!KiSystemServiceCopyEnd
04 ntdll
Any pointers?
These days, a much more important consideration than the scarcity of memory is its locality. On a system with 1 or 10 TB of RAM not all of it is equal from the point of view of the core that your thread happens to be executing on