Random IRQL_NOT_LESS_OR_EQUAL

Hi everybody,

I’ve been coding a windows driver for educational purposes, trying to familiarize with the APIs, the do/don’t, etc…
In order to “test” the stability, make sure I’m not leaking memory, as I’m not disposing of a computer lab of any sort, I’m running multiple VMWare instances on my main machine with various windows versions, and wrote a script that will start my driver then stop it after a minute, over and over.
I’ve let it ran overnight and one of my machines crashed with IRQL_NOT_LESS_OR_EQUAL.
When looking into the crash dump, one of the system threads that I start has its IRQL raised to DISPATCH_LEVEL, and the crash is expected because I’m trying to acquire a mutex with KeWaitForSingleObject.
What I don’t understand, is how the IRQL was raised to DISPATCH_LEVEL, considering I’m not using any API that would raise the IRQL (or lower it).
The list of APIs i’m using is as follow:
ObfDereferenceObject
RtlRandomEx
PsLookupProcessByProcessId
ZwQuerySystemInformation
PsGetProcessId
ExAllocatePoolWithTag
ExFreePoolWithTag
KeWaitForSingleObject
KeReleaseMutex

If anyone know why the IRQL of my thread was raised to DISPATCH_LEVEL, or simply if it can be a VMWare issue, let me know!

Thanks

It’s not VMWare… It is definitely something you, or your driver, is doing.

Tell us more about your driver. WHERE are you getting this crash? At what entry point? What are you doing at the point of the crash.

Have you done !analyze -v in WinDbg?? Please (be sure your symbols are setup correctly) and post the output here.

Peter

KeWaitForSingleObject raises the IRQL to DISPATCH_LEVEL if the object you are waiting for is in paged memory you will see this.

KeWaitForSingleObject raises the IRQL to DISPATCH_LEVEL if the object you are waiting for is in paged memory you will see this.

GOOD one, Mr. @Don_Burn! Absolutely correct. Putting the Dispatcher Object in paged pool… classic rookie mistake. Good pickup.

Peter

The KMUTEX is contained into a structure that is effectively allocated in a PagedPool, then initialized with KeInitializeMutex.

Should I allocate this structure in a non paged pool to fix this issue?

Thank you for your help

It the structure is not large make it non-paged pool, otherwise consider splitting the structure so that things that need non-paged are in one struct and the rest in another with pointers between them.

Thank you both for your time, I will correct this as soon as possible

@Megapat2000 **Always** default to non-pages pool for **everything** unless you have a truly enormous structure. Even then… where you put it will depend on your needs. I guess I now have to ask: Why KMUTEX? Are you not writing this driver using WDF?? Peter

Thank you for the advice Peter. I will definitely have a look at Wdf. However, reading the doc of POOL_TYPE, it is stated that NonPagedPool is a scarce resource, whereas nothing is specified for PagedPool, explaining why I went for the PagedPool. Do you mind explaining why I should go with NonPaged instead of Paged?

KeWaitForSingleObject needs to take a spinlock to do its work, which raises IRQL to DISPATCH_LEVEL. Paging cannot occur at DISPATCH_LEVEL so you need to have the object you are waiting for be in NonPagedPool. Yes non-paged memory is a limited resource but a typical driver does not need much.

The warnings about “scarce resource” are wildly over-stated, and mostly archaic. They date back to the days when a fixed amount of the OS 32-bit address space was dedicated to use for non-paged pool (and when you ran out of it, you were screwed). These days, with 64-bit addressing and systems with tons and tons of memory, it is wrong and misleading to label non-paged pool as “scarce.”

Now, bear in mind… anything that’s non-paged is permanently pinned in memory, thereby leaving less physical memory available for other uses, and thus (perhaps some) less “flexibility” for the OS to get useful work done. The question becomes: What cost are we, as driver writers, willing to pay in order to give the OS a bit more (marginal amount of) flexibility? My answer is I’m not willing to pay much for this at all.

In kernel-mode, we quite frequently bounce-around between executions states when we are preemptible/dispatchable (and thus able to service page faults) and execution states where page faults are fatal events. This makes the calculation of “should this structure be pagable” more complex than it otherwise would be.

MY personal rule is: Everything in my device-level drivers is non-pagable, and all my pool allocations are from non-paged pool, unless there’s an obvious reason to make it otherwise. This is what I teach my students. (There’s a corollary to this rule that all locks are spin locks, but we can set that aside for now). Now, for file systems and file system mini-filters… your mileage may vary. But even there, I most opt for “non-paged” as my first choice.

Pageable kernel-mode code in 64-bit systems in the 21st century is, as a general rule, silly. Note I said as a general rule — there are exceptions. In a world where 64-bit addressing is ubiquitous and a system with about 100 logical CPUs and multi-hundred gigabytes of memory costs as about what two good quality workstations cost 20 years earlier, it is just plain foolish to stress over a few KB (or even a few MB) of physical memory use.

Make it non-pageable, and sleep well at night, not having to worry over if your code runs in some previously unanticipated state and happens to touch your page able allocations/routines.

Peter

ETA: I should write a blog post on this…

ETA: File system minifilters may have more of a case for using paged memory than the device-level drivers and filters on which my personal work is most frequently focused.

ETA: A bit of additional clarification… better writing, maybe?

In addition to Peter’s remarks, remember that Windows NT was designed at a time when physical memory was a very scarce resource. Something like 4 MB for the whole system. Today, the minimum systems requirements are something like one thousand times larger and I personally have multiple systems with around 1 million times this amount.

Physical memory, like any resource ought not to be wasted, but in engineering there are always tradeoffs. In this case, your time to correctly optimize the use of physical memory versus the availability of physical memory for other purposes. The time it takes to get that optimization right can be great, and the consequences of getting it wrong are severe, then the value of the OS being able to use it for something else has to be significant before your should attempt this. When most systems routinely have multiple GB of unused physical memory, that tradeoff doesn’t make a lot of sense