KeDelayExecutionThread() does not work if no kernel debugger is attached

I’m trying to call KeDelayExecutionThread() from my driver to delay execution and allow my device to do some processing. However, execution only seems to delay when WinDBG is attached. Here is the code I’m using:

 // Delays for <delay> microseconds
 void wrDelayUS(uint32_t delay)
 {
     LARGE_INTEGER sleepTime;

     // KeDelayExecutionThread() uses units of 100 ns
     sleepTime.QuadPart = delay * 10;
     sleepTime.QuadPart *= -1;
     KeDelayExecutionThread(KernelMode, FALSE, &sleepTime);
 }

I call the above function like so, to try and wait for 3 seconds:

wrDelayUS(1000000);
wrDelayUS(1000000);
wrDelayUS(1000000);

This works with a kernel debugger. But without a debugger, it just completely ignores the delay and plows through. What could be causing this?

Take a look at Timer Accuracy https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/timer-accuracy which is referenced off the page for KeDelayExecutionThread. It explains that if you use a short relative expiration time such as you are doing, you can depending on the version of Windows have no wait.

Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com

I’ve read through that, but what expiration time is considered short? I just tried running it with 10 second expiration time to test it, and got the same result. I don’t think the system clock ticks slower than that. It also waits properly with a debugger attached, so I don’t think clock granularity is the issue.

Overflow? Pass in a UInt64 or cast delay to a ULONGLONG before multiplying by 10.

d

Bent from my phone


From: Synthwave
Sent: Thursday, October 4, 2018 5:39 PM
To: Doron Holan
Subject: Re: [NTDEV] KeDelayExecutionThread() does not work if no kernel debugger is attached

OSR https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.osr.com%2F&amp;data=02|01|doron.holan%40microsoft.com|f118568aeb4e4058178408d62a5afd79|72f988bf86f141af91ab2d7cd011db47|1|0|636742967621934373&amp;sdata=AvwNCe48gb2dYiVnzJrf82wNspTGGLI5DVeGFABh6Ek%3D&amp;reserved=0
Synthwave commented on KeDelayExecutionThread() does not work if no kernel debugger is attached

I’ve read through that, but what expiration time is considered short? I just tried running it with 10 second expiration time to test it, and got the same result. I don’t think the system clock ticks slower than that. It also waits properly with a debugger attached, so I don’t think clock granularity is the issue.

Update: I found the problem. Turns out the bug is in the driver installation. An old version of the driver is stuck in the operating system, and overrides any newer versions of the driver. With WinDBG attached, kdfiles temporarily replaces the old driver with an updated binary from the host machine while the debugger is running.

Background: I was trying to fix a driver bug which would disappear whenever a kernel debugger was connected. With some makeshift, hacky debugging, I narrowed the problem down to KeDelayExecutionThread() not working. The developer before me was using it incorrectly (using absolute rather than relative time). I fixed that problem and installed a fixed version of the driver, but KeDelayExecutionThread() still wasn’t working without a kernel debugger attached, which made no sense. It was at this point that I made this post.

However, it turns out that my old (unfixed) driver is stuck in the machine, and it overrides any updated versions of the driver. Uninstalling the driver via devcon.exe remove had always worked, until now. I’m not sure why the old driver is surviving devcon.exe remove, and why it’s not showing up in the device manager. When I attach a debugger, kdfiles maps the updated driver from the host machine, and successfully replaces the old driver. But only while the debugger is attached. Once I detach the debugger and restart the target machine, the old driver comes back and takes over. I plugged my device into a different test machine and KeDelayExecutionThread() works without a debugger.

I suppose a follow-up question would be why the old driver is stuck. But this is no longer a KeDelayExecutionThread() problem.

I suppose a follow-up question would be why the old driver is stuck

Since you’re giving us exactly no information on which to base our opinions… I’ll take a guess: There’s something that has a handle open to one of the driver’s Device Objects??

Peter

As a complete aside:

Also, Mr. Synthwave, please do not cross-post. From the Community Guidelines that I’m sure you read before posting, as we ask:

PLEASE DO NOT POST THE SAME QUESTION TO IN MULTIPLE CATEGORIES. Also, please do the Community members the courtesy of searching the archives before you post a question.

(your post to NTFSD has been deleted)

Peter

Maybe KeStallExecutionProcessor is what you need here. Works well for small delays and no confusing math :wink:

Maybe KeStallExecutionProcessor is what you need here. Works well for small delays and no confusing math

Well, it depends…

On one hand, the very fact that the OP uses KeDelayExecutionThread() somehow suggests that KeStallExecutionProcessor() is hardly a good option here. The thing is, the shortest delay that that KeDelayExecutionThread() may possibly achieve is measured in clock ticks, and, hence, is just HUGE by KeStallExecutionProcessor()'s standards. In fact, the time interval that KeDelayExecutionThread() 's code in itself runs before/after the thread actually goes blocking is, probably, a bit too large by KeStallExecutionProcessor()'s standards anyway. Therefore, KeStallExecutionProcessor() does not really seem to be an ideal option here, at least not at the first glance.

OTOH, a bit of extra unnecessary spinning at low IRQL, no matter how unreasonable it may seem to be at the first glance, is not really going to do you any more or less tangible harm, at least not if it happens sporadically once in a while. Assuming that it happens in context of an “ordinary” thread, the worst thing that may happen here is that the target thread simply uses up its quantum in a pointless spinning loop and gets subsequently preempted. By the time it gets scheduled on the CPU again, the requested delay is more than likely to have had already expired. Therefore, unless this “unreasonable” code executes over and over again, this option seems to be not that bad…

Anton Bassov

Maybe KeStallExecutionProcessor is what you need here.

Except he calls the function and waits for three seconds – ignoring the published guidelines that specify the max time to wait with KeStallExecutionProcessor is 50 usecs – that’s objectively speaking an awfully long time to busy-wait.

a bit of extra unnecessary spinning at low IRQL, no matter how unreasonable it may seem to be at the first glance, is not really going to do you any more or less tangible harm

Yeah, certainly right. And to be “plug compatible” with KeDelayExecutionThread it’d have to be at a fully dispatchable IRQL. But three seconds worth of such a wait? Ugh…

Peter

Pavel_A wrote:

OSR https://community.osr.com/
Pavel_A commented on KeDelayExecutionThread() does not work if no kernel debugger is attached

Maybe KeStallExecutionProcessor is what you need here. No confusing math for small delays

“Small delays”?  Did you all miss the sentence where he says he’s
delaying for 3 seconds?  Calling KeStallExecutionProcessor for 3 seconds
would not be healthy for your Windows session.

@Peter_Viscarola said:

I suppose a follow-up question would be why the old driver is stuck

Since you’re giving us exactly no information on which to base our opinions… I’ll take a guess: There’s something that has a handle open to one of the driver’s Device Objects??

I know this is a vague follow-up question. Truth is I’m still just figuring out the way WDF works (and Windows drivers in general), which is why I don’t know which information is relevant to you. You can consider this issue closed if you want, as the main bug I came for is fixed (or at least figured out). Regardless, as for the handle, could such a handle remain open through a restart of the machine? I’m going to go through the driver stores and symbolic links to manually remove it, but I’ll still need to figure it what caused this in the first place.

As a complete aside:

Also, Mr. Synthwave, please do not cross-post. From the Community Guidelines that I’m sure you read before posting, as we ask:

PLEASE DO NOT POST THE SAME QUESTION TO IN MULTIPLE CATEGORIES. Also, please do the Community members the courtesy of searching the archives before you post a question.

(your post to NTFSD has been deleted)

Yeah I know, sorry. I tried to delete my post in NTFSD (it’s not a file systems post) but there is no such functionality. I reported it for deletion and posted here. My bad!

@anton_bassov said:
Well, it depends…

On one hand, the very fact that the OP uses KeDelayExecutionThread() somehow suggests that KeStallExecutionProcessor() is hardly a good option here. The thing is, the shortest delay that that KeDelayExecutionThread() may possibly achieve is measured in clock ticks, and, hence, is just HUGE by KeStallExecutionProcessor()'s standards. In fact, the time interval that KeDelayExecutionThread() 's code in itself runs before/after the thread actually goes blocking is, probably, a bit too large by KeStallExecutionProcessor()'s standards anyway. Therefore, KeStallExecutionProcessor() does not really seem to be an ideal option here, at least not at the first glance.

OTOH, a bit of extra unnecessary spinning at low IRQL, no matter how unreasonable it may seem to be at the first glance, is not really going to do you any more or less tangible harm, at least not if it happens sporadically once in a while. Assuming that it happens in context of an “ordinary” thread, the worst thing that may happen here is that the target thread simply uses up its quantum in a pointless spinning loop and gets subsequently preempted. By the time it gets scheduled on the CPU again, the requested delay is more than likely to have had already expired. Therefore, unless this “unreasonable” code executes over and over again, this option seems to be not that bad…

Anton Bassov

I’m still playing around with the delay, and if it’s short enough I’ll try using KeStallExecutionProcessor(). Oh, and I don’t plan on actually keeping a three-second delay, don’t worry. It was just to prove that the delay indeed wasn’t working. The actual delay is shorter. I’ll still need to play around with that.

But three seconds worth of such a wait? Ugh.

If I got it right, the OP made a delay THAT large just in order to be 100% sure that KeDelayExecutionThread() actually returns on the spot, i.e. simply for the testing purposes. Let’s face it - 3 secs seems to be just a hell of a delay, by the computer’s standards. Therefore, I just cannot envision a scenario when such a delay may be practically useful (apart from intentionally annoying users, of course - some shitty DRM “protection” schemes sometimes, indeed, resort to tricks of this kind)

Anton Bassov

Synthwave wrote:

I’m still playing around with the delay, and if it’s short enough I’ll
try using KeStallExecutionProcessor(). Oh, and I don’t plan on
actually keeping a three-second delay, don’t worry. It was just to
prove that the delay indeed wasn’t working. The actual delay is
shorter. I’ll still need to play around with that.

PLEASE NOTE that KeStallExecutionProcessor is almost NEVER the right
answer.  It is implemented as a tight CPU loop.  Unless you have a piece
of hardware that causes damage if you don’t respond within a few
microseconds, you should always use KeDelayExecutionThread, which
suspends the thread and allows the CPU to do something else.

I’m going to go through the driver stores and symbolic links to manually remove it

The is almost always a very bad idea. Remove it with Device Manager or leave it alone. Resorting to manual measures is almost certain to cause problems on that machine later.

Peter