Windows 2008 R2 Issue

Recently we upgraded some of our servers to Windows 2008 R2, one of the computer crashed from time as following, another one issue is: if I leave the computer for a few days, I come back and my mouse and keyboard don’t work. Or, the right screen resolution changes to 640X480 and can’t be reset. Or, run out of system resources and have to reboot.

*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

CLOCK_WATCHDOG_TIMEOUT (101)
An expected clock interrupt was not received on a secondary processor in an
MP system within the allocated interval. This indicates that the specified
processor is hung and not processing interrupts.
Arguments:
Arg1: 0000000000000019, Clock interrupt time out interval in nominal clock ticks.
Arg2: 0000000000000000, 0.
Arg3: fffff8800205d180, The PRCB address of the hung processor.
Arg4: 0000000000000002, 0.

Debugging Details:

BUGCHECK_STR: CLOCK_WATCHDOG_TIMEOUT_8_PROC

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

PROCESS_NAME: System

CURRENT_IRQL: d

STACK_TEXT:
fffff880023b4d88 fffff80001c86443 : 0000000000000101 0000000000000019 0000000000000000 fffff8800205d180 : nt!KeBugCheckEx
fffff880023b4d90 fffff80001ce25f7 : 0000000000000000 fffff80000000002 0000000000002711 00000000000000ff : nt! ?? ::FNODOBFM::string'+0x4e3e fffff880023b4e20 fffff80001c29895 : fffff80001c4e460 fffff880023b4fd0 fffff80001c4e460 fffff80000000000 : nt!KeUpdateSystemTime+0x377 fffff880023b4f20 fffff80001cd63f3 : fffff80001e52e80 0000000000000001 fffffa800b456250 fffffa800b456102 : hal!HalpHpetClockInterrupt+0x8d fffff880023b4f50 fffff80001c992c9 : 0000000000000000 0000000000000000 0000000000000000 ffff00000135a32e : nt!KiInterruptDispatchNoLock+0x163 fffff880023b50e0 fffff80001f7064f : 0000000000000000 fffff880023b5700 0000000000000000 0000000000000000 : nt!KeFlushProcessWriteBuffers+0x65 fffff880023b5150 fffff80001fd4a06 : fffff8a0061340a8 000000000001ff58 fffff880023b52e0 0000000000000000 : nt!ExpGetProcessInformation+0x7f fffff880023b52a0 fffff80001fd5e49 : fffff8a0061340a8 fffffa806365734b 0000000000000000 fffff880023b5c20 : nt!ExpQuerySystemInformation+0xf14 fffff880023b5640 fffff80001cd9153 : fffff880023b5710 fffff80001cd80fd 0000000000000001 fffffa8009dfce70 : nt!NtQuerySystemInformation+0x4d fffff880023b5680 fffff80001cd56f0 : fffff88001826a9c 0000000000000001 fffff880023b5844 fffffa800b174270 : nt!KiSystemServiceCopyEnd+0x13 fffff880023b5818 fffff88001826a9c : 0000000000000001 fffff880023b5844 fffffa800b174270 fffff88003f87d58 : nt!KiServiceLinkage fffff880023b5820 fffff8800182656d : fffffa8009dfce70 0000000000000000 fffff80020206f49 fffff80001ce4ac1 : cng!GatherRandomKey+0x22c fffff880023b5be0 fffff80001fd3541 : 0000000000000000 fffffa8014771a00 fffffa8014771a00 fffffa8009a48b60 : cng!scavengingWorkItemRoutine+0x3d fffff880023b5c80 fffff80001ce7161 : fffff80001e7d500 fffff80001fd3504 fffffa8009a48b60 0000000000000000 : nt!IopProcessWorkItem+0x3d fffff880023b5cb0 fffff80001f7d166 : 407c407cd145d145 fffffa8009a48b60 0000000000000080 fffffa8009a3e040 : nt!ExpWorkerThread+0x111 fffff880023b5d40 fffff80001cb8486 : fffff8800213f180 fffffa8009a48b60 fffff8800214a0c0 b04cb04cbf70bf70 : nt!PspSystemThreadStartup+0x5a fffff880023b5d80 0000000000000000 : fffff880023b6000 fffff880023b0000 fffff880023b5410 00000000`00000000 : nt!KxStartSystemThread+0x16

STACK_COMMAND: kb

SYMBOL_NAME: ANALYSIS_INCONCLUSIVE

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Unknown_Module

IMAGE_NAME: Unknown_Image

DEBUG_FLR_IMAGE_TIMESTAMP: 0

FAILURE_BUCKET_ID: X64_CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

BUCKET_ID: X64_CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

Followup: MachineOwner

Anyone any suggestion?

Thanks
Eric

> CLOCK_WATCHDOG_TIMEOUT (101)

Arg3: fffff8800205d180, The PRCB address of the hung processor.

You need to get the stack trace for the stuck processor. Use “dt nt!_KPRCB
fffff8800205d180 Number” to get the processor number.

Or, the right screen resolution changes to 640X480 and can’t be reset.
Or, run out of system resources and have to reboot.

What’s the output of !vm ?


Pavel Lebedinsky/Windows Kernel Test
This posting is provided “AS IS” with no warranties, and confers no rights.

“dt nt!_KPRCB fffff8800205d180 Number” return “+0x024 Number : 2”

I dump all processors stack trace as following:

!running

System Processors: (00000000000000ff)
Idle Processors: (00000000000000da) (0000000000000000) (0000000000000000) (0000000000000000)

Prcbs Current Next
0 fffff80001e52e80 fffffa8009a48b60 fffffa800babaae0 …
2 fffff8800205d180 fffff880020680c0 fffffa80100b6b60 …
5 fffff880021b0180 fffffa800fc5eb60 …

1: kd> !cpuinfo
CP F/M/S Manufacturer MHz PRCB Signature MSR 8B Signature Features
0 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
1 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
2 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
3 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
4 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
5 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
6 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
7 6,26,5 GenuineIntel 2666 0000001100000000 211b3dfe
Cached Update Signature 0000001100000000
Initial Update Signature 0000001100000000
1: kd> !cpuid
CP F/M/S Manufacturer MHz
0 6,26,5 GenuineIntel 2666
1 6,26,5 GenuineIntel 2666
2 6,26,5 GenuineIntel 2666
3 6,26,5 GenuineIntel 2666
4 6,26,5 GenuineIntel 2666
5 6,26,5 GenuineIntel 2666
6 6,26,5 GenuineIntel 2666
7 6,26,5 GenuineIntel 2666
1: kd> 1k
Child-SP RetAddr Call Site
fffff880009d9ca0 fffff80001ce26cc nt!PoIdle+0x51f
fffff880009d9d80 0000000000000000 nt!KiIdleLoop+0x2c
1: kd> 2k
Child-SP RetAddr Call Site
fffff88002085b88 fffff80001c33401 hal!HalpHalt+0x1
fffff88002085b90 fffff80001c26380 hal!HalpResetThisProcessor+0x1dd
fffff88002085bc0 fffff80001cd63f3 hal!HalpApicRebootService+0x58
fffff88002085bf0 fffff80001ce26d2 nt!KiInterruptDispatchNoLock+0x163
fffff88002085d80 0000000000000000 nt!KiIdleLoop+0x32
1: kd> 3k
Child-SP RetAddr Call Site
fffff880020f6ca0 fffff80001ce26cc nt!PoIdle+0x51f
fffff880020f6d80 0000000000000000 nt!KiIdleLoop+0x2c
1: kd> 4k
Child-SP RetAddr Call Site
fffff88002167ca0 fffff80001ce26cc nt!PoIdle+0x51f
fffff88002167d80 0000000000000000 nt!KiIdleLoop+0x2c
1: kd> 5k
*** WARNING: Unable to verify timestamp for win32k.sys
Child-SP RetAddr Call Site
fffff8800e57ac18 fffff80001cd9153 win32k!NtUserPostMessage+0x3
fffff8800e57ac20 0000000074c7fcba nt!KiSystemServiceCopyEnd+0x13
00000000068ae778 0000000000000000 0x74c7fcba
1: kd> 6k
Child-SP RetAddr Call Site
fffff8800225bca0 fffff80001ce26cc nt!PoIdle+0x51f
fffff8800225bd80 0000000000000000 nt!KiIdleLoop+0x2c
1: kd> 7k
Child-SP RetAddr Call Site
fffff880022ccca0 fffff80001ce26cc nt!PoIdle+0x51f
fffff880022ccd80 0000000000000000 nt!KiIdleLoop+0x2c
1: kd> 0k
Child-SP RetAddr Call Site
fffff880023b4d88 fffff80001c86443 nt!KeBugCheckEx
fffff880023b4d90 fffff80001ce25f7 nt! ?? ::FNODOBFM::string'+0x4e3e fffff880023b4e20 fffff80001c29895 nt!KeUpdateSystemTime+0x377 fffff880023b4f20 fffff80001cd63f3 hal!HalpHpetClockInterrupt+0x8d fffff880023b4f50 fffff80001c992c9 nt!KiInterruptDispatchNoLock+0x163 fffff880023b50e0 fffff80001f7064f nt!KeFlushProcessWriteBuffers+0x65 fffff880023b5150 fffff80001fd4a06 nt!ExpGetProcessInformation+0x7f fffff880023b52a0 fffff80001fd5e49 nt!ExpQuerySystemInformation+0xf14 fffff880023b5640 fffff80001cd9153 nt!NtQuerySystemInformation+0x4d fffff880023b5680 fffff80001cd56f0 nt!KiSystemServiceCopyEnd+0x13 fffff880023b5818 fffff88001826a9c nt!KiServiceLinkage fffff880023b5820 fffff8800182656d cng!GatherRandomKey+0x22c fffff880023b5be0 fffff80001fd3541 cng!scavengingWorkItemRoutine+0x3d fffff880023b5c80 fffff80001ce7161 nt!IopProcessWorkItem+0x3d fffff880023b5cb0 fffff80001f7d166 nt!ExpWorkerThread+0x111 fffff880023b5d40 fffff80001cb8486 nt!PspSystemThreadStartup+0x5a fffff880023b5d80 00000000`00000000 nt!KxStartSystemThread+0x16

> 1: kd> 2k

Child-SP RetAddr Call Site
fffff88002085b88 fffff80001c33401 hal!HalpHalt+0x1
fffff88002085b90 fffff80001c26380 hal!HalpResetThisProcessor+0x1dd
fffff88002085bc0 fffff80001cd63f3 hal!HalpApicRebootService+0x58
fffff88002085bf0 fffff80001ce26d2 nt!KiInterruptDispatchNoLock+0x163
fffff88002085d80 0000000000000000 nt!KiIdleLoop+0x32

This might be a known issue that’s currently being investigated, possibly
caused by a hardware problem. Is it reproducible?


Pavel Lebedinsky/Windows Kernel Test
This posting is provided “AS IS” with no warranties, and confers no rights.

Yes, It happened once almost every day. It only happened after upgraded to windows 2008 R2,
it never happened in windows 2008 before.

> Yes, It happened once almost every day. It only happened after

upgraded to windows 2008 R2, it never happened in windows
2008 before.

Can you try disabling the use of ACPI C-states in the BIOS and see
if that helps?


Pavel Lebedinsky/Windows Kernel Test
This posting is provided “AS IS” with no warranties, and confers no rights.

After disabled ACPI C-states,It is working fine for one week till now.

Thanks Pavel.