Getting BSOD with microsoft drivers when driver verifier is enabled

During our driver testing with driver verifier enabled, we found that system is crashing when verifier is enabled for our driver but no where in stack and other thread our driver is visible. Then we have enabled driver verifier for one of the Microsoft driver i.e. afd.sys, even with this driver we are facing same BSOD while restarting the machine. Issue is intermittent and machine specific. Below is the dump detail-

UNEXPECTED_KERNEL_MODE_TRAP (7f)
This means a trap occurred in kernel mode, and it's a trap of a kind
that the kernel isn't allowed to have/catch (bound trap) or that
is always instant death (double fault). The first number in the
BugCheck params is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a portion of those codes:
If kv shows a taskGate
use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
use .trap on that value
Else
.trap on the appropriate frame will show where the trap was taken
(on x86, this will be the ebp that goes with the procedure KiTrap)
Endif
kb will then show the corrected stack.
Arguments:
Arg1: 00000008, EXCEPTION_DOUBLE_FAULT
Arg2: 808fb400
Arg3: 00000000
Arg4: 00000000

Debugging Details:

Unable to load image \SystemRoot\system32\DRIVERS\vsepflt.sys, Win32 error 0n2
Unable to load image \SystemRoot\system32\DRIVERS\vnetWFP.sys, Win32 error 0n2
*** WARNING: Check Image - Checksum mismatch - Dump: 0x634e9, File: 0x61d3a - C:\ProgramData\Dbg\sym\halmacpi.dll\9DF976C76b000\halmacpi.dll

KEY_VALUES_STRING: 1

Key  : Analysis.CPU.mSec
Value: 13015

Key  : Analysis.Elapsed.mSec
Value: 49960

Key  : Analysis.IO.Other.Mb
Value: 2

Key  : Analysis.IO.Read.Mb
Value: 1

Key  : Analysis.IO.Write.Mb
Value: 22

Key  : Analysis.Init.CPU.mSec
Value: 5968

Key  : Analysis.Init.Elapsed.mSec
Value: 41583

Key  : Analysis.Memory.CommitPeak.Mb
Value: 95

Key  : Analysis.Version.DbgEng
Value: 10.0.27725.1000

Key  : Analysis.Version.Description
Value: 10.2408.27.01 amd64fre

Key  : Analysis.Version.Ext
Value: 1.2408.27.1

Key  : Bugcheck.Code.KiBugCheckData
Value: 0x7f

Key  : Bugcheck.Code.LegacyAPI
Value: 0x7f

Key  : Bugcheck.Code.TargetModel
Value: 0x7f

Key  : Failure.Bucket
Value: 0x7f_8_VRF_nt!KiUnexpectedInterruptTail

Key  : Failure.Hash
Value: {0f940c74-0e9e-edb6-9ad1-2f67a807ac88}

Key  : Hypervisor.Enlightenments.Value
Value: 8992

Key  : Hypervisor.Enlightenments.ValueHex
Value: 2320

Key  : Hypervisor.Flags.AnyHypervisorPresent
Value: 1

Key  : Hypervisor.Flags.ApicEnlightened
Value: 0

Key  : Hypervisor.Flags.ApicVirtualizationAvailable
Value: 0

Key  : Hypervisor.Flags.AsyncMemoryHint
Value: 0

Key  : Hypervisor.Flags.CoreSchedulerRequested
Value: 0

Key  : Hypervisor.Flags.CpuManager
Value: 0

Key  : Hypervisor.Flags.DeprecateAutoEoi
Value: 0

Key  : Hypervisor.Flags.DynamicCpuDisabled
Value: 0

Key  : Hypervisor.Flags.Epf
Value: 0

Key  : Hypervisor.Flags.ExtendedProcessorMasks
Value: 0

Key  : Hypervisor.Flags.HardwareMbecAvailable
Value: 0

Key  : Hypervisor.Flags.MaxBankNumber
Value: 0

Key  : Hypervisor.Flags.MemoryZeroingControl
Value: 0

Key  : Hypervisor.Flags.NoExtendedRangeFlush
Value: 1

Key  : Hypervisor.Flags.NoNonArchCoreSharing
Value: 0

Key  : Hypervisor.Flags.Phase0InitDone
Value: 1

Key  : Hypervisor.Flags.PowerSchedulerQos
Value: 0

Key  : Hypervisor.Flags.RootScheduler
Value: 0

Key  : Hypervisor.Flags.SynicAvailable
Value: 0

Key  : Hypervisor.Flags.UseQpcBias
Value: 0

Key  : Hypervisor.Flags.Value
Value: 536584

Key  : Hypervisor.Flags.ValueHex
Value: 83008

Key  : Hypervisor.Flags.VpAssistPage
Value: 1

Key  : Hypervisor.Flags.VsmAvailable
Value: 0

Key  : Hypervisor.RootFlags.AccessStats
Value: 0

Key  : Hypervisor.RootFlags.CrashdumpEnlightened
Value: 0

Key  : Hypervisor.RootFlags.CreateVirtualProcessor
Value: 0

Key  : Hypervisor.RootFlags.DisableHyperthreading
Value: 0

Key  : Hypervisor.RootFlags.HostTimelineSync
Value: 0

Key  : Hypervisor.RootFlags.HypervisorDebuggingEnabled
Value: 0

Key  : Hypervisor.RootFlags.IsHyperV
Value: 0

Key  : Hypervisor.RootFlags.LivedumpEnlightened
Value: 0

Key  : Hypervisor.RootFlags.MapDeviceInterrupt
Value: 0

Key  : Hypervisor.RootFlags.MceEnlightened
Value: 0

Key  : Hypervisor.RootFlags.Nested
Value: 0

Key  : Hypervisor.RootFlags.StartLogicalProcessor
Value: 0

Key  : Hypervisor.RootFlags.Value
Value: 0

Key  : Hypervisor.RootFlags.ValueHex
Value: 0

Key  : Stack.Pointer
Value: Invalid

Key  : WER.OS.Branch
Value: 19h1_release

Key  : WER.OS.Version
Value: 10.0.18362.1

BUGCHECK_CODE: 7f

BUGCHECK_P1: 8

BUGCHECK_P2: ffffffff808fb400

BUGCHECK_P3: 0

BUGCHECK_P4: 0

FILE_IN_CAB: MEMORY.DMP

VIRTUAL_MACHINE: VMware

FAULTING_THREAD: 92518980

TSS: 00000028 -- (.tss 0x28)
eax=00000000 ebx=000000e1 ecx=8e83f3fa edx=000000e1 esi=9555cfe0 edi=9555d1b4
eip=81f8b454 esp=9555cfe0 ebp=9555d06c iopl=0 nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010082
nt!KiUnexpectedInterruptTail+0x4d:
81f8b454 0f2906 movaps xmmword ptr [esi],xmm0 ds:0023:9555cfe0=????????????????????????????????
Resetting default scope

BLACKBOXBSD: 1 (!blackboxbsd)

BLACKBOXNTFS: 1 (!blackboxntfs)

PROCESS_NAME: services.exe

BAD_STACK_POINTER: 820b56c0

STACK_TEXT:

9555d06c 8256d44a (T) 00000001 00fffff6 000000e1 nt!KiUnexpectedInterruptTail+0x4d

9555d1a0 8256d166 (T) 00000000 80887120 00000001 hal!HalpInterruptSendIpi+0x2aa
9555d1c0 81ebc28e 00000001 9555d258 00000000 hal!HalRequestIpi+0x46
9555d26c 81ebbfbb 00000000 9555d288 00000008 nt!KiIpiSendRequest+0x2be
9555d298 81e706ce 81f190c0 00000001 9555d2c4 nt!KiIpiSendFlushAwakePacket+0x49
9555d308 81f30ed3 c054ea90 9555d570 00000000 nt!MiFlushTbList+0x1de
9555d31c 81e84224 9555d570 c0602a70 9555d570 nt!MiDeleteNonPagedPoolTail+0x21
9555d37c 81e83e44 00000000 c0602a70 9555d570 nt!MiWalkPageTablesRecursively+0x7d4
9555d3dc 81e6e185 00000001 00001000 820d7300 nt!MiWalkPageTablesRecursively+0x3f4
9555d498 81ea45ed a9d51000 00000001 40004000 nt!MiWalkPageTables+0x3b5
9555d5c0 81ea4341 40004000 00000001 9555d644 nt!MiClearNonPagedPtes+0xbf
9555d60c 81ea4233 00004000 89a0a100 00004000 nt!MmFreePoolMemory+0x9b
9555d624 81e97d9e 00004000 02000103 00000000 nt!RtlpHpFreeVA+0x2f
9555d664 81e63f68 00000051 00001000 ffffffff nt!RtlpHpSegMgrCommit+0x10a
9555d6a8 81e643e7 00000000 ffffffff 00000001 nt!RtlpHpSegPageRangeCommit+0x108
9555d710 81e63b2d 9555fca0 a9d50200 81e63b2d nt!RtlpHpSegPageRangeCoalesce+0x217
9555d760 81ea4199 0000000c 00000000 89a0a000 nt!RtlpHpSegPageRangeShrink+0xdd
9555d784 81ea40e1 00000000 fffcdfa4 0000fe3f nt!RtlpHpSegFree+0x7f
9555d7ac 8208ea23 00000000 00000004 00000004 nt!RtlpHpFreeHeap+0x89
9555d7f8 81f9d0ad a9d51db8 acada550 00000000 nt!ExpFreeHeapSpecialPool+0x232
9555d858 820ae030 9555d880 8245a442 a9d51db8 nt!ExFreeHeapPool+0x14d86d
9555d860 8245a442 a9d51db8 00000000 00000000 nt!ExFreePoolWithTag+0x10
9555d880 82452322 00000000 92f1b978 9555d89c nt!VfIoFreeIrp+0xd1
9555d890 81e53241 00000000 9555d954 8217e733 nt!IovFreeIrpPrivate+0x53
9555d89c 8217e733 a9d51db8 fa84023a 914e2dd0 nt!IoFreeIrp+0x21
9555d954 8217801c 914e2dd0 0cbfb180 92e59bf8 nt!IopParseDevice+0x5c3
9555da5c 8217a54f 00000240 89bfb180 00000000 nt!ObpLookupObjectName+0x69c
9555daf8 82156c98 9555dc7c 89bfb180 00000000 nt!ObOpenObjectByNameEx+0x1af
9555db80 82156970 9555dc7c 9555dc94 00000000 nt!IopCreateFile+0x318
9555dbc0 81f8a28b 9555dca8 00000080 9555dc7c nt!NtOpenFile+0x30

9555dbc0 81f77cd1 (T) 9555dca8 00000080 9555dc7c nt!KiSystemServicePostCall

9555dc50 8215352a (T) 9555dca8 00000080 9555dc7c nt!ZwOpenFile+0x11
9555dca0 830639c2 9555dcec 00000080 9555dd24 nt!IoGetDeviceObjectPointer+0x62
9555dd28 830630ec 959f2eb8 9180dd00 9180dd0e mountmgr!MountMgrQueryDosVolumePath+0xc2
9555dd40 81f5ab7b 83066a70 006d0030 a9d11f20 mountmgr!MountMgrDeviceControl+0xec
9555dd58 82451b39 ace116f0 9180dd00 00000000 nt!IopfCallDriver+0x3f
9555dd74 81f9de06 00000000 00000010 00000000 nt!IovCallDriver+0x1ed
9555dd88 82153452 000000c8 a67196f8 00000002 nt!IofCallDriver+0x14a4f6
9555dfdc 82155b93 914e2dd0 a67196f8 fa843b26 nt!IoVolumeDeviceToDosName+0x1aa
9555e048 82153273 00000001 a67196f8 000000c8 nt!IopQueryNameInternal+0x2f3
9555e074 821d46d1 996b7180 9555e350 fa843cb6 nt!IoQueryFileDosDeviceName+0x37
9555e7d8 81f8a28b ffffffff 0000002b 00000000 nt!NtQueryInformationProcess+0xa91

9555e7d8 81f77835 (T) ffffffff 0000002b 00000000 nt!KiSystemServicePostCall

9555e864 831c8bf1 (T) ffffffff 0000002b 00000000 nt!ZwQueryInformationProcess+0x11
WARNING: Stack unwind information not available. Following frames may be wrong.
9555e8c0 831cb837 0000029c 00000001 9a25aa20 vsepflt+0x6bf1
9555e8e0 831cbabf 0000029c acad2bac acad2b20 vsepflt+0x9837
9555e8f4 831cd31c 0000029c 00000000 00000000 vsepflt+0x9abf
9555e924 831cd4a9 acad2b20 acb02b10 acb02b10 vsepflt+0xb31c
9555e940 831ccbda acb02b10 00000003 a9cd0238 vsepflt+0xb4a9
9555e954 98971d3b acb02b10 9555ed58 831ccb8e vsepflt+0xabda
9555e970 989729dc 97143140 a9cd0238 9a2c22e8 vnetWFP+0x1d3b
9555e99c 98971f97 9555ece4 9555ed58 00000000 vnetWFP+0x29dc
9555e9d8 837138bb 9555ece4 9555ed58 00000000 vnetWFP+0x1f97
9555ea50 83713005 9555ed58 00000000 9500ff00 NETIO!ProcessCallout+0x56b
9555eb3c 83710f7a 9555ed58 9555ebd0 00000000 NETIO!ArbitrateAndEnforce+0x625
9555ec80 837ce767 00000028 9555ece4 9555ed58 NETIO!KfdClassify+0x34a
9555efd0 837d0151 838cdf34 00000000 9555f030 tcpip!WfpAlepAuthorizeOrClassifyListen+0x857
9555f084 837cfd2a 000014c2 92fb4aa0 aca662d8 tcpip!WfpAleAuthorizeListen+0x103
9555f194 837cf68a aca66324 a873beb8 a30a3ee8 tcpip!TcpCreateListenerWorkQueueRoutine+0x67c
9555f1e8 837cf4d4 9555f240 98a1ec18 903bdf78 tcpip!TcpCreateListener+0x1b2
9555f1f0 98a1ec18 903bdf78 9555f208 00000014 tcpip!TcpTlProviderListen+0x10
9555f240 98a1ee81 00000010 00000090 30c3d3eb afd!AfdTLListen+0x60
9555fa70 98a213b9 9595a5e0 97142de0 9555fab4 afd!AfdStartListen+0x253
9555fa80 81f5ab7b 97142de0 a873beb8 a873beb8 afd!AfdDispatchDeviceControl+0x59
9555fa98 82451b39 00060000 97142de0 acacaa00 nt!IopfCallDriver+0x3f
9555fab4 81f9de06 00000000 a873beb8 acacaa00 nt!IovCallDriver+0x1ed
9555facc 8217034d fa8420d6 0000057c 00c7f8c0 nt!IofCallDriver+0x14a4f6
9555fbb8 8220edea 00000000 00000000 00c7f8d8 nt!IopXxxControlFile+0x5dd
9555fbe4 81f8a28b 0000057c 00000584 00000000 nt!NtDeviceIoControlFile+0x2a

9555fbe4 77081570 (T) 0000057c 00000584 00000000 nt!KiSystemServicePostCall

00c7f8f0 00000000 (T) 00000000 00000000 00000000 0x77081570

STACK_COMMAND: .tss 0x28 ; kb

SYMBOL_NAME: nt!KiUnexpectedInterruptTail+4d

MODULE_NAME: nt

IMAGE_NAME: ntkrpamp.exe

BUCKET_ID_FUNC_OFFSET: 4d

FAILURE_BUCKET_ID: 0x7f_8_VRF_nt!KiUnexpectedInterruptTail

OS_VERSION: 10.0.18362.1

BUILDLAB_STR: 19h1_release

OSPLATFORM_TYPE: x86

OSNAME: Windows 10

FAILURE_ID_HASH: {0f940c74-0e9e-edb6-9ad1-2f67a807ac88}

Followup: MachineOwner

Please help me here. Thanks in advance.

is apparently blowing through too much stack with verifier enabled by calling into the filesystem. Talk to vmware :slight_smile:

I'm an old-timer and I get the grues when I see the word "Recursive" on a kernel stack...

1 Like

Definitely a stack overflow, though it looks to me like there's more stack being using by the inbox components than the VMware ones (should be able to confirm with kf). Not much you can do about that other than:

  1. Don't use VMware or whatever mutant feature of VMware this is (Google says vsepflt is part of VMware vShield, whatever that is)
  2. Don't use a 32-bit VM, the x64 stack is 2x larger (even better: go to ARM64 where the stack is ~2.6x larger!)
1 Like