Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.More Info on Driver Writing and Debugging
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
Easiest approach to debug a Windows 10 machine physically that has Inaccessible boot device BSOD?
Hi,
I have a physical machine that i need to debug before the Inaccessible boot device BSOD happens so i can understand what's going on.
My question is how should i approach this? I never debugged a physical machine before, let alone debugging a Inaccessible boot device BSOD which means that even disk stack is probably not initialized.
So how can i debug this kernel and find out what is causing this BSOD? Should i use KDNet debugging method and connect another machine via Ethernet and then turn on the debug mode of the target machine during boot? Is the network stack even working at that stage? And how can i generate the network key to give to windbg, when the target machine is not booting at all?
I basically just want to know what is causing this and since this is a Inaccessible boot device BSOD, no memory.dmp is generated.
Howdy, Stranger!
| Upcoming OSR Seminars | ||
|---|---|---|
| OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
| Kernel Debugging | 16-20 October 2023 | Live, Online |
| Developing Minifilters | 13-17 November 2023 | Live, Online |
| Internals & Software Drivers | 4-8 Dec 2023 | Live, Online |
| Writing WDF Drivers | 10-14 July 2023 | Live, Online |
Comments
Also i should note that when i use a Windows disk to bring up the repair mode and a command prompt and run the bcdedit /debug on, it says the system cannot find the file specified. But the bcdedit /dbgsettings net hostip:w.x.y.z port:n command works and gives me a key, but when i try to use that key on the host to connect to the target, it doesn't work and i can't connect to it. (I press F8 during boot and turn on the debugging mode)
Yes, you need to debug the target machine… using windbg and preferably via Ethernet.
In addition to /debug on, you’ll want to specify /bootdebug on. Without the /bootdebug switch, you can’t connect until system start time.
Peter Viscarola
OSR
@OSRDrivers
So i managed to attach to the target machine using KDNet with ethernet. The only suspicious thing i found was that when i brought up the command prompt in repair mode, and ran diskpart list disk, there was no * under the gpt of any disk, even tho the system is UEFI. Is this normal? If not, what does it mean?
And how should we usually pinpoint what is causing the INACCESSIBLE_BOOT_DEVICE BSOD? Looked through the upper and lower filter of disk class and no third party driver was installed. And nothing interesting on the stack of any core when BSOD happens (BSOD happens in PnpBootDeviceWait).
The first argument of the BSOD is just the ARC string of the boot disk ("\ArcName\multi..) and the second is 0xC0000034 : STATUS_OBJECT_NAME_NOT_FOUND.
Nothing was changed in the BIOS setting recently either, and no hardware change or anything.
What do !storagekd.storunit and !storagekd.storclass say?
-scott
OSR
Hi Scott, this is the output of the commands: (This is actually from a VMware based guest that had the same problem, and is not a physical machine)
1: kd> !storagekd.storunit STORPORT Units: ================== Product SCSI ID Object Extension Pnd Out Ct State -------------------------------------------------------------------------------------- NVMe VMware Vir 0 0 0 ffffe507dca020a0 ffffe507dca021f0 0 0 0 Stopped NECVMWar VMware SAT 1 0 0 ffffe507dc9a1060 ffffe507dc9a11b0 0 0 0 Stopped 1: kd> !storagekd.storclass There are no storage class devices 1: kd> !storunit ffffe507dca020a0 DO: ffffe507dca020a0 Ext: ffffe507dca021f0 Adapter: ffffe507dca031a0 Stopped Vendor: NVMe Product: VMware Virtual N SCSI ID: (0, 0, 0) Enumerated SlowLock: Free RemLock: 268435456 PageCount: 0 QueueTagList: ffffe507dca022f0 Outstanding: Head: 0000000000000000 Tail: 0000000000000000 Timeout: 0 (Ticking Down) DeviceQueue: ffffe507dca02380 Depth: 512 Status: Not Frozen PauseCount: 0 BusyCount: 0 IO Gateway: Busy Count: 0 Pause Count: 1 Requests: Outstanding: 0 Device: 0 ByPass: 0 [Device-Queued Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Bypass-Queued Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Outstanding Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Completed Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- 1: kd> !storunit ffffe507dc9a1060 DO: ffffe507dc9a1060 Ext: ffffe507dc9a11b0 Adapter: ffffe507dcb651a0 Stopped Vendor: NECVMWar Product: VMware SATA CD01 SCSI ID: (1, 0, 0) Enumerated SlowLock: Free RemLock: 268435456 PageCount: 0 QueueTagList: ffffe507dc9a12b0 Outstanding: Head: 0000000000000000 Tail: 0000000000000000 Timeout: 0 (Ticking Down) DeviceQueue: ffffe507dc9a1340 Depth: 31 Status: Not Frozen PauseCount: 0 BusyCount: 0 IO Gateway: Busy Count: 0 Pause Count: 0 Requests: Outstanding: 0 Device: 0 ByPass: 0 [Device-Queued Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Bypass-Queued Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Outstanding Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- [Completed Requests] IRP SRB Type SRB XRB Command MDL SGList Timeout ----------------------------------------------------------------------------------------------------------------------------------- 1: kd> !storadapter ffffe507dca031a0 ADAPTER DeviceObj: ffffe507dca03050 AdapterExt: ffffe507dca031a0 DriverObj: ffffe507dcae65d0 DeviceState: Working LowerDO: ffffe507dcb93df0 PhysicalDO: ffffe507dcbc2060 SlowLock: Free RemLock: -666 SystemPowerState: Working AdapterPowerState: D0 IO Model: Full Duplex Bus: 19 Slot: 0 DMA: ffffe507dc99cc80 Interrupt: 0000000000000000 Allocated ResourceList: ffffe507dc8149e0 Translated ResourceList: ffffe507dc8143a0 Gateway: Outstanding: 0 Lower: 128 High: 128 PortConfigInfo: ffffe507dca032d0 HwInit: ffffe507dc76c420 HwDeviceExt: ffffe507dca08010 (12032 bytes) SrbExt: 8352 bytes LUExt: 0 bytes Normal Logical Units: Product SCSI ID Object Extension Pnd Out Ct State --------------------------------------------------------------------------------------- NVMe VMware Vir 0 0 0 ffffe507dca020a0 ffffe507dca021f0 0 0 0 Stopped Zombie Logical Units: Product SCSI ID Object Extension Pnd Out Ct State -------------------------------------------------------------------------------------- !storloglist ffffe507dca031a0 1: kd> !storadapter ffffe507dcb651a0 ADAPTER DeviceObj: ffffe507dcb65050 AdapterExt: ffffe507dcb651a0 DriverObj: ffffe507dcae6540 DeviceState: Working LowerDO: ffffe507dcb91df0 PhysicalDO: ffffe507dcb93060 SlowLock: Free RemLock: -666 SystemPowerState: Working AdapterPowerState: D0 IO Model: Full Duplex Bus: 2 Slot: 4 DMA: ffffe507dcb548d0 Interrupt: 0000000000000000 Allocated ResourceList: ffffe507dc7ada80 Translated ResourceList: ffffe507dc7adb40 Gateway: Outstanding: 0 Lower: 31 High: 31 PortConfigInfo: ffffe507dcb652d0 HwInit: ffffe507dc76d230 HwDeviceExt: ffffe507dcb1dd10 (672 bytes) SrbExt: 10784 bytes LUExt: 0 bytes Normal Logical Units: Product SCSI ID Object Extension Pnd Out Ct State --------------------------------------------------------------------------------------- NECVMWar VMware SAT 1 0 0 ffffe507dc9a1060 ffffe507dc9a11b0 0 0 0 Stopped Zombie Logical Units: Product SCSI ID Object Extension Pnd Out Ct State -------------------------------------------------------------------------------------- !storloglist ffffe507dcb651a0OK, so the storage adapter is enumerating the LUN but disk driver failed to start for some reason. Does !devnode 0 21 say anything? And any upper or lower filters registered for disk:
!reg querykey \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\Class{4d36e967-e325-11ce-bfc1-08002be10318}
-scott
OSR
So how did you find out this out (That storage adapter is enumerating the LUN but the disk driver failed to start) ? I'm asking this because i want to learn what do the storage experts look for in the output of these commands in these situation? Because there are a lot of stuff that i don't get in the output of these commands.
This is the output of the command that you asked
1: kd> !devnode 0 21 Dumping IopRootDeviceNode (= 0xffffe507dc1139e0) DevNode 0xffffe507dc8e0340 for PDO 0xffffe507dcb95060 InstancePath is "PCI\VEN_8086&DEV_10D3&SUBSYS_07D015AD&REV_00\000C29FFFFC8AC8E00" ServiceName is "e1i65x64" State = DeviceNodeRemoved (0x312) Previous State = DeviceNodeInitialized (0x302) Problem = CM_PROB_USED_BY_DEBUGGER Problem Status = 0x00000000 1: kd> !devnode 0 2 Dumping IopRootDeviceNode (= 0xffffe507dc1139e0) DevNode 0xffffe507dc1139e0 for PDO 0xffffe507dc106d60 Parent 0000000000 Sibling 0000000000 Child 0xffffe507dc0ba9e0 InstancePath is "HTREE\ROOT\0" State = DeviceNodeStarted (0x308) Previous State = DeviceNodeEnumerateCompletion (0x30d) StateHistory[06] = DeviceNodeEnumerateCompletion (0x30d) StateHistory[05] = DeviceNodeEnumeratePending (0x30c) StateHistory[04] = DeviceNodeStarted (0x308) StateHistory[03] = DeviceNodeEnumerateCompletion (0x30d) StateHistory[02] = DeviceNodeEnumeratePending (0x30c) StateHistory[01] = DeviceNodeStarted (0x308) StateHistory[00] = DeviceNodeUninitialized (0x301) StateHistory[19] = Unknown State (0x0) StateHistory[18] = Unknown State (0x0) StateHistory[17] = Unknown State (0x0) StateHistory[16] = Unknown State (0x0) StateHistory[15] = Unknown State (0x0) StateHistory[14] = Unknown State (0x0) StateHistory[13] = Unknown State (0x0) StateHistory[12] = Unknown State (0x0) StateHistory[11] = Unknown State (0x0) StateHistory[10] = Unknown State (0x0) StateHistory[09] = Unknown State (0x0) StateHistory[08] = Unknown State (0x0) StateHistory[07] = Unknown State (0x0) Flags (0x00000131) DNF_MADEUP, DNF_ENUMERATED, DNF_IDS_QUERIED, DNF_NO_RESOURCE_REQUIRED UserFlags (0x0000000a) DNUF_DONT_SHOW_IN_UI, DNUF_NOT_DISABLEABLE CapabilityFlags (0x000001c0) UniqueID, SilentInstall, RawDeviceOK DisableableDepends = 3 (including self) 1: kd> !reg querykey \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e967-e325-11ce-bfc1-08002be10318} Sorry <\REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e967-e325-11ce-bfc1-08002be10318}> is not cached =========================================================================================== Falling back to traversing the tree of nodes. Hive ffff87089660c000 KeyNode fffff8011752967c [SubKeyAddr] [SubKeyName] fffff801175299bc 0000 fffff80117fefdac Configuration fffff80117ff00cc Properties Use '!reg keyinfo ffff87089660c000 <SubKeyAddr>' to dump the subkey details [ValueType] [ValueName] [ValueData] REG_SZ Class DiskDrive REG_SZ ClassDesc @c_diskdrive.inf,%ClassDesc%;Disk drives REG_MULTI_SZ IconPath %SystemRoot%\System32\setupapi.dll,-53\0 REG_MULTI_SZ UpperFilters partmgr\0 REG_SZ EnumPropPages32 storprop.dll,DiskPropPageProvider REG_SZ NoInstallClass 1 REG_SZ SilentInstall 1 REG_MULTI_SZ LowerFilters EhStorClass\0The storage adapter enumerates the bus and creates a PDO for each storage device it finds. StorPort calls these "units", so !storunit shows you things the storage adapter found:
3: kd> !storunit STORPORT Units: ================== Product SCSI ID Object Extension Pnd Out Ct State -------------------------------------------------------------------------------------- 0 0 0 ffffbf83a8344050 ffffbf83a83441a0 0 0 0 Working 1 0 0 ffffbf83a8321050 ffffbf83a83211a0 0 0 0 WorkingThe function driver for a disk is going to be the Disk Class Driver. It gets notified of the arrival of the disk and then creates an FDO for the disk device. This is what you'll see in !storclass:
You can see the relation between the unit PDO and disk FDO with !devstack:
Two other things I can think of:
3: kd> !wmitrace.strdump (WmiTrace) StrDump Generic LoggerContext Array @ 0xFFFFBF83A5D42C40 [64 Elements] ... Logger Id 0x09 @ 0xFFFFBF83A5D7C040 Named 'EventLog-System' ... 3: kd> !logdump 9 (WmiTrace) LogDump for Logger Id 0x09 Found Buffers: 2 Messages: 23, sorting entries-scott
OSR
Thank you for the detailed answer Scott,
This is the output i get when i dumped the system even log:
!logdump 0x0a (WmiTrace) LogDump for Logger Id 0x0a Found Buffers: 2 Messages: 17, sorting entries [0]0004.0008:: 133056199158101700 [Microsoft-Windows-Kernel-General//Info ]The operating system started at system time ?2022?-?08?-?22T05:31:55.500000000Z. [0]0004.0008:: 133056199158102383 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 153)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158102790 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 208)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103265 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 20)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103282 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 21)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103664 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 25)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103668 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 27)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103741 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 208)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158103747 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 26)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158104090 [({15ca44ff-4d7a-4baa-bba5-0998955e531e}, 32)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199158141778 [({a68ca8b7-004f-d7b6-a698-07e2de0f1f5d}, 20)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.006C:: 133056199291695877 [({2d9f3a42-01d4-4733-97f7-041e8021dc84}, 0)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199292264430 [({0bf2fb94-7b60-4b4d-9766-e82f658df540}, 3)]Event metadata not found; if you have the manifest, you may load it with the -man switch [0]0004.0008:: 133056199295334661 [Microsoft-Windows-FilterManager//]File System Filter 'FileInfo' (10.0, ?1989?-?07?-?13T05:45:12.000000000Z) has successfully loaded and registered with Filter Manager. [0]0004.0008:: 133056199295349590 [Microsoft-Windows-FilterManager//]File System Filter 'Wof' (10.0, ?2096?-?10?-?27T17:54:16.000000000Z) has successfully loaded and registered with Filter Manager. [0]0004.0008:: 133056199295367098 [Microsoft-Windows-FilterManager//]File System Filter 'WdFilter' (10.0, ?2073?-?10?-?13T13:52:50.000000000Z) has successfully loaded and registered with Filter Manager. [0]0004.0008:: 133056199295573114 [({3ff37a1c-a68d-4d6e-8c9b-f79e8b16c482}, 100)]Event metadata not found; if you have the manifest, you may load it with the -man switchSo unfortunately it seems like there is not much useful information in it, and i couldn't find any useful information regarding the "Event metadata not found" error in the log by googling, only one OSR thread without any answer.
And disk!DiskAddDevice never gets called when i put a breakpoint on it (i put a bp on it with very early with the help of initial break cycle), although its DriverEntry does get called so at least it gets loaded.
Very mysterious...Any chance you can put the dump somewhere that I can take a look? Not sure what I'm looking for yet but it's a strange one.
Also: The GUIDs are the providers but their manifests aren't registered for some reason. You can use logman to see if the provider is registered on your host:
logman query providers {15ca44ff-4d7a-4baa-bba5-0998955e531e} Provider GUID ------------------------------------------------------------------------------- Microsoft-Windows-Kernel-Boot {15CA44FF-4D7A-4BAA-BBA5-0998955E531E}You can extract the manifest with the PerfView utility:
PerfView userCommand DumpRegisteredManifest {15CA44FF-4D7A-4BAA-BBA5-0998955E531E}https://github.com/microsoft/perfview/releases/tag/v3.0.4
That being said, it doesn't look like any of those messages are interesting...
-scott
OSR
Unfortunately we are not allowed to share the dump files as it might contain customer data.
Can this happen because of a corrupted GPT partition? Note that when i boot the machine using a LIVE windows disk, the boot partition and its files/folders does get detected without any problems.
Also could this be happening because of a UEFI bootkit? Any suggestion on what other commands i should try out?
Have you checked your IRP_MN_START routine? have you handled it properly? I suspect this is causing the inaccessible boot device error.
That can't be the issue because currently we do not have any filter driver registered in this machine, you can check the
!reg querykeyoutput that i shared above.