HAL Verifier Detected Violation

OSR_Community_User · August 30, 2006, 11:05am

Hi,

I have a lower disk filter driver for w2k3 server and sometimes i get memory pool corruption during the boot time. I enable the driver verifier against all drivers in the storage stack (classpnp, disk, my driver, storport, qlogic miniport, mpio etc), then i get assertion in nt!ViMapDoubleBuffer() with the following debugging output:

* * * * * * * * HAL Verifier Detected Violation * * * * * * * *
* *
* * VF: Virtual address 85800000 is after the first MDL 84A47188
* *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

*** Verifier assertion failed ***
(B)reak, (I)gnore, (W)arn only, (R)emove assert? b

This is my first time to encounter this type of issue. I don’t know where i should start. I don’t see any unexpected behaviors in my driver. The Io that causes the assertion is sent from IoReadDiskSignature(). My driver just passes it down to the mpio pseudo. The detailed call stack is attached below.

Can anyone help me out? point me where i should start to debug this bug.

Thanks,
JT

ChildEBP RetAddr Args to Child
f7899e20 809d73fc 00000200 84a47188 f7899e60 nt!DbgBreakPoint
f7899e30 809da61a 00000000 0000001f 00622b98 nt!VfAssert+0x6d
f7899e60 809db3cc 84a4a1f0 84a47188 85800000 nt!ViMapDoubleBuffer+0x15c
f7899e98 f707d48c 867225e0 86723848 84a47188 nt!VfBuildScatterGatherList+0x11c
f7899ec8 f707bb73 86723a88 86723848 84a47188 storport!RaidDmaBuildScatterGatherList+0x2c
f7899f04 f707bbc5 86723a88 87540e00 f7899f38 storport!RaidAdapterScatterGatherExecute+0x49
f7899f14 f7082e8c 86723900 87540e88 868c7b20 storport!RaidAdapterExecuteXrb+0x21
f7899f38 f707e2d5 00000000 00000001 00000000 storport!RaUnitStartIo+0x9c
f7899f58 f708224b 868c7b20 00538eb8 00000000 storport!RaidStartIoPacket+0x49
f7899f78 f7083418 868c7a10 87538eb8 80a78be4 storport!RaidUnitSubmitRequest+0x4d
f7899f94 f707d726 868c7a10 87538eb8 f7899fd4 storport!RaUnitScsiIrp+0x90
f7899fa4 809d457d 868c7958 87538eb8 87538f8c storport!RaDriverScsiIrp+0x2a
f7899fd4 80859657 f732497e f789a02c f732497e nt!IovCallDriver+0x112
f7899fe0 f732497e 00000001 8088d90c 00000001 nt!IofCallDriver+0x13
f789a02c f7325648 8581f2c0 87538eb8 87538f8c mpio!MPIOReadWrite+0x53e
f789a1a0 f7326ec3 8581f2c0 87538eb8 00000b82 mpio!MPIOPdoHandleRequest+0xd8
f789a1dc f7322fcd 8581f2c0 87538eb8 80a78be4 mpio!MPIOPdoInternalDeviceControl+0x73
f789a290 f7322281 8581f2c0 87538eb8 00000000 mpio!MPIOPdoCommonDeviceControl+0x45d
f789a2d4 f731a5fd 8581f2c0 87538eb8 00000000 mpio!MPIOPdoDispatch+0x3a1
f789a2ec 809d457d 8581f2c0 87538eb8 87538fb0 mpio!MPIOGlobalDispatch+0x2d
f789a31c 80859657 f73751ac f789a3f0 f73751ac nt!IovCallDriver+0x112
f789a328 f73751ac 80a78be4 858037e0 f789a35b nt!IofCallDriver+0x13
f789a524 809d457d 858037e0 87538eb8 87538fd4 EmcpBase!PowerWinScsi+0x1a7
f789a554 80859657 809e5b79 f789a574 809e5b79 nt!IovCallDriver+0x112
f789a560 809e5b79 80a78be4 84a4a940 00000000 nt!IofCallDriver+0x13
f789a574 809d457d 84a4a940 87538eb8 8756eeb8 nt!ViFilterDispatchGeneric+0x2a
f789a5a4 80859657 f702fa20 f789a5c0 f702fa20 nt!IovCallDriver+0x112
f789a5b0 f702fa20 87516f08 84a487d0 f789a5f4 nt!IofCallDriver+0x13
f789a5c0 f702f635 87516f08 84a4a490 8756efdc CLASSPNP!SubmitTransferPacket+0xbb
f789a5f4 f702f712 00000000 00000200 80a78be4 CLASSPNP!ServiceTransferRequest+0x1e4
f789a618 809d457d 84a4a3d8 00000000 00000000 CLASSPNP!ClassReadWrite+0x159
f789a648 80859657 808e602c f789a678 808e602c nt!IovCallDriver+0x112
f789a654 808e602c 84a487d0 00040300 00000000 nt!IofCallDriver+0x13
f789a678 808ba5d2 84a4a3d8 00000200 00000000 nt!FstubReadSector+0x6e
f789a6a0 f74f067d 84a4a3d8 00000200 f789a6bc nt!IoReadDiskSignature+0x41
f789a6d0 f74f16dc 84a4a3d8 849a3c88 84a4a490 disk!DiskReadSignature+0x31
f789a6f8 f70393f8 84a4a3d8 00000002 87758f94 disk!DiskInitFdo+0x18a
f789a720 f7039617 84a4a3d8 80a78be4 84a4a3d8 CLASSPNP!ClassPnpStartDevice+0x1db
f789a748 809d457d 84a4a3d8 87758e70 87758fb0 CLASSPNP!ClassDispatchPnp+0x162
f789a778 80859657 809e5c61 f789a798 809e5c61 nt!IovCallDriver+0x112
f789a784 809e5c61 80a78be4 84a4a2b0 00000000 nt!IofCallDriver+0x13
f789a798 809d457d 84a4a3d8 87758e70 87758e70 nt!ViFilterDispatchPnp+0xd7
f789a7c8 80859657 f74cdc88 f789a808 f74cdc88 nt!IovCallDriver+0x112
f789a7d4 f74cdc88 80a78be4 85802020 00000000 nt!IofCallDriver+0x13
f789a808 809d457d 85802020 87758e70 87759000 PartMgr!PmPnp+0x2c2
f789a838 80859657 808f6a25 f789a870 808f6a25 nt!IovCallDriver+0x112
f789a844 808f6a25 858037e0 f789a8b0 00000000 nt!IofCallDriver+0x13
f789a870 808e78e2 85802020 f789a88c 00000000 nt!IopSynchronousCall+0xbe
f789a8b4 808e7a50 858037e0 8580f2f8 00000001 nt!IopStartDevice+0x4d
f789a8d0 808e7a15 858037e0 00000001 8580f2f8 nt!PipProcessStartPhase1+0x4e
f789ab28 808e8787 867e1a50 00000000 00000000 nt!PipProcessDevNodeTree+0x1db
f789ab5c 8080d3ea 00000003 808ae598 00000000 nt!PiProcessReenumeration+0x60
f789ab84 8080531b 00000000 00000000 80092190 nt!PipDeviceActionWorker+0x16b
f789ab9c 80a2d6d2 00000000 00000006 00000000 nt!PipRequestDeviceAction+0x118
f789ac00 80a2eddc 80088000 f789ac44 00000000 nt!IopInitializeBootDrivers+0x373
f789ac70 80a2fe21 80088000 00000000 867c1db0 nt!IoInitSystem+0x61b
f789ada0 808d121e 80088000 f789addc 8092ccff nt!Phase1InitializationDiscard+0xad0
f789adac 8092ccff 80088000 00000000 00000000 nt!Phase1Initialization+0xd
f789addc 80841a96 808d1211 80088000 00000000 nt!PspSystemThreadStartup+0x2e

Peter_Viscarola_OSR · August 30, 2006, 11:50am

Hmmmm… This assert is telling you that the VA shown extends beyond the space mapped by the MDL (that is 85800000 - the starting virtual address in the MDL >= the byte count in the MDL).

What’s the MDL Byte count? In fact, what are ALL the fields in the MDL in question (the one that’s at 0x84A47188)??

Peter
OSR

OSR_Community_User · August 30, 2006, 12:29pm

Since It’s easy to reproduce the issue, I rebooted the machine without saving the memory dump. The following is the debugging data from my new reproduction:

dt _MDL 849be218
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x849be220
+0x010 StartVa : 0x849bd000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0x538

The system always asserts when the second read is sent from disk.sys in DiskInitFdo() to the storport driver. The first read is from HALExamineMBR and the second read is from IoReadDiskSignature. They use the same Irp and Srb (Classpnp does it). The first read is completed without any issue. The MDL and SRB associated with the first IRP are:

dt _MDL 0x849be168
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x0000003f
+0x010 StartVa : 0x84a8f000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0

dt _SCSI_REQUEST_BLOCK 87516fb4
+0x000 Length : 0x40
+0x002 Function : 0 ‘’
+0x003 SrbStatus : 0 ‘’
+0x004 ScsiStatus : 0 ‘’
+0x005 PathId : 0 ‘’
+0x006 TargetId : 0 ‘’
+0x007 Lun : 0 ‘’
+0x008 QueueTag : 0 ‘’
+0x009 QueueAction : 0x20 ’ ’
+0x00a CdbLength : 0xa ‘’
+0x00b SenseInfoBufferLength : 0x12 ‘’
+0x00c SrbFlags : 0x200342
+0x010 DataTransferLength : 0x200
+0x014 TimeOutValue : 0x3c
+0x018 DataBuffer : 0x84a8f000
+0x01c SenseInfoBuffer : 0x87516fa0
+0x020 NextSrb : (null)
+0x024 OriginalRequest : 0x87538eb8
+0x028 SrbExtension : (null)
+0x02c InternalStatus : 0
+0x02c QueueSortKey : 0
+0x030 Cdb : [16] “(”
(28 00 00 00 00 00 00 00-01 00 00 00 00 00 00 00)

The MDL and SRB associated with the second IRP are:

dt _MDL 849be218
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x849be220
+0x010 StartVa : 0x849bd000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0x538

dt _SCSI_REQUEST_BLOCK 87516fb4
+0x000 Length : 0x40
+0x002 Function : 0 ‘’
+0x003 SrbStatus : 0 ‘’
+0x004 ScsiStatus : 0 ‘’
+0x005 PathId : 0 ‘’
+0x006 TargetId : 0 ‘’
+0x007 Lun : 0 ‘’
+0x008 QueueTag : 0 ‘’
+0x009 QueueAction : 0x20 ’ ’
+0x00a CdbLength : 0xa ‘’
+0x00b SenseInfoBufferLength : 0x12 ‘’
+0x00c SrbFlags : 0x200342
+0x010 DataTransferLength : 0x200
+0x014 TimeOutValue : 0x3c
+0x018 DataBuffer : 0x849bd538
+0x01c SenseInfoBuffer : 0x87516fa0
+0x020 NextSrb : (null)
+0x024 OriginalRequest : 0x87538eb8
+0x028 SrbExtension : (null)
+0x02c InternalStatus : 0
+0x02c QueueSortKey : 0
+0x030 Cdb : [16] “(”
(28 00 00 00 00 00 00 00-01 00 00 00 00 00 00 00)

OSR_Community_User · August 30, 2006, 2:19pm

A ByteCount of 0x200 and a ByteOffset of 0x538 would appear to be not a
good thing.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@emc.com
Sent: Wednesday, August 30, 2006 12:29 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] HAL Verifier Detected Violation

Since It’s easy to reproduce the issue, I rebooted the machine without
saving the memory dump. The following is the debugging data from my new
reproduction:

dt _MDL 849be218
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x849be220
+0x010 StartVa : 0x849bd000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0x538

The system always asserts when the second read is sent from disk.sys in
DiskInitFdo() to the storport driver. The first read is from
HALExamineMBR and the second read is from IoReadDiskSignature. They use
the same Irp and Srb (Classpnp does it). The first read is completed
without any issue. The MDL and SRB associated with the first IRP are:

dt _MDL 0x849be168
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x0000003f
+0x010 StartVa : 0x84a8f000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0

dt _SCSI_REQUEST_BLOCK 87516fb4
+0x000 Length : 0x40
+0x002 Function : 0 ‘’
+0x003 SrbStatus : 0 ‘’
+0x004 ScsiStatus : 0 ‘’
+0x005 PathId : 0 ‘’
+0x006 TargetId : 0 ‘’
+0x007 Lun : 0 ‘’
+0x008 QueueTag : 0 ‘’
+0x009 QueueAction : 0x20 ’ ’
+0x00a CdbLength : 0xa ‘’
+0x00b SenseInfoBufferLength : 0x12 ‘’
+0x00c SrbFlags : 0x200342
+0x010 DataTransferLength : 0x200
+0x014 TimeOutValue : 0x3c
+0x018 DataBuffer : 0x84a8f000
+0x01c SenseInfoBuffer : 0x87516fa0
+0x020 NextSrb : (null)
+0x024 OriginalRequest : 0x87538eb8
+0x028 SrbExtension : (null)
+0x02c InternalStatus : 0
+0x02c QueueSortKey : 0
+0x030 Cdb : [16] “(”
(28 00 00 00 00 00 00 00-01 00 00 00 00 00 00 00)

The MDL and SRB associated with the second IRP are:

dt _MDL 849be218
+0x000 Next : (null)
+0x004 Size : 32
+0x006 MdlFlags : 138
+0x008 Process : (null)
+0x00c MappedSystemVa : 0x849be220
+0x010 StartVa : 0x849bd000
+0x014 ByteCount : 0x200
+0x018 ByteOffset : 0x538

dt _SCSI_REQUEST_BLOCK 87516fb4
+0x000 Length : 0x40
+0x002 Function : 0 ‘’
+0x003 SrbStatus : 0 ‘’
+0x004 ScsiStatus : 0 ‘’
+0x005 PathId : 0 ‘’
+0x006 TargetId : 0 ‘’
+0x007 Lun : 0 ‘’
+0x008 QueueTag : 0 ‘’
+0x009 QueueAction : 0x20 ’ ’
+0x00a CdbLength : 0xa ‘’
+0x00b SenseInfoBufferLength : 0x12 ‘’
+0x00c SrbFlags : 0x200342
+0x010 DataTransferLength : 0x200
+0x014 TimeOutValue : 0x3c
+0x018 DataBuffer : 0x849bd538
+0x01c SenseInfoBuffer : 0x87516fa0
+0x020 NextSrb : (null)
+0x024 OriginalRequest : 0x87538eb8
+0x028 SrbExtension : (null)
+0x02c InternalStatus : 0
+0x02c QueueSortKey : 0
+0x030 Cdb : [16] “(”
(28 00 00 00 00 00 00 00-01 00 00 00 00 00 00 00)

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · August 30, 2006, 4:23pm

Thank Peter and Mark for your help. I have found the bug. It’s in my driver. I forget to re-initialize some fields in my driver internal data structure when I re-use it for the next io.

-JT