DeadLock vs WedLock

Prokash_Sinha-1 · August 15, 2004, 10:56am

Recently I was asked to present a small algorithm for x86 spinlock (using
asm), and I can do it at home.
If you happen to know please keep quite:). And I dont do this kind of stuff
every day or every week
or even every month. Defintely wish to do it, analyze it etc.etc but very
few people are that
(un)lucky to have a chance doing it. SO I WAS BUSY SEARCHING MY JUNKS AND
WEB TO FINDOUT BIT
MORE THAN WHAT I VAGUELY LEARNED IN ADVANCE OPERATING SYSTEMS COURSE. Yes in
my case most of
computer science course started with a prefix ADVANCE basically due to two
reasons: 1) Actual
course work I did was after several years of programming work so graduate
advisor was trying
to accelarate the process so that I can earn decent living again 2) I wanted
to be on a weight
loss program, since starting with Bob Tarjan’s Data structure book of SIAM
series was sure a
terse approach ( 70 or 80 pages of precise and accurate writing,
a sentence skipped => a topic skipped).

As I was searching, I bumped into blogs.msdn.com/cbrumme and others - just
deligtful ! Highly
recommonded reading for any systems programmer.

And as Murphy’s law is active and well, I was handed out a real WedLock
problem to analyze.

The problem is -

There is a WDM driver sitting inside the storage stack a filter below disk
class. The device
object it defines a DISK_*, so I would think it is a disk driver ( need to
quickly look at
this ramifications latter ), and it attaches using
IoAttachDeviceToDeviceStack(…). But it
can be thought of a virtual bus driver ( I need to verify on this …). The
driver only
demand starts.

The deadlock is on dual-processor, and hyperthread machines with Windows 2k,
but does not
occur on XP w/sp1 for the same machines… And it is when the machine starts
to wakeup after
standby mode. STARTED LOOKING AT THE LOCKS BUT DID NOT GET VERY FAR TO
SPITOUT RELATED INFO
FOR HELP, mainly because I need to understand little bit more about the
problem. The deadlock
happens when sbp2port driver tries to Acquire a spin lock using
KeAcquireSpinLockAtDpcLevel()
that is a NOP on uniprocessor ( no wonder the bug was not caught there !).
It is clear that
the POWER UP path of the driver is leaving something out of synch.

BUT THE POINT THE WRITER WAS MAKING IS THAT THERE IS A BIT IN 8XXXXXXX,
(surely in the kernel)
if set then this happens, and if not set it does not. Since the stacks on
both ~n does not
show this driver, it is highly likely that it is a MS bug. BOY BOY BOY, I
WISH IT WAS THAT
EASY TO FIND MS BUGS AROUND THAT AREA. I was almost going to say no, it is
bug created by
Hilbert, and one of the soviet scientist named “Kaichian” or something who
proved it long
time back so what do we care, ship the product with a KB that we just found
a bug of those
fellows, but sure one can’t make someone that wishful !!!

Without the driver in the machine, everything is fine, whatelse someone need
to justify that
as t -> inifinity, P( it is my bug) -> 1.

For design/architectural review -

I always try to get the managers out of the crowd.
Always tell the owner, you are the boss, drive it in a way so that you
can use others help
Dont try to hide, the machines are not that intelligent to obey your
request to hide.

In case I get stuck in the analysis, I would spitout the stacks, locks their
contentions etc
for real help. For now I’m trying to prun the output of very verbose locks
command output…

-pro

OSR_Community_User · August 17, 2004, 4:10pm

I FINALLY GOT AROUND TO ANALYZE THIS (RANDOM ) DEADLOCK(S). I captured
two scenairo ( but they are identical,
Interms of analysis so I’m putting up just one.

The system is windows 2000 spx ( not exactly sure, will check soon ).

At the very last step I’m stuck since there is no thread to look for
(7fffffff )

Would appreciate if anyone sheds some light.

It happens when we try to let the standy and wakeup happens one or two
times.

I will try to catch some more instances to make fairly sure that the
cause is basically same, hence the analysis is somewhat like what I
spitted out below.

-pro

0: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks…

Resource @ nt!IopDeviceTreeLock (0x8054e380) Shared 1 owning threads
<<<<– (1)
Contention Count = 1
Threads: 820a6da0-02<*>
KD: Scanning for held locks…

Resource @ 0x81bad7c0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0
KD: Scanning for held locks.

Resource @ 0x81af39c0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0
KD: Scanning for held locks…

Resource @ 0x820a0bc0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0
KD: Scanning for held locks.
1528 total locks, 4 locks currently held

0: kd> !strct eresource 0x8054e380 <<<— (1)
struct _ERESOURCE (sizeof=56)
+00 struct _LIST_ENTRY SystemResourcesList
+00 struct _LIST_ENTRY *Flink = 820C6AD8
+04 struct _LIST_ENTRY *Blink = 8209E120
+08 struct _OWNER_ENTRY *OwnerTable = 81F3A348
+0c int16 ActiveCount = 0001
+0e uint16 Flag = 0000
+10 struct _KSEMAPHORE *SharedWaiters = 81B1A668 <<<— (2)
+14 struct _KEVENT *ExclusiveWaiters = 00000000
+18 struct _OWNER_ENTRY OwnerThreads[2]
+18 OwnerThreads[0]
+18 uint32 OwnerThread = 00000000
+1c int32 OwnerCount = 00000000
+1c uint32 TableSize = 00000000
+20 OwnerThreads[1]
+20 uint32 OwnerThread = 820a6da0
+24 int32 OwnerCount = 00000002
+24 uint32 TableSize = 00000002
+28 uint32 ContentionCount = 00000001
+2c uint16 NumberOfSharedWaiters = 0000
+2e uint16 NumberOfExclusiveWaiters = 0000
+30 void *Address = 00000000
+30 uint32 CreatorBackTraceIndex = 00000000
+34 uint32 SpinLock = 00000000

0: kd> !strct KSEMAPHORE 81B1A668 <<<— (2)
struct _KSEMAPHORE (sizeof=20)
+00 struct _DISPATCHER_HEADER Header
+00 byte Type = 05
.
+01 byte Absolute = 00
.
+02 byte Size = 05
.
+03 byte Inserted = 00
.
+04 int32 SignalState = 00000000
+08 struct _LIST_ENTRY WaitListHead <<<<---- (*)
+08 struct _LIST_ENTRY *Flink = 81B1A670 <<<<—(3)
+0c struct _LIST_ENTRY *Blink = 81B1A670
+10 int32 Limit = 7fffffff

0: kd> !list 81B1A668+8 <<<<— (3)
struct _LIST_ENTRY (sizeof=8)
+0 struct _LIST_ENTRY *Flink = 81B1A670
+4 struct _LIST_ENTRY *Blink = 81B1A670

0: kd> dl 81B1A668+8 <<<<---- (3)
81b1a670 81b1a670 81b1a670 7fffffff 84000003

0: kd> !strct KWAIT_BLOCK 81B1A670
struct _KWAIT_BLOCK (sizeof=24)
+00 struct _LIST_ENTRY WaitListEntry
+00 struct _LIST_ENTRY *Flink = 81B1A670
+04 struct _LIST_ENTRY *Blink = 81B1A670
+08 struct _KTHREAD *Thread = 7FFFFFFF <<<<—<<<—
(???)
+0c void *Object = 84000003
+10 struct _KWAIT_BLOCK *NextWaitBlock = 02018001
+14 uint16 WaitKey = 744e
+16 uint16 WaitType = 6e66

0: kd> !strct eresource 0x81bad7c0
struct _ERESOURCE (sizeof=56)
+00 struct _LIST_ENTRY SystemResourcesList
+00 struct _LIST_ENTRY *Flink = 81AE5E9C
+04 struct _LIST_ENTRY *Blink = 81B1FD3C
+08 struct _OWNER_ENTRY *OwnerTable = 00000000
+0c int16 ActiveCount = 0001
+0e uint16 Flag = 0008
+10 struct _KSEMAPHORE *SharedWaiters = 00000000 <<<— Noone
waiting
+14 struct _KEVENT *ExclusiveWaiters = 00000000
+18 struct _OWNER_ENTRY OwnerThreads[2]
+18 OwnerThreads[0]
+18 uint32 OwnerThread = 00000000
+1c int32 OwnerCount = 00000000
+1c uint32 TableSize = 00000000
+20 OwnerThreads[1]
+20 uint32 OwnerThread = 820a76a3
+24 int32 OwnerCount = 00000001
+24 uint32 TableSize = 00000001
+28 uint32 ContentionCount = 00000000
+2c uint16 NumberOfSharedWaiters = 0000
+2e uint16 NumberOfExclusiveWaiters = 0000
+30 void *Address = 00000000
+30 uint32 CreatorBackTraceIndex = 00000000
+34 uint32 SpinLock = 00000000

0: kd> !strct eresource 0x81af39c0
struct _ERESOURCE (sizeof=56)
+00 struct _LIST_ENTRY SystemResourcesList
+00 struct _LIST_ENTRY *Flink = 8206DFBC
+04 struct _LIST_ENTRY *Blink = 81AE76DC
+08 struct _OWNER_ENTRY *OwnerTable = 00000000
+0c int16 ActiveCount = 0001
+0e uint16 Flag = 0008
+10 struct _KSEMAPHORE *SharedWaiters = 00000000 <<<— Noone
waiting
+14 struct _KEVENT *ExclusiveWaiters = 00000000
+18 struct _OWNER_ENTRY OwnerThreads[2]
+18 OwnerThreads[0]
+18 uint32 OwnerThread = 00000000
+1c int32 OwnerCount = 00000000
+1c uint32 TableSize = 00000000
+20 OwnerThreads[1]
+20 uint32 OwnerThread = 820a76a3
+24 int32 OwnerCount = 00000001
+24 uint32 TableSize = 00000001
+28 uint32 ContentionCount = 00000000
+2c uint16 NumberOfSharedWaiters = 0000
+2e uint16 NumberOfExclusiveWaiters = 0000
+30 void *Address = 00000000
+30 uint32 CreatorBackTraceIndex = 00000000
+34 uint32 SpinLock = 00000000

0: kd> !strct eresource 0x820a0bc0
struct _ERESOURCE (sizeof=56)
+00 struct _LIST_ENTRY SystemResourcesList
+00 struct _LIST_ENTRY *Flink = 81AF7240
+04 struct _LIST_ENTRY *Blink = 81B01C40
+08 struct _OWNER_ENTRY *OwnerTable = 00000000
+0c int16 ActiveCount = 0001
+0e uint16 Flag = 0008
+10 struct _KSEMAPHORE *SharedWaiters = 00000000 <<<— Noone
waiting on this
+14 struct _KEVENT *ExclusiveWaiters = 00000000
+18 struct _OWNER_ENTRY OwnerThreads[2]
+18 OwnerThreads[0]
+18 uint32 OwnerThread = 00000000
+1c int32 OwnerCount = 00000000
+1c uint32 TableSize = 00000000
+20 OwnerThreads[1]
+20 uint32 OwnerThread = 820a76a3
+24 int32 OwnerCount = 00000001
+24 uint32 TableSize = 00000001
+28 uint32 ContentionCount = 00000000
+2c uint16 NumberOfSharedWaiters = 0000
+2e uint16 NumberOfExclusiveWaiters = 0000
+30 void *Address = 00000000
+30 uint32 CreatorBackTraceIndex = 00000000
+34 uint32 SpinLock = 00000000

0: kd> !locks -v
**** DUMP OF ALL RESOURCE OBJECTS ****

Resource @ nt!MmSystemWsLock (0x80559020) Available
Contention Count = 24

Resource @ nt!IopDeviceTreeLock (0x8054e380) Shared 1 owning threads
Contention Count = 1
Threads: 820a6da0-02<*>

THREAD 820a6da0 Cid 8.24 Teb: 00000000 Win32Thread: 00000000
WAIT: (Suspended) KernelMode Non-Alertable
eb843a54 NotificationEvent
IRP List:
81abd848: (0006,0100) Flags: 00000000 Mdl: 00000000
Not impersonating
Owning Process 820ae5e0
Wait Start TickCount 8964 Elapsed Ticks: 1282
Context Switch Count 256
UserTime 0:00:00.0000
KernelTime 0:00:00.0078
Start Address nt!ExpWorkerThread (0x804eb3d2)
Stack Init eb844000 Current eb8439c8 Base eb844000 Limit eb841000
Call 0
Priority 12 BasePriority 12 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child
eb8439e0 80500321 81abcbe8 eb843abc 00000000 nt!KiSwapThread+0x1b1
eb843a08 eb4416d9 eb843a54 00000005 00000000
nt!KeWaitForSingleObject+0x1a3
eb843a94 eb440cd6 81f383f0 eb843abc 81b16dc8 +0xeb4416d9
eb843b14 eb692465 81f383f0 81f383f0 81f384a8 +0xeb440cd6
00000000 00000000 00000000 00000000 00000000 +0xeb692465

Resource @ 0x81b1fd3c Available

Resource @ 0x81bad7c0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0

THREAD 820a76a0 Cid 8.14 Teb: 00000000 Win32Thread: 00000000
RUNNING
Not impersonating
Owning Process 820ae5e0
Wait Start TickCount 8964 Elapsed Ticks: 1282
Context Switch Count 609
UserTime 0:00:00.0000
KernelTime 0:00:20.0031
Start Address nt!ExpWorkerThread (0x804eb3d2)
Stack Init eb834000 Current eb833d34 Base eb834000 Limit eb831000
Call 0
Priority 13 BasePriority 13 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child
eb833cbc 8053e236 00000001 00000002 000000d1
nt!RtlpBreakWithStatusInstruction
eb833cbc 80539782 00000001 00000002 000000d1
nt!KeUpdateSystemTime+0x14e
eb833d68 804f4357 8201c030 81b16000 8054b440
nt!KefAcquireSpinLockAtDpcLevel+0x12
eb833d78 804eb481 81ae0f08 00000000 00000000
nt!IopProcessWorkItem+0xf
eb833da8 80529a82 81ae0f08 00000000 00000000
nt!ExpWorkerThread+0xaf
eb833ddc 8053f012 804eb3d2 00000000 00000000
nt!PspSystemThreadStartup+0x54
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

Resource @ 0x81af39c0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0

THREAD 820a76a0 Cid 8.14 Teb: 00000000 Win32Thread: 00000000
RUNNING
Not impersonating
Owning Process 820ae5e0
Wait Start TickCount 8964 Elapsed Ticks: 1282
Context Switch Count 609
UserTime 0:00:00.0000
KernelTime 0:00:20.0031
Start Address nt!ExpWorkerThread (0x804eb3d2)
Stack Init eb834000 Current eb833d34 Base eb834000 Limit eb831000
Call 0
Priority 13 BasePriority 13 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child
eb833cbc 8053e236 00000001 00000002 000000d1
nt!RtlpBreakWithStatusInstruction
eb833cbc 80539782 00000001 00000002 000000d1
nt!KeUpdateSystemTime+0x14e
eb833d68 804f4357 8201c030 81b16000 8054b440
nt!KefAcquireSpinLockAtDpcLevel+0x12
eb833d78 804eb481 81ae0f08 00000000 00000000
nt!IopProcessWorkItem+0xf
eb833da8 80529a82 81ae0f08 00000000 00000000
nt!ExpWorkerThread+0xaf
eb833ddc 8053f012 804eb3d2 00000000 00000000
nt!PspSystemThreadStartup+0x54
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

Resource @ 0x820a0bc0 Shared 1 owning threads
Threads: 820a76a3-01<*> *** Actual Thread 820A76A0

THREAD 820a76a0 Cid 8.14 Teb: 00000000 Win32Thread: 00000000
RUNNING
Not impersonating
Owning Process 820ae5e0
Wait Start TickCount 8964 Elapsed Ticks: 1282
Context Switch Count 609
UserTime 0:00:00.0000
KernelTime 0:00:20.0031
Start Address nt!ExpWorkerThread (0x804eb3d2)
Stack Init eb834000 Current eb833d34 Base eb834000 Limit eb831000
Call 0
Priority 13 BasePriority 13 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child
eb833cbc 8053e236 00000001 00000002 000000d1
nt!RtlpBreakWithStatusInstruction
eb833cbc 80539782 00000001 00000002 000000d1
nt!KeUpdateSystemTime+0x14e
eb833d68 804f4357 8201c030 81b16000 8054b440
nt!KefAcquireSpinLockAtDpcLevel+0x12
eb833d78 804eb481 81ae0f08 00000000 00000000
nt!IopProcessWorkItem+0xf
eb833da8 80529a82 81ae0f08 00000000 00000000
nt!ExpWorkerThread+0xaf
eb833ddc 8053f012 804eb3d2 00000000 00000000
nt!PspSystemThreadStartup+0x54
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

Resource @ 0x81af7240 Available

Resource @ 0x81af0bc0 Available
1528 total locks, 4 locks currently held

Prokash_Sinha-1 · August 17, 2004, 7:20pm

I just finished 3 different (apprently ) analysis, all leads me to the same
thing, basically the targetting thread is non-existing !!!. It is win2k with sp4. If anyone has any other trick to attack it, or shed some light, pls do so…
I’m out of possibilities for a moment, so will try to look thru the Power Mangement path of the PDO handling …

-pro

Prokash_Sinha-1 · August 18, 2004, 12:36am

Finally another round of tests to capture a relatively better hang, got me
into some sensable call stacks that I need to
analyze. Now at least I get the driver on the stack ( note that I dont know
much about the driver architecture/design, that I have to pick as I go
along ). In this particular instance, I see one resouce that has contention,
and it is owned ( not available). It has exclusive and shared waiters, but
both of them leading me to bogus threads…

At least now I could analyze bit more. I’m sure it is going to be long rocky
ride for me, but once I’m over with (hopefully), I would try to put up the
things I tried, including turning XP’s verifier for deadlock detection ( and
quite a bit more :). At least I’m lucky that I could reproduce at will, and
before a month go by, I will have better understanding, BUT IMAGINE IF
SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN CUSTOMERS’ SITE. And it
is a rare
event, then the complexity of this kind of problem goes thru the roof :).
Hence derivational logic might become the only way out, any derivational
logic has lot of underlying (hidden) assumptions often we call ‘internal
workings’ or ‘what happens when …’. Rest depends on individuals …

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
Society Prokash Sinha
Sent: Tuesday, August 17, 2004 4:19 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] DeadLock vs WedLock

I just finished 3 different (apprently ) analysis, all leads me to the same
thing, basically the targetting thread is non-existing !!!. It is win2k with
sp4. If anyone has any other trick to attack it, or shed some light, pls do
so…
I’m out of possibilities for a moment, so will try to look thru the Power
Mangement path of the PDO handling …

-pro

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Maxim_S_Shatskih · August 18, 2004, 10:56am

!process 0 7 is the common way of solving deadlocks.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Prokash Sinha”
To: “Windows System Software Devs Interest List”
Sent: Wednesday, August 18, 2004 8:34 AM
Subject: RE: [ntdev] DeadLock vs WedLock

> Finally another round of tests to capture a relatively better hang, got me
> into some sensable call stacks that I need to
> analyze. Now at least I get the driver on the stack ( note that I dont know
> much about the driver architecture/design, that I have to pick as I go
> along ). In this particular instance, I see one resouce that has contention,
> and it is owned ( not available). It has exclusive and shared waiters, but
> both of them leading me to bogus threads…
>
> At least now I could analyze bit more. I’m sure it is going to be long rocky
> ride for me, but once I’m over with (hopefully), I would try to put up the
> things I tried, including turning XP’s verifier for deadlock detection ( and
> quite a bit more :). At least I’m lucky that I could reproduce at will, and
> before a month go by, I will have better understanding, BUT IMAGINE IF
> SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN CUSTOMERS’ SITE. And it
> is a rare
> event, then the complexity of this kind of problem goes thru the roof :).
> Hence derivational logic might become the only way out, any derivational
> logic has lot of underlying (hidden) assumptions often we call ‘internal
> workings’ or ‘what happens when …’. Rest depends on individuals …
>
> -pro
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
> Society Prokash Sinha
> Sent: Tuesday, August 17, 2004 4:19 PM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] DeadLock vs WedLock
>
>
> I just finished 3 different (apprently ) analysis, all leads me to the same
> thing, basically the targetting thread is non-existing !!!. It is win2k with
> sp4. If anyone has any other trick to attack it, or shed some light, pls do
> so…
> I’m out of possibilities for a moment, so will try to look thru the Power
> Mangement path of the PDO handling …
>
> -pro
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@garlic.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · August 18, 2004, 11:00am

Also try !stacks 2

Calvin Guan Software Engineer
ATI Technologies Inc. www.ati.com

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Wednesday, August 18, 2004 10:55 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] DeadLock vs WedLock

!process 0 7 is the common way of solving deadlocks.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Prokash Sinha”
> To: “Windows System Software Devs Interest List”
> Sent: Wednesday, August 18, 2004 8:34 AM
> Subject: RE: [ntdev] DeadLock vs WedLock
>
>
> > Finally another round of tests to capture a relatively
> better hang, got me
> > into some sensable call stacks that I need to
> > analyze. Now at least I get the driver on the stack ( note
> that I dont know
> > much about the driver architecture/design, that I have to
> pick as I go
> > along ). In this particular instance, I see one resouce
> that has contention,
> > and it is owned ( not available). It has exclusive and
> shared waiters, but
> > both of them leading me to bogus threads…
> >
> > At least now I could analyze bit more. I’m sure it is going
> to be long rocky
> > ride for me, but once I’m over with (hopefully), I would
> try to put up the
> > things I tried, including turning XP’s verifier for
> deadlock detection ( and
> > quite a bit more :). At least I’m lucky that I could
> reproduce at will, and
> > before a month go by, I will have better understanding, BUT
> IMAGINE IF
> > SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN
> CUSTOMERS’ SITE. And it
> > is a rare
> > event, then the complexity of this kind of problem goes
> thru the roof :).
> > Hence derivational logic might become the only way out, any
> derivational
> > logic has lot of underlying (hidden) assumptions often we
> call ‘internal
> > workings’ or ‘what happens when …’. Rest depends on
> individuals …
> >
> > -pro
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
> > Society Prokash Sinha
> > Sent: Tuesday, August 17, 2004 4:19 PM
> > To: Windows System Software Devs Interest List
> > Subject: Re: [ntdev] DeadLock vs WedLock
> >
> >
> > I just finished 3 different (apprently ) analysis, all
> leads me to the same
> > thing, basically the targetting thread is non-existing !!!.
> It is win2k with
> > sp4. If anyone has any other trick to attack it, or shed
> some light, pls do
> > so…
> > I’m out of possibilities for a moment, so will try to look
> thru the Power
> > Mangement path of the PDO handling …
> >
> > -pro
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@garlic.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ati.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Prokash_Sinha-1 · August 18, 2004, 11:28am

Yes I will look thru those today. The problem is that it never shows up on
XP, so there
must be a significant change in the power management path. The writer took
an example from
Oney’s book and modified, but using the FSM to transit between powerstate,
and I think something
is cooking there in terms of iterative calls to the PowerHandling routine of
the driver that is either
direct iteration ( sort of recursion) or indirect thru I/O Manager calls …
NEED TO FIND MORE…

I’m almost sure that it is not drivers own define of KLOCKs are others that
is gettin in the way, that
would have been relatively easy, and would have shown up on XP …

thx Max,

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Maxim S. Shatskih
Sent: Wednesday, August 18, 2004 7:55 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] DeadLock vs WedLock

!process 0 7 is the common way of solving deadlocks.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Prokash Sinha”
To: “Windows System Software Devs Interest List”
Sent: Wednesday, August 18, 2004 8:34 AM
Subject: RE: [ntdev] DeadLock vs WedLock

> Finally another round of tests to capture a relatively better hang, got me
> into some sensable call stacks that I need to
> analyze. Now at least I get the driver on the stack ( note that I dont
know
> much about the driver architecture/design, that I have to pick as I go
> along ). In this particular instance, I see one resouce that has
contention,
> and it is owned ( not available). It has exclusive and shared waiters, but
> both of them leading me to bogus threads…
>
> At least now I could analyze bit more. I’m sure it is going to be long
rocky
> ride for me, but once I’m over with (hopefully), I would try to put up the
> things I tried, including turning XP’s verifier for deadlock detection (
and
> quite a bit more :). At least I’m lucky that I could reproduce at will,
and
> before a month go by, I will have better understanding, BUT IMAGINE IF
> SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN CUSTOMERS’ SITE. And
it
> is a rare
> event, then the complexity of this kind of problem goes thru the roof :).
> Hence derivational logic might become the only way out, any derivational
> logic has lot of underlying (hidden) assumptions often we call ‘internal
> workings’ or ‘what happens when …’. Rest depends on individuals …
>
> -pro
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
> Society Prokash Sinha
> Sent: Tuesday, August 17, 2004 4:19 PM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] DeadLock vs WedLock
>
>
> I just finished 3 different (apprently ) analysis, all leads me to the
same
> thing, basically the targetting thread is non-existing !!!. It is win2k
with
> sp4. If anyone has any other trick to attack it, or shed some light, pls
do
> so…
> I’m out of possibilities for a moment, so will try to look thru the Power
> Mangement path of the PDO handling …
>
> -pro
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@garlic.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Prokash_Sinha-1 · August 18, 2004, 11:40am

RE: [ntdev] DeadLock vs WedLockThanx Calvin,

I would look thru those sort of ideas :). Pls be ready to shoot anything
comes in mind, I’m bit rusty on using the proper cmds, and ply thru the krnl
sturctures…

I was using NT insiders article, it is an excellent way to analyze deadlocks
when nothing but a random *fairly indirect implication, sort of sideeffect
causes such
deadlocks*. But since those structures are dynamically allocated, hence not
initialized, I was getting garbage values to get to the thread(s). It is
sort of zero-knowledge analysis, IMO.

-pro
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Calvin Guan
Sent: Wednesday, August 18, 2004 7:58 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] DeadLock vs WedLock

Also try !stacks 2

Calvin Guan Software Engineer
ATI Technologies Inc. www.ati.com

-----Original Message-----
> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> Sent: Wednesday, August 18, 2004 10:55 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] DeadLock vs WedLock
>
>
> !process 0 7 is the common way of solving deadlocks.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> ----- Original Message -----
> From: “Prokash Sinha”
> To: “Windows System Software Devs Interest List”
> Sent: Wednesday, August 18, 2004 8:34 AM
> Subject: RE: [ntdev] DeadLock vs WedLock
>
>
> > Finally another round of tests to capture a relatively
> better hang, got me
> > into some sensable call stacks that I need to
> > analyze. Now at least I get the driver on the stack ( note
> that I dont know
> > much about the driver architecture/design, that I have to
> pick as I go
> > along ). In this particular instance, I see one resouce
> that has contention,
> > and it is owned ( not available). It has exclusive and
> shared waiters, but
> > both of them leading me to bogus threads…
> >
> > At least now I could analyze bit more. I’m sure it is going
> to be long rocky
> > ride for me, but once I’m over with (hopefully), I would
> try to put up the
> > things I tried, including turning XP’s verifier for
> deadlock detection ( and
> > quite a bit more :). At least I’m lucky that I could
> reproduce at will, and
> > before a month go by, I will have better understanding, BUT
> IMAGINE IF
> > SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN
> CUSTOMERS’ SITE. And it
> > is a rare
> > event, then the complexity of this kind of problem goes
> thru the roof :).
> > Hence derivational logic might become the only way out, any
> derivational
> > logic has lot of underlying (hidden) assumptions often we
> call ‘internal
> > workings’ or ‘what happens when …’. Rest depends on
> individuals …
> >
> > -pro
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
> > Society Prokash Sinha
> > Sent: Tuesday, August 17, 2004 4:19 PM
> > To: Windows System Software Devs Interest List
> > Subject: Re: [ntdev] DeadLock vs WedLock
> >
> >
> > I just finished 3 different (apprently ) analysis, all
> leads me to the same
> > thing, basically the targetting thread is non-existing !!!.
> It is win2k with
> > sp4. If anyone has any other trick to attack it, or shed
> some light, pls do
> > so…
> > I’m out of possibilities for a moment, so will try to look
> thru the Power
> > Mangement path of the PDO handling …
> >
> > -pro
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@garlic.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ati.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Prokash_Sinha-1 · August 18, 2004, 11:00pm

As I suspected …

Not sure why I could not get thru the direct analysis :). But when
I did !locks -v I got enough information(s) to track
down,
all the structures made sense …

Due to power handling code ( either using modified(?) fsm of oney’s example
or using 1394 power example of MS code ) if the threads visits I/O Manger
thru PoCallDriver() it wrecks havoc. Note that there is a call tobe made
before PoCall*(), is so far Okay, but need to revisit !!! . I dont have the
detail w/me, but intending to gather
all for a later posting here. AGAIN NONE OF THESE ARE ANY SYMPTOM TO XP, it
is perfect as far as this particular path of
execution!. SO IT BOILS DOWN TO MORE ANALYSIS TO CORRELATE BETWEEN XP’s
handling and win2k handling of this path.

Afterall the path I’ve been tracing now make sense a bit, so I can proceed
further …

Knowing what ‘I know or I think I know’ would definitely make my days, but a
big question I suppose !!!

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Prokash Sinha
Sent: Wednesday, August 18, 2004 8:27 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] DeadLock vs WedLock

Yes I will look thru those today. The problem is that it never shows up on
XP, so there
must be a significant change in the power management path. The writer took
an example from
Oney’s book and modified, but using the FSM to transit between powerstate,
and I think something
is cooking there in terms of iterative calls to the PowerHandling routine of
the driver that is either
direct iteration ( sort of recursion) or indirect thru I/O Manager calls …
NEED TO FIND MORE…

I’m almost sure that it is not drivers own define of KLOCKs are others that
is gettin in the way, that
would have been relatively easy, and would have shown up on XP …

thx Max,

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Maxim S. Shatskih
Sent: Wednesday, August 18, 2004 7:55 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] DeadLock vs WedLock

!process 0 7 is the common way of solving deadlocks.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Prokash Sinha”
To: “Windows System Software Devs Interest List”
Sent: Wednesday, August 18, 2004 8:34 AM
Subject: RE: [ntdev] DeadLock vs WedLock

> Finally another round of tests to capture a relatively better hang, got me
> into some sensable call stacks that I need to
> analyze. Now at least I get the driver on the stack ( note that I dont
know
> much about the driver architecture/design, that I have to pick as I go
> along ). In this particular instance, I see one resouce that has
contention,
> and it is owned ( not available). It has exclusive and shared waiters, but
> both of them leading me to bogus threads…
>
> At least now I could analyze bit more. I’m sure it is going to be long
rocky
> ride for me, but once I’m over with (hopefully), I would try to put up the
> things I tried, including turning XP’s verifier for deadlock detection (
and
> quite a bit more :). At least I’m lucky that I could reproduce at will,
and
> before a month go by, I will have better understanding, BUT IMAGINE IF
> SOMETHING LIKE THIS HAPPENS ON A RELEASE VERSION IN CUSTOMERS’ SITE. And
it
> is a rare
> event, then the complexity of this kind of problem goes thru the roof :).
> Hence derivational logic might become the only way out, any derivational
> logic has lot of underlying (hidden) assumptions often we call ‘internal
> workings’ or ‘what happens when …’. Rest depends on individuals …
>
> -pro
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Programmers
> Society Prokash Sinha
> Sent: Tuesday, August 17, 2004 4:19 PM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] DeadLock vs WedLock
>
>
> I just finished 3 different (apprently ) analysis, all leads me to the
same
> thing, basically the targetting thread is non-existing !!!. It is win2k
with
> sp4. If anyone has any other trick to attack it, or shed some light, pls
do
> so…
> I’m out of possibilities for a moment, so will try to look thru the Power
> Mangement path of the PDO handling …
>
> -pro
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@garlic.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com