bug check on PsTerminateSystemThread

NTFSD
1

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b ... enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool...
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0

***
*** An error (or corruption) in the pool was detected;
*** Pool Region unknown (0xFFFFFFFFFFDFF000)
***
*** Use !poolval ffdff000 for more details.
***

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

2

So you have a piece of paged pool memory accessed with interrupts
disabled. From the name I'd guess this is a callback object. Do you
create any callback objects (ExRegisterCallback)? Is that a tag you
recognize from your own driver? If not, you'll need to track down from
whence this object is allocated (probably by watching for that tag
allocation).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 18-21, 2006 (note new date - MS scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alon
Sent: Wednesday, March 08, 2006 11:01 AM
To: ntfsd redirect
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b ... enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
************************************************************************
*******
*
*
* Bugcheck Analysis
*
*
*
************************************************************************
*******

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool...
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0

***
*** An error (or corruption) in the pool was detected;
*** Pool Region unknown (0xFFFFFFFFFFDFF000)
***
*** Use !poolval ffdff000 for more details.
***

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

Questions? First check the IFS FAQ at

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

So, you have a multiprocessor system, a driver of your flavor, and things
work wonderfully as long as you only use one processor in all those
processors you have. But ... use the full power of your system (enable all
processors) and your driver pukes, not all the time but sometimes.

  1. You could have a synchronization/serialization problem in your driver,
    and are touching memory that is not paged in or has been released, such as
    a completed IRP or an IRP that has been passed down.
  2. You could have done something silly like allocate sizeof(pointer)
    instead of sizeof(struct).

Gary G. Little

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Wednesday, March 08, 2006 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b ... enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
**************************************************************************
*****
*
*
* Bugcheck Analysis
*
*
*
**************************************************************************
*****

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool...
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0

***
*** An error (or corruption) in the pool was detected;
*** Pool Region unknown (0xFFFFFFFFFFDFF000)
***
*** Use !poolval ffdff000 for more details.
***

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

Questions? First check the IFS FAQ at

You are currently subscribed to ntfsd as: xxxxx@seagate.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

Your thread called PsTerminateSystemThread while at high IRQL. That
function can be called only at passive. Investigate why your thread was
at high IRQL.

System was attempting to get callback block that is set up for
process/thread notification purposes, so it can notify all interested
parties. However that block is allocated in paged pool and hence the
crash.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alon
Sent: Wednesday, March 08, 2006 8:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b ... enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
************************************************************************
*******
*
*
* Bugcheck Analysis
*
*
*
************************************************************************
*******

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool...
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0

***
*** An error (or corruption) in the pool was detected;
*** Pool Region unknown (0xFFFFFFFFFFDFF000)
***
*** Use !poolval ffdff000 for more details.
***

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

Questions? First check the IFS FAQ at

You are currently subscribed to ntfsd as: xxxxx@appstream.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

Hi,

Well I’m not using at all ExRegisterCallback, and I
don’t recognize this pool tag…
I really don’t know where this ‘Cbrb’ came from or
from where ExGetCallBackBlockRoutine is coming.
Really don’t have a clue…any idea where to start
from? Any explansion why it happen only on MP?

Alon

“Tony Mason” wrote in message
news:xxxxx@ntfsd…
So you have a piece of paged pool memory accessed with
interrupts
disabled. From the name I’d guess this is a callback
object. Do you
create any callback objects (ExRegisterCallback)? Is
that a tag you
recognize from your own driver? If not, you’ll need
to track down from
whence this object is allocated (probably by watching
for that tag
allocation).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File
Systems class in
Boston, MA April 18-21, 2006 (note new date - MS
scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
Alon
Sent: Wednesday, March 08, 2006 11:01 AM
To: ntfsd redirect
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b … enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
****************************************************************


Bugcheck Analysis


*
*****************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool…
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0


 An error (or corruption) in the pool was detected;
Pool Region unknown (0xFFFFFFFFFFDFF000)

Use !poolval ffdff000 for more details.

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

6

Someone else did ask “are you calling this function at elevated IRQL or
with interrupts disabled?” That would also cause this problem (assuming
callback structures are allocated from paged pool.)

Try adding a PAGED_CODE() macro before your call to
PsTerminateSystemThread, see if that triggers (before the call) to
confirm you aren’t at elevated IRQL.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 18-21, 2006 (note new date - MS scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alon
Sent: Wednesday, March 08, 2006 2:44 PM
To: ntfsd redirect
Subject: Re:[ntfsd] bug check on PsTerminateSystemThread

Hi,

Well I’m not using at all ExRegisterCallback, and I
don’t recognize this pool tag…
I really don’t know where this ‘Cbrb’ came from or
from where ExGetCallBackBlockRoutine is coming.
Really don’t have a clue…any idea where to start
from? Any explansion why it happen only on MP?

Alon

“Tony Mason” wrote in message
news:xxxxx@ntfsd…
So you have a piece of paged pool memory accessed with
interrupts
disabled. From the name I’d guess this is a callback
object. Do you
create any callback objects (ExRegisterCallback)? Is
that a tag you
recognize from your own driver? If not, you’ll need
to track down from
whence this object is allocated (probably by watching
for that tag
allocation).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File
Systems class in
Boston, MA April 18-21, 2006 (note new date - MS
scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
Alon
Sent: Wednesday, March 08, 2006 11:01 AM
To: ntfsd redirect
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b … enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
****************************************************************


Bugcheck Analysis


*
*****************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool…
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0


 An error (or corruption) in the pool was detected;
Pool Region unknown (0xFFFFFFFFFFDFF000)

Use !poolval ffdff000 for more details.

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

7

Are you using DriverStudio orC++? Then make sure you don’t free your thread object(eg. creating your system thread object in the stack and leave the function). Hope this helps.

From: xxxxx@lists.osr.com on behalf of Alon
Sent: Thu 3/9/2006 6:43 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] bug check on PsTerminateSystemThread

Hi,

Well I’m not using at all ExRegisterCallback, and I
don’t recognize this pool tag…
I really don’t know where this ‘Cbrb’ came from or
from where ExGetCallBackBlockRoutine is coming.
Really don’t have a clue…any idea where to start
from? Any explansion why it happen only on MP?

Alon

“Tony Mason” wrote in message
news:xxxxx@ntfsd…
So you have a piece of paged pool memory accessed with
interrupts
disabled. From the name I’d guess this is a callback
object. Do you
create any callback objects (ExRegisterCallback)? Is
that a tag you
recognize from your own driver? If not, you’ll need
to track down from
whence this object is allocated (probably by watching
for that tag
allocation).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File
Systems class in
Boston, MA April 18-21, 2006 (note new date - MS
scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
Alon
Sent: Wednesday, March 08, 2006 11:01 AM
To: ntfsd redirect
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b … enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
****************************************************************


Bugcheck Analysis


*
*****************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool…
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0


 An error (or corruption) in the pool was detected;
Pool Region unknown (0xFFFFFFFFFFDFF000)

Use !poolval ffdff000 for more details.

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@pctools.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

8

Hi,

Well, Thank you for the suggestions.
I think i’ve found the problem.
I typed the following:

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc

reminder:

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

so I typed unassamble to eax+4:
0: kd> u bad9f3dc
BCHKD+0x1e3dc:
bad9f3dc 55 push ebp
bad9f3dd 8bec mov ebp,esp
bad9f3df 83ec00 sub esp,0x0
bad9f3e2 53 push ebx
bad9f3e3 56 push esi
bad9f3e4 57 push edi
bad9f3e5 837d1000 cmp dword ptr
[ebp+0x10],0x0
bad9f3e9 ff750c push dword ptr [ebp+0xc]

It looks like a BCHKD (Bounds Checker) function, and
that is probably the cause for the problem…
Pity I thought it is my fault ;-(

BTW: I think there is a problem with the WinDbg since
0: kd> ln bad9f3dc
returnd nothing (instead BCHKD+0x1e3dc)

What do you say?

Alon

“Tony Mason” wrote in message
news:xxxxx@ntfsd…
Someone else did ask “are you calling this function at
elevated IRQL or
with interrupts disabled?” That would also cause this
problem (assuming
callback structures are allocated from paged pool.)

Try adding a PAGED_CODE() macro before your call to
PsTerminateSystemThread, see if that triggers (before
the call) to
confirm you aren’t at elevated IRQL.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File
Systems class in
Boston, MA April 18-21, 2006 (note new date - MS
scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
Alon
Sent: Wednesday, March 08, 2006 2:44 PM
To: ntfsd redirect
Subject: Re:[ntfsd] bug check on
PsTerminateSystemThread

Hi,

Well I’m not using at all ExRegisterCallback, and I
don’t recognize this pool tag…
I really don’t know where this ‘Cbrb’ came from or
from where ExGetCallBackBlockRoutine is coming.
Really don’t have a clue…any idea where to start
from? Any explansion why it happen only on MP?

Alon

“Tony Mason” wrote in message
news:xxxxx@ntfsd…
So you have a piece of paged pool memory accessed with
interrupts
disabled. From the name I’d guess this is a callback
object. Do you
create any callback objects (ExRegisterCallback)? Is
that a tag you
recognize from your own driver? If not, you’ll need
to track down from
whence this object is allocated (probably by watching
for that tag
allocation).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File
Systems class in
Boston, MA April 18-21, 2006 (note new date - MS
scheduled plugfest the
same week again.)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
Alon
Sent: Wednesday, March 08, 2006 11:01 AM
To: ntfsd redirect
Subject: [ntfsd] bug check on PsTerminateSystemThread

Hi everyone,

I run my driver (MyDrv.sys) with driver verifier on:
0: kd> !verifier
Verify Level 5b … enabled options are:
Special pool
Special irql
All pool allocations checked on unload
Io subsystem checking enabled
Enhanced Io checking enabled
when I run without /onecpu flag on Boot.ini, which
means:
Windows XP Kernel Version 2600 (Service Pack 2) MP (2
procs) Free x86 compatible
I have from time to time crashes that look like this:
(I put a lot of data here - if you need more - just
let me know)

0: kd> !analyze -v
****************************************************************


Bugcheck Analysis


*
*****************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or
completely invalid) address at an
interrupt request level (IRQL) that is too high. This
is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack
backtrace.
Arguments:
Arg1: e10362c4, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write
operation
Arg4: 8060b11a, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: e10362c4 Paged pool

CURRENT_IRQL: ff

FAULTING_IP:
nt!ExGetCallBackBlockRoutine+8
8060b11a 8b4004 mov eax,[eax+0x4]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8060b11a to 805436d0

STACK_TEXT:
a7c6ec68 8060b11a badb0d00 e10362c6 a7c6ecc8
nt!KiTrap0E+0x238
a7c6ecdc 805d0967 e10362c0 81e508b8 81e50b00
nt!ExGetCallBackBlockRoutine+0x8
a7c6ed64 805d1098 00000000 00000000 81e508b8
nt!PspExitThread+0xb3
a7c6ed84 805d141e 81e508b8 00000000 a7c6edac
nt!PspTerminateThreadByPointer+0x52
a7c6ed94 a6ab193c 00000000 00000000 00000000
nt!PsTerminateSystemThread+0x24
a7c6edac 805ce794 833a6c20 00000000 00000000
MyDrv!ThreadFunction+0x8c
a7c6eddc 805450ce a6ab18b0 833a6c20 00000000
nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
MyDrv!ThreadFunction+8c
a6ab193c 8be5 mov esp,ebp

FAULTING_SOURCE_CODE:
62: // Exit the thread
63: PsTerminateSystemThread( Status ) ;

SYMBOL_STACK_INDEX: 5

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: MyDrv!AThreadFunction+8c

MODULE_NAME: MyDrv

IMAGE_NAME: MyDrv.Sys

DEBUG_FLR_IMAGE_TIMESTAMP: 440da4e2

FAILURE_BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

BUCKET_ID: 0xA_VRF_MyDrv!AThreadFunction+8c

Followup: MachineOwner

0: kd> !pool ffdff13c
Pool page ffdff13c region is Unknown
ffdff000 is not a valid small pool allocation,
checking large pool…
unable to get pool big page table - either wrong
symbols or pool tagging is disabled
ffdff000 is freed (or corrupt) pool
Bad previous allocation size @ffdff000, last size was
0


 An error (or corruption) in the pool was detected;
Pool Region unknown (0xFFFFFFFFFFDFF000)

Use !poolval ffdff000 for more details.

0: kd> .trap a7c6ec68
ErrCode = 00000000
eax=e10362c0 ebx=e10362c0 ecx=e10362c0 edx=e10362c6
esi=81e508b8 edi=8235c660
eip=8060b11a esp=a7c6ecdc ebp=a7c6ecdc iopl=0
nv up di ng nz na po nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010086
nt!ExGetCallBackBlockRoutine+0x8:
8060b11a 8b4004 mov eax,[eax+0x4]
ds:0023:e10362c4=bad9f3dc
0: kd> dd eax
e10362c0 00000010 bad9f3dc 00000000 00000000
e10362d0 0c060403 61564d43 00240000 000c6b76
e10362e0 80000004 0000005c 00000001 00000001
e10362f0 6f4c734f 72656461 68746150 00000000
e1036300 00010406 52706341 0c030401 6d4e624f
e1036310 00300030 00300030 00300030 00640036
e1036320 000c0403 656c5252 e10809d8 e14e0418
e1036330 e2c6be98 00000070 00000000 00000000

0: kd> !pool eax
Pool page e10362c0 region is Paged pool
e1036000 size: 40 previous size: 0 (Allocated)
Ppen
e1036040 size: 8 previous size: 40 (Free)
T..
e1036048 size: 28 previous size: 8 (Allocated)
CMVa
e1036070 size: 30 previous size: 28 (Allocated)
CMVa
e10360a0 size: 28 previous size: 30 (Allocated)
NtFs
e10360c8 size: 8 previous size: 28 (Free)
CMVa
e10360d0 size: 10 previous size: 8 (Allocated)
ObDi
e10360e0 size: 68 previous size: 10 (Allocated)
Ntfo
e1036148 size: 18 previous size: 68 (Allocated)
Ppsu
e1036160 size: 8 previous size: 18 (Free)
CMVa
e1036168 size: 68 previous size: 8 (Allocated)
MmSt
e10361d0 size: 28 previous size: 68 (Allocated)
ObNm
e10361f8 size: 8 previous size: 28 (Free)
CMVI
e1036200 size: 28 previous size: 8 (Allocated)
NtFs
e1036228 size: 20 previous size: 28 (Allocated)
ArbR
e1036248 size: 68 previous size: 20 (Allocated)
ScPA
e10362b0 size: 8 previous size: 68 (Free)
ObSq
*e10362b8 size: 18 previous size: 8 (Allocated)
*Cbrb
Owning component : Unknown (update pooltag.txt)
e10362d0 size: 30 previous size: 18 (Allocated)
CMVa
e1036300 size: 8 previous size: 30 (Free)
AcpR
e1036308 size: 18 previous size: 8 (Allocated)
ObNm
e1036320 size: 60 previous size: 18 (Free)
RRle
e1036380 size: 30 previous size: 60 (Allocated)
MmSt
e10363b0 size: 30 previous size: 30 (Allocated)
RRle
e10363e0 size: 10 previous size: 30 (Free)
RRle
e10363f0 size: 20 previous size: 10 (Allocated)
Pp
e1036410 size: 10 previous size: 20 (Allocated)
ObDi
e1036420 size: 28 previous size: 10 (Free)
CMVa
e1036448 size: 188 previous size: 28 (Allocated)
CMSc (Protected)
e10365d0 size: 190 previous size: 188 (Allocated)
CMSc (Protected)
e1036760 size: 160 previous size: 190 (Allocated)
CMSc (Protected)
e10368c0 size: 178 previous size: 160 (Allocated)
CMSc (Protected)
e1036a38 size: 150 previous size: 178 (Allocated)
CMSc (Protected)
e1036b88 size: 90 previous size: 150 (Allocated)
CMSc (Protected)
e1036c18 size: 108 previous size: 90 (Allocated)
CMSc (Protected)
e1036d20 size: 158 previous size: 108 (Allocated)
CMSc (Protected)
e1036e78 size: 188 previous size: 158 (Allocated)
CMSc (Protected)

Thanks!

Alon


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com