DFS_FILE_SYSTEM bugcheck

NTFSD
1

Hi,

we have customer that is experiencing a DFS_FILE_SYSTEM bugcheck and would
appreciate some help in understanding the windbg output.

our driver is a mini-filter but currently does absolutely nothing ( long
story ); the bugcheck occurs on a XPSP2 fully patched wks and the DFS
servers are all running Windows 2003 Server. It appears that the bugcheck
happens when someone browses to the dfs root folder, not subfolders.

I see an access denied being passed as argument to the exception filter but
am failing to see how our driver is/could be involved. could it be a
compatibility issue between SAV and the mini-filters?

according to our customer, the bugcheck only occurs if our product is
installed.

thank you in advance...

Marco

0: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************

DFS_FILE_SYSTEM (82)
Arguments:
Arg1: ba5b92e0
Arg2: ba5b34c2
Arg3: 00000000
Arg4: 00000000

Debugging Details:

CUSTOMER_CRASH_COUNT: 3

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0x82

LAST_CONTROL_TRANSFER: from ba5b929c to 804f9f13

STACK_TEXT:
9b974b0c ba5b929c 00000082 ba5b92e0 ba5b34c2 nt!KeBugCheckEx+0x1b
9b974b30 ba5b92e0 00000000 00000000 9b9753bc Mup!DfsBugCheck+0x2a
9b974b40 ba5b34c2 89cb9610 c0000005 9b974b70 Mup!DfsExceptionFilter+0x3f
9b974b50 80538ea1 9b974b78 00000000 9b974b78 Mup!DfsFsdCreate+0xce
9b974b78 8054618e 9b975024 9b9753ac 9b974d20 nt!_except_handler3+0x61
9b974b9c 80546160 9b975024 9b9753ac 9b974d20 nt!ExecuteHandler2+0x26
9b974c4c 804fe570 9b975024 9b974d20 00000000 nt!ExecuteHandler+0x24
9b975008 80541415 9b975024 00000000 9b975078 nt!KiDispatchException+0x13e
9b975070 805413c6 9b9750fc 806e494f badb0d00 nt!CommonDispatchException+0x4d
9b975084 804e8a1c 9b9750ec a313d518 8a273100 nt!Kei386EoiHelper+0x18a
9b9750fc ba6abc06 89ca66fc 8a29f328 00000000
nt!ExFreeToPagedLookasideList+0x28
9b975160 ba6b30f9 8a29f328 8a3f04c8 00000001
fltMgr!FltpGetStreamListCtrl+0x5a
9b97517c ba6a8425 89d9f728 00000000 89d9f728
fltMgr!FltpCreatePostCompletion+0x65
9b975190 ba6a8817 89d9f784 8a3eee70 9b9751d0
fltMgr!FltpProcessIoCompletion+0x4b
9b9751a0 ba6a8ec5 8a34fcb0 8a3eee70 89d9f728
fltMgr!FltpPassThroughCompletion+0x89
9b9751d0 ba6b5153 9b9751f0 c2ca00c1 00000000
fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x269
9b97520c 804ef163 8a34fcb0 8a3eef4c 8a3f04c8 fltMgr!FltpCreate+0x1e3
9b97521c ba5b6c39 89ca667c 89ca6678 00000000 nt!IopfCallDriver+0x31
9b975250 ba5b8186 89ca6678 ba5ab308 89ca6678 Mup!DnrRedirectFileOpen+0x437
9b9752d4 ba5b9266 00ca6678 00f800d8 8a3eef70 Mup!DnrNameResolve+0x6d5
9b975304 ba5b342c 89cb9610 8a3eee70 8a68ef70
Mup!DnrStartNameResolution+0x280
9b975374 ba5b17ec 89cb9610 8a68eeb8 8a3eee70 Mup!DfsCommonCreate+0x237
9b9753bc ba5b188c 8a68eeb8 8a3eee70 8a3eee80 Mup!DfsFsdCreate+0xe0
9b975414 804ef163 8a68eeb8 8a3eee70 8a3eee70 Mup!MupCreate+0xbc
9b975424 80581fe2 8a68eea0 8a290b1c 9b9755bc nt!IopfCallDriver+0x31
9b975504 805bdf06 8a68eeb8 00000000 8a290a78 nt!IopParseDevice+0xa12
9b97557c 805ba58e 00000000 9b9755bc 00000240 nt!ObpLookupObjectName+0x53c
9b9755d0 80574f33 00000000 00000000 54b12700 nt!ObOpenObjectByName+0xea
9b97564c 805758aa 9b9756f4 80100080 9b975708 nt!IopCreateFile+0x407
9b9756a8 a33bfaff 9b9756f4 80100080 9b975708 nt!IoCreateFile+0x8e
WARNING: Stack unwind information not available. Following frames may be
wrong.
9b975758 a3399994 e1d78248 e19790c0 e1d78240 savrt+0x13aff
9b975778 a3399b43 e1d78240 9b9757d8 e19790e0 SYMEVENT+0xf994
9b97578c a33900ff 9b9757d8 e1dd9c1c e19790e0 SYMEVENT+0xfb43
9b9757a0 a3398848 9b9757d8 00000000 9b9757d8 SYMEVENT+0x60ff
9b9757bc a33917b9 9b9757d8 804f0028 a339187a SYMEVENT+0xe848
9b9757f8 804ef163 89da0918 89ca0008 89ca0008 SYMEVENT+0x77b9
9b975808 ba6a8e67 00000000 89ca0008 00000000 nt!IopfCallDriver+0x31
9b97582c ba6b5153 9b97584c 8a34fcb0 00000000
fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x20b
9b975868 804ef163 8a34fcb0 89ca00e4 89dc7298 fltMgr!FltpCreate+0x1e3
9b975878 ba5b6c39 89de6294 89de6290 00000000 nt!IopfCallDriver+0x31
9b9758ac ba5b8186 89de6290 ba5ab308 89de6290 Mup!DnrRedirectFileOpen+0x437
9b975930 ba5b9266 00de6290 00380018 89ca0108 Mup!DnrNameResolve+0x6d5
9b975960 ba5b342c 8a250930 89ca0008 89d2f368
Mup!DnrStartNameResolution+0x280
9b9759d0 ba5b17ec 8a250930 89d2f2b0 89ca0008 Mup!DfsCommonCreate+0x237
9b975a18 ba5b188c 89d2f2b0 89ca0008 89ca0018 Mup!DfsFsdCreate+0xe0
9b975a70 804ef163 89d2f2b0 89ca0008 89ca0008 Mup!MupCreate+0xbc
9b975a80 80581fe2 89d2f298 8a3a15fc 9b975c18 nt!IopfCallDriver+0x31
9b975b60 805bdf06 89d2f2b0 00000000 8a3a1558 nt!IopParseDevice+0xa12
9b975bd8 805ba58e 00000000 9b975c18 00000040 nt!ObpLookupObjectName+0x53c
9b975c2c 80574f33 00000000 00000000 00000101 nt!ObOpenObjectByName+0xea
9b975ca8 805758aa 0257d974 80100000 0350e104 nt!IopCreateFile+0x407
9b975d04 8057906b 0257d974 80100000 0350e104 nt!IoCreateFile+0x8e
9b975d44 805409ac 0257d974 80100000 0350e104 nt!NtOpenFile+0x27
9b975d44 7c90eb94 0257d974 80100000 0350e104 nt!KiFastCallEntry+0xfc
0350e148 00000000 00000000 00000000 00000000 0x7c90eb94

STACK_COMMAND: kb

FOLLOWUP_IP:
Mup!DfsBugCheck+2a
ba5b929c 90 nop

SYMBOL_STACK_INDEX: 1

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: Mup

IMAGE_NAME: Mup.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 41107ef8

SYMBOL_NAME: Mup!DfsBugCheck+2a

FAILURE_BUCKET_ID: 0x82_Mup!DfsBugCheck+2a

BUCKET_ID: 0x82_Mup!DfsBugCheck+2a

Followup: MachineOwner

0: kd> dd 9b974b70 L2
9b974b70 9b975024 9b974d20
0: kd> .exr 9b975024
ExceptionAddress: 806e494f (hal!ExAcquireFastMutex+0x0000000f)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000001
Parameter[1]: 00000000
Attempt to write to address 00000000
0: kd> .cxr 9b974d20
eax=00000000 ebx=89d9f728 ecx=00000000 edx=00000000 esi=89ca66fc
edi=8a3f04c8
eip=806e494f esp=9b9750ec ebp=9b9750fc iopl=0 nv up ei pl nz na po
cy
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010203
hal!ExAcquireFastMutex+0xf:
806e494f f0ff09 lock dec dword ptr [ecx]
ds:0023:00000000=????????

0: kd> kb 0x30
ChildEBP RetAddr Args to Child
9b9750e8 804ed722 8a3f04c8 00000000 89d9f728 hal!ExAcquireFastMutex+0xf
9b9750fc ba6abc06 89ca66fc 8a29f328 00000000
nt!FsRtlLookupPerStreamContextInternal+0x14
9b975160 ba6b30f9 8a29f328 8a3f04c8 00000001
fltMgr!FltpGetStreamListCtrl+0x5a
9b97517c ba6a8425 89d9f728 00000000 89d9f728
fltMgr!FltpCreatePostCompletion+0x65
9b975190 ba6a8817 89d9f784 8a3eee70 9b9751d0
fltMgr!FltpProcessIoCompletion+0x4b
9b9751a0 ba6a8ec5 8a34fcb0 8a3eee70 89d9f728
fltMgr!FltpPassThroughCompletion+0x89
9b9751d0 ba6b5153 9b9751f0 c2ca00c1 00000000
fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x269
9b97520c 804ef163 8a34fcb0 8a3eef4c 8a3f04c8 fltMgr!FltpCreate+0x1e3
9b97521c ba5b6c39 89ca667c 89ca6678 00000000 nt!IopfCallDriver+0x31
9b975250 ba5b8186 89ca6678 ba5ab308 89ca6678 Mup!DnrRedirectFileOpen+0x437
9b9752d4 ba5b9266 00ca6678 00f800d8 8a3eef70 Mup!DnrNameResolve+0x6d5
9b975304 ba5b342c 89cb9610 8a3eee70 8a68ef70
Mup!DnrStartNameResolution+0x280
9b975374 ba5b17ec 89cb9610 8a68eeb8 8a3eee70 Mup!DfsCommonCreate+0x237
9b9753bc ba5b188c 8a68eeb8 8a3eee70 8a3eee80 Mup!DfsFsdCreate+0xe0
9b975414 804ef163 8a68eeb8 8a3eee70 8a3eee70 Mup!MupCreate+0xbc
9b975424 80581fe2 8a68eea0 8a290b1c 9b9755bc nt!IopfCallDriver+0x31
9b975504 805bdf06 8a68eeb8 00000000 8a290a78 nt!IopParseDevice+0xa12
9b97557c 805ba58e 00000000 9b9755bc 00000240 nt!ObpLookupObjectName+0x53c
9b9755d0 80574f33 00000000 00000000 54b12700 nt!ObOpenObjectByName+0xea
9b97564c 805758aa 9b9756f4 80100080 9b975708 nt!IopCreateFile+0x407
9b9756a8 a33bfaff 9b9756f4 80100080 9b975708 nt!IoCreateFile+0x8e
WARNING: Stack unwind information not available. Following frames may be
wrong.
9b975758 a3399994 e1d78248 e19790c0 e1d78240 savrt+0x13aff
9b975778 a3399b43 e1d78240 9b9757d8 e19790e0 SYMEVENT+0xf994
9b97578c a33900ff 9b9757d8 e1dd9c1c e19790e0 SYMEVENT+0xfb43
9b9757a0 a3398848 9b9757d8 00000000 9b9757d8 SYMEVENT+0x60ff
9b9757bc a33917b9 9b9757d8 804f0028 a339187a SYMEVENT+0xe848
9b9757f8 804ef163 89da0918 89ca0008 89ca0008 SYMEVENT+0x77b9
9b975808 ba6a8e67 00000000 89ca0008 00000000 nt!IopfCallDriver+0x31
9b97582c ba6b5153 9b97584c 8a34fcb0 00000000
fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x20b
9b975868 804ef163 8a34fcb0 89ca00e4 89dc7298 fltMgr!FltpCreate+0x1e3
9b975878 ba5b6c39 89de6294 89de6290 00000000 nt!IopfCallDriver+0x31
9b9758ac ba5b8186 89de6290 ba5ab308 89de6290 Mup!DnrRedirectFileOpen+0x437
9b975930 ba5b9266 00de6290 00380018 89ca0108 Mup!DnrNameResolve+0x6d5
9b975960 ba5b342c 8a250930 89ca0008 89d2f368
Mup!DnrStartNameResolution+0x280
9b9759d0 ba5b17ec 8a250930 89d2f2b0 89ca0008 Mup!DfsCommonCreate+0x237
9b975a18 ba5b188c 89d2f2b0 89ca0008 89ca0018 Mup!DfsFsdCreate+0xe0
9b975a70 804ef163 89d2f2b0 89ca0008 89ca0008 Mup!MupCreate+0xbc
9b975a80 80581fe2 89d2f298 8a3a15fc 9b975c18 nt!IopfCallDriver+0x31
9b975b60 805bdf06 89d2f2b0 00000000 8a3a1558 nt!IopParseDevice+0xa12
9b975bd8 805ba58e 00000000 9b975c18 00000040 nt!ObpLookupObjectName+0x53c
9b975c2c 80574f33 00000000 00000000 00000101 nt!ObOpenObjectByName+0xea
9b975ca8 805758aa 0257d974 80100000 0350e104 nt!IopCreateFile+0x407
9b975d04 8057906b 0257d974 80100000 0350e104 nt!IoCreateFile+0x8e
9b975d44 805409ac 0257d974 80100000 0350e104 nt!NtOpenFile+0x27
9b975d44 7c90eb94 0257d974 80100000 0350e104 nt!KiFastCallEntry+0xfc
0350e148 00000000 00000000 00000000 00000000 0x7c90eb94

--
Cheers,

Marco

mperetti [at] beyondtrust [dot] com
http://leastprivilege.blogspot.com
http://www.beyondtrust.com

2

The file object in question doesn’t support filter contexts but filter manager is trying to manipulate it as if it did. I’ve seen this issue before and it is based upon the assumption that the file object in question points to an FSRTL_FCB_COMMON_HEADER which is not always the case for DFS. If the bits are set properly in this non-common-header case, it will look like the FCB supports filter contexts even when it does not.

Of course, if you installed a *different* mini-filter, I’d expect it to crash as well. It isn’t your filter that is causing this problem, it is the fact that with your mini-filter on the system, filter manager is now attaching to this stack. Without your mini-filter, no attachment and no crash.

It is also possible that you might be able to eliminate this if you don’t have a post-create callback function (since this looks like it happens in post-create.)

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

3

Hi Tony,

thank you for the valuable information – that is indeed what I thought. I am not sure I fully understand how I could elimate the issue however. I do have indeed a post-create but all I do there is return FLT_POSTOP_FINISHED_PROCESSING.

thank you again,

Marco

4

wrote news:xxxxx@ntfsd…
> Hi Tony,
>
> thank you for the valuable information – that is indeed what I thought. I
> am not sure I fully understand how I could elimate the issue however. I do
> have indeed a post-create but all I do there is return
> FLT_POSTOP_FINISHED_PROCESSING.
>
> thank you again,
>
> Marco
>
>

If I am interpreting your call stack correctly, the crash occurs while the
FltMgr is “preparing” the call of your PostOp. So, the only alternative
would be to relinquish PostOp for that particular sort of FCB. Unfortunately
(as far as I know) FCB isn’t setup in PreCreate.

Funny, MS itself states “To support Stream and StreamHandle contexts a file
system must use the FSRTL_ADVANCED_FCB_HEADER”.

5

I’m following up on this issue because we now have an existing customer that has observed virtually the same problem. In looking into this further it appears to be very specific to the path name being used. DFS appears to set FileObject->FsContext when it passes in the call to IRP_MJ_CREATE (now there’s a nice check for a file system: assert that this value is not set on entry!) My theory is that it is doing this to detect a re-entrant call (let’s ignore the fact that over the years I have observed that the litmus test for a re-entrancy check is “can this be stacked” because this one certainly cannot.)

At any rate, my further theory: if the FSD is successful, it overwrites the FsContext value so by the time we get to the post-create path, things are fine. But if the create is NOT successful, this value hasn’t been overwritten. Under some set of circumstances it looks like this FsContext value, if interpreted as a file header, indicates that filter contexts are supported, even though it is not. Frustratingly, this is likely to relate to the specific configuration in use (so you might try this in your own facility and never see the issue.)

This doesn’t seem to be related to the specific behavior of the filter; if anything, it would seem to be an issue between DFS and the FsRtl package, with Filter Manager (and mini-filters) caught in the middle.

In our case, we can simply set the FileObject->FsContext to NULL (we’re the ones returning the “error” which in this case is STATUS_REPARSE.) I do not believe this is a general solution to the issue, however, since it would void the DFS re-entrancy check logic, but it is something you might want to try in this specific customer situation to see if it resolves the immediate problem (e.g., this analysis is correct.)

Frank’s observation now looks insightful: “Unfortunately (as far as I know) FCB isn’t set up in PreCreate.” It isn’t, but in this particular case it LOOKS like it is.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com