Thanks for the response Ladik.
You say:
Correct me someone If I am wrong, but this approach will never
work. What IRPs do you handle by calling user mode and
why do you do it at all ?
I do this because the data that I want to expose as a file system is
implemented by a user-mode service.
I’m very interested in why you say this approach will never work. Can you
be more specific please? So far, I have implemented CREATE, CLEANUP,
DEVICE_CONTROL, DIRECTORY_CONTROL, SET_INFO, QUERY_INFO this way. (IRPs that
don’t need to be sent to the user-mode app (at least for now) are CLOSE,
SHUTDOWN, LOCK_CONTROL, FLUSH_BUFFERS. The rest I have not implemented.)
Note that I do not believe this problem is specific to my implementation
using a user-mode service. This problem would occur for any file system
that asynchronously implements QUERY_INFO and tries to do caching I/O, since
the problem, as far as I can tell, is in the combination of
MmCreateSection() and FsRtlGetFileSize(). Has anyone implemented QUERY_INFO
asynchronously?
One solution would be to detect inside the IRP dispatch function that
“someone has already acquired this FCB’s resource and will hold on to it
until the IRP completes”. If I knew this, I would know that I don’t need to
re-acquire the resource in my driver once I get the response from the
user-mode app.
How could I detect this? I was thinking that if the resource is already
acquired upon entry to the IRP dispatch function, and
IoIsOperationSynchronous() tells me this is a synchronous IRP, then I assume
this is true. I need to experiment with this, its just a theory at this
point.
You say:
In your FSD, can it happen that you are holding a lock
while calling user mode ? If yes, the the deadlock
will come sooner or later. If not, there’s a sync problem
almost certainly.
I am being very careful not to hold any locks after sending the packet to
the user-mode service, exactly for this reason: to prevent deadlocks.
However, in this case, its FsRtlGetFileSize() that is holding the FCB’s
Resource, exclusively at that, while it calls KeWaitForSingleObject() to
wait for the QUERY_INFO IRP to complete. So I guess the answer to your
question is yes, although the code holding the lock is not code I have
control over. [See the end of this email for call stack]
Because of the last statement, if I cannot find a reliable way to detect
that the resource is already acquired as mentioned above, I will probably
need to change my design to assume the resources are always acquired,
instead of never acquired, across calls to the user-mode service. This
would imply a lot more robustness code to make sure locks are correctly
released regardless of the correct or incorrect behaviour of the user-mode
app, but I think this is feasible.
Another solution would be to make sure QUERY_INFO can always be completed
synchronously. This would require caching at the driver level, and now
implies a cache coherency issue.
Here is the call stack of the thread that owns the FCB’s resource. I don’t
know if its MmCreateSection() or FsRtlGetFileSize() that acquires the
resource on the FCB. However, I do know that in this thread: 1) the
resource is acquired, 2) QUERY_INFO is called on my driver, and then 3)
KeWaitForSingleObject() is called without first releasing the resource.
THREAD 857c4340 Cid 03dc.0114 Teb: 7ffdf000 Win32Thread: e11bdb68 WAIT:
(Executive) KernelMode Non-Alertable
f2f54c0c NotificationEvent
Not impersonating
DeviceMap e155c220
Owning Process 857c45b8 Image: cmd.exe
Wait Start TickCount 113411 Ticks: 6 (0:00:00:00.093)
Context Switch Count 1670 LargeStack
UserTime 00:00:00.0140
KernelTime 00:00:11.0000
Start Address kernel32!BaseProcessStartThunk (0x77e5e3a4)
Win32 Start Address 0x4ad22a81
Stack Init f2f55000 Current f2f54b80 Base f2f55000 Limit f2f52000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f2f54b98 80a3e898 857c43b0 857c4340 80a33c6e nt!KiSwapContext+0x2f (FPO:
[Uses EBP] [0,0,4])
f2f54ba4 80a33c6e 85b70290 00000103 85b44f90 nt!KiSwapThread+0x46 (FPO:
[0,0,0])
f2f54bcc 80b11367 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x22e
(FPO: [Non-Fpo])
f2f54c24 80a5c62e 85b44f01 f2f54c9c 00000000 nt!FsRtlGetFileSize+0x115 (FPO:
[Non-Fpo])
f2f54cc4 80b887e9 f2f54d08 000f001f 00000000 nt!MmCreateSection+0x5fe (FPO:
[Non-Fpo])
f2f54d34 80ad5a48 0013e9b8 000f001f 00000000 nt!NtCreateSection+0x121 (FPO:
[Non-Fpo])
f2f54d34 7c834684 0013e9b8 000f001f 00000000 nt!KiFastCallEntry+0x158 (FPO:
[0,3] TrapFrame @ f2f54d64)
0013ee18 00000000 00000000 00000000 00000000 ntdll!KiFastSystemCallRet (FPO:
[0,0,0])
=================================================
Roger Tawa
http://tawacentral.net/
[One thing about paradigms: shift happens.]
[When you stop, you’re done.]
-----Original Message-----
From: Roger Tawa [mailto:xxxxx@tawacentral.net]
Sent: Monday, January 09, 2006 7:08 PM
To: ‘xxxxx@lists.osr.com’
Subject: deadlock during cached read
Hi all,
I am starting to implement the cached read IRP for my file system, and I am
getting a deadlock. I’m pretty sure I understand why it deadlocks, but I’m
not sure what the best solution is and would like to have any suggestions
you might have.
I am running in WinXP SP2, checked build, automatic updates off. I am using
DDK 3790, and testing with the checked build of my driver. My file system
works with a user-mode NT service. That is, the handling of most IRPs sends
a packet to the user-mode service and returns STATUS_PENDING. At some later
point, the service calls back into the driver with the response, and the IRP
is completed.
The scenario that causes the deadlock is when I use cmd.exe to copy a file
from my file system to C:. My test file is very small (44 bytes). I see
the initial read IRP, which calls CcCopyRead(), which causes a second read
IRP to come in recursively, this time with the paging io flag set. For now,
to keep things simple, I just zero the memory and return that as the
contents of the file.
At some point after the read IRPs, FsRtlGetFileSize() is called when
creating an Mm section. This sends a QUERY_INFO IRP to my file system,
which sends a packet to the NT service and returns with STATUS_PENDING. The
thread that called FsRtlGetFileSize() then waits for the IRP to complete,
while holding the file’s resource exclusively.
Later, when the NT service processes the packet and responds, my driver
tries to get the file’s resource before completing the IRP and hangs.
Is there any way to know when I get the QUERY_INFO IRP that it is being
called from FsRtlGetFileSize()?
Thanks in advance.
=================================================
Roger Tawa
http://tawacentral.net/
[One thing about paradigms: shift happens.]
[When you stop, you’re done.]