My concern: a read that is done in order to satisfy the write operation.
Imagine:
Thread ‘A’ does a ‘write’ operation to 200 bytes. Filter ‘X’ below you
(or FSD ‘X’) decides to post your operation to a work thread (because
Thread ‘A’ requested asynchronous I/O, for example). Filter ‘Y’, also
below you, decides to block-and-wait for the asynchronous I/O. Now
thread ‘B’ (the worker thread) starts operating on the asynchronous I/O,
causes a page fault (after all, the write is on a subsection of a page)
and the paging read operation arrives (in Thread ‘B’ context) in your
filter. You block the thread ‘B’ read request, so neither Thread ‘B’
nor Thread ‘A’ finish up.
I made this a bit elaborate because I’m trying to pre-empt the simple
solutions here (like tracking the fact that Thread ‘A’ is already
groveling around in the FS so you let that second paging read go
through). I can also construct reasonable scenarios in which filters
above yours also flip around thread contexts for their own nefarious
reasons - or maybe they are doing a read on the data BEFORE you do your
write to preserve the data in the file. As someone recently pointed out
on here, you have to be able to handle re-entrancy…
Block user level calls all you want. The moment you start blocking
paging I/O operations, you’re dead. Of course, paging I/O operations
will now include memory mapped files, which means applications that are
bypassing your serialization anyway.
Of course, two applications that read and write to the same file without
their own interlocking are already broken, so you can’t really fix that
case anyway. You just have to make sure your own data structures are
properly (cleanly) updated.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Monday, August 29, 2005 5:02 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
OK, maybe I can put it more simply:
When a read comes along and there is a write in progress, what’s a good
way
for the reader to wait for the write to complete?
More info: All readers and writers are communicating through a single
shared data structure for a file, based on the volume and file ID.
(This is
essentially the File context that may be come sometime in the future.)
A
link to this structure is part of the stream handle context built during
the
create processing.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Monday, August 29, 2005 4:34 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
Ken,
Well, I’m going to be dubious here because this sounds like a deadlock
just waiting to happen.
Are the writes user mode, or non-cached or paging? Do you allow paging
I/O traffic to bypass while holding this lock?
In general, you can block user mode I/O activity and get away with it
just fine. But blocking paging I/O or OS level activity is likely to
lead to a deadlock scenario here.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Monday, August 29, 2005 4:23 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
Tony:
I have an internal queue I’m trying to synchronize access to. (BTW,
this is
a mini-filter.)
The problem is that there could be simultaneous readers and writers of
the
same file. If a write comes in, I need to block other readers and
writers
until the write completes because I need to update the queue iff the
write
was successful. So I lock the queue in the pre-write, but cannot unlock
it
until the post-write.
Does a synchronization event sound reasonable for this?
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Monday, August 29, 2005 3:43 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
You do understand that you block APCs when holding an ERESOURCE is to
prevent arbitrary re-entrancy created deadlocks, right? Those same
deadlocks are present for ALL lock types.
Holding bona fide locks of this type is a “bad idea” in any case,
because it is just begging for the “post it to a worker thread and then
block and wait” scenario we’ve seen oh so many times in this business.
Using an ERESOURCE you’d at least be able to see it quickly - with a
KEVENT you’ll have a tougher time debugging it.
Bottom line: holding locks to calls outside your filter does not play
well with other filter drivers.
If you describe what you are trying to do, it might be possible for us
to suggest some alternatives.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Monday, August 29, 2005 2:46 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
Thanks, Ravi.
A sanity check: as I mentioned in my original question, I want to hold
this
lock from a pre-write to a post-write operation. An ERESOURCE blocks
kernel
APCs, so that wouldn’t be appropriate, right? I’m using a
synchronization
event instead – sound reasonable?
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ravisankar
Pudipeddi
Sent: Monday, August 29, 2005 1:43 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
I second this. Please stick to ERESOURCEs. Push locks also don’t do well
under contention. They are only needed for really well-understood
scenarios on really high performance paths, but with low contention and
based on experimentation. They are nearly impossible to debug, you just
have to pretty much just get the code right. ERESOURCes are much better
supported by verifier, debugger extensions.
Ravi
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Sunday, August 28, 2005 7:35 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRQL for FltReleasePushLock
Sorry, but the documentation *is* correct in this case. Push locks may
come from paged pool and some of the code that manipulates them is
pageable and thus cannot be used at IRQL > APC_LEVEL.
In addition, push locks are not really good to use from a debugging
perspective because they are *impossible* to debug in deadlock cases.
There’s a reason that ERESOURCE locks are the lock of preference in file
systems.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Cross
Sent: Sunday, August 28, 2005 9:38 AM
To: ntfsd redirect
Subject: [ntfsd] IRQL for FltReleasePushLock
NTFSD Folk:
I just noticed a significant point about FltReleasePushLock() which says
“Callers of FltReleasePushLock must be running at IRQL <= APC_LEVEL.”
Really? Most other FltMgr resources can be released at “IRQL <=
DISPATCH_LEVEL if the context was allocated from nonpaged pool”.
I want to acquire a push lock in pre-write and release it in post-write,
but post-write can be called at IRQL <= DISPATCH_LEVEL. There are other
methods available, but push locks seem useful.
I’m hoping it’s a documentation error…
Ken
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com