Kiran,
Are you sure that the writes are coming down through the lazy writer?
In other words, did you see the full stack of the paging writes to your
file and see that the top-most routines in the worker thread were
something like CcWorkerThread then CcWriteBehind?
You need to first determine if the large number of NoModWrite pages are
for your file or not. You can take a look at the CONTROL_AREA for the
data section which backs your stream to determine if this section is
marked to disable modified write. The CONTROL_AREA is the first pointer
in the SECTION_OBJECT_POINTERS structure from the file object. The
debugger extension “!ca” will dump the interesting information from the
CONTROL_AREA structure, including the interpretations of the flags. A
flag will be set if it is marked for NoModWrite.
If your stream is *not* marked to disable the ModWrite, you could have
either the lazy writer or the mapped page writer flushing the dirty data
in your file. If the mapped page writer does the flush, the dirty page
count which the cache manager tracks does not get reduced until the lazy
writer tries to flush the same data. Assuming we don’t have the mapped
page wirter and lazy writer colliding, when the lazy writer issues the
flush, the memory manager will no-op the writes because it know that the
pages have already been flushed and Mm will return STATUS_SUCCESS to the
lazy writer.
Note that the memory manager’s tracking of dirty pages is the truth.
The tracking of dirty pages which the cache manager does is correct with
respect to what the cache manager knows, but this can be out of date if
the memory manager is also flushing dirty data to the stream.
For a data stream which is still in use, the common way for modified
write to be disabled in it is that the FSRTL_FLAG2_DO_MODIFIED_WRITE was
*not* set in the FSRTL_COMMON_HEADER when caching was initialized on the
stream. In this case, the cache manager will call
MmDisableModifiedWriteOfSection() on behalf of the file system to
disable the mapped page writer.
Molly Brown
Microsoft Corporation
This posting is provided “AS IS” with no warranties and confers no
rights.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Wednesday, April 07, 2004 7:55 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] CcCopyWrite and StreamFileObjects
Kiran,
Well, this whole area is ugly and my initial response was wrong.
First: modified no write state indicates the pages are not to be written
back to the disk. This is a page level attribute, explicitly set when
some part of the OS wants to ensure that a page is NOT written out
prematurely. For example, the log mechanism used by NTFS needs to do
this in order to control the ordering of write back operations
explicitly. The most common case for this is during section tear-down
operations (when the last thing MM wants is the modified page writer to
start flushing back pages IT is trying to flush back due to a section
tear down operation.) For example, if you are calling
MmForceSectionClosed, it will cause pages to go into this state.
As for stream file objects, I was incorrect. The policy really depends
upon the state of the Flags2 field (FSRTL_FLAG2_DO_MODIFIED_WRITE) which
is not set for all stream file objects (versus the FO_STREAM_FILE bit
that is.) My mistake stemmed from the fact that this is ONLY done in
stream file objects, but that doesn’t mean it is done for ALL stream
file objects.
So, the bottom line is that if the pages for your files are in the
modified no write case, this suggests that either you are using the
FSRTL_FLAG2_DO_MODIFIED_WRITE bit, there is an explicit force for those
pages to become modified-no-write (changing the PFN entry, for instance)
or the section(s) are being torn down but something ELSE is interfering
with the process so that never completes.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Kiran Joshi
Sent: Wednesday, April 07, 2004 9:23 AM
To: ntfsd redirect
Subject: Re:[ntfsd] CcCopyWrite and StreamFileObjects
Hello Tony,
First of all thanks for your reply.
I have couple of questions :
-
If the lazy writer does not write the dirty pages to disk, then why I
am seeing paging IO writes in the cotext of the lazy writer (When i see
the call stack in the write dispatch, I see the write originated in
CcWriteBehind function). Also my Accquire/ReleaseforLazywrite call back
function is also getting invoked. So thats why, I infered that the lazy
writer is writting the dirty data to disk.
-
Does “stream file objects are controlled by FSD” indirectly mean that
FSD should only use mapping/pinnig inteface with stream file objects?
Copy interfaces should never be used by FSD with stream file objects.
-
I periodically call CcFlushCaceh for the above file object, I hope
you were not referring to this.
thanks
-Kiran
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Kiran,
Stream file objects are not written back by the lazy writer. This is by
design of the cache manager - stream file objects are explicitly
controlled by the FSD (or in your case the “extra” component) and are
not written back automatically by the lazy writer or the modified page
writer (well, MPW will write them back when things get really bad!)
So, are you remembering to flush out your dirty pages? If not, you
might wish to consider that (MmFlushSection).
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Kiran Joshi
Sent: Tuesday, April 06, 2004 2:17 PM
To: ntfsd redirect
Subject: [ntfsd] CcCopyWrite and StreamFileObjects
Hello All,
I am facing a problem when I use CcCopyRead/Write interface
functions with StreamFileObjects. If extensive data is written to the
cache using CcCopyWrite, and even after successfully completeing all the
paging IO writes issued by the lazy writer for all the dirty data, I can
see a huge number pages belonging to the above streamfile, as being
marked as dirty.
The !memusage shows a huge value for the ModifiedNoWrite pages, and in
the file listing I can see my streamfile object having large number of
dirty pages. To confirm that pages remain marked dirty even after lazy
writter has written those pages, I took the system to a limit where the
system started to thrash very badly.
So the question which I had was is it ok to use CcCopyRead/Write
functions with streamfile objects. Or is it that only mapping/pinning
interfaces work properly with streamfileobjects?
I am sure that lazy writter is issuing paging IO writes for all the
dirty pages, and I am completing all these write requests successfully
then why is it not marking those pages as clean?
Note: My driver is not FSFD or FSD. I want to use cache mananger to
manage the data for me, instead of rolling my own caching mechanism and
I dont want to use mapping/pinning interface.
thanks
-Kiran
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com