Hi!
Suppose an application ‘A’ opens a file in cached mode and start performing read/write operations on it. And then an application ‘B’ opens the same file in Non-cached mode and issues write request to it… To be specific, lets say ‘B’ issues 2 non-cached writes:
- A write which does not extend the EOF.
- A write which extends the EOF.
What actions does the FSD take to ensure that the contents in the cache are updated and same as that present on the disk? Or does Cache contain different data than the one present on the disk in these conditions?
Thanks.
Tushar A.
IIRC in this scenario the noncached write will be turned to a cached write
followed by immediate flush.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntfsd…
> Hi!
> Suppose an application ‘A’ opens a file in cached mode and start performing
read/write operations on it. And then an application ‘B’ opens the same file in
Non-cached mode and issues write request to it… To be specific, lets say ‘B’
issues 2 non-cached writes:
> 1. A write which does not extend the EOF.
> 2. A write which extends the EOF.
>
> What actions does the FSD take to ensure that the contents in the cache are
updated and same as that present on the disk? Or does Cache contain different
data than the one present on the disk in these conditions?
>
> Thanks.
> Tushar A.
>
Sir you said “IIRC in this scenario the noncached write will be turned to a cached write
followed by immediate flush.”
Does this mean that it is upto the FSD to decide whether to treat NON-CACHED I/O as PURELY NON-CACHED?
And in this case the FSD will forward the request to the Cache manager. The contents of the cache will be updated. And later, the file contents will be flushed. In short this is similar to what happens when you open the file with FILE_WRITE_THROUGH. First the cache contents are updated and then immediately contents are flushed. Right?
One more question: How is the cache immediately flushed? Does the FSD call CcFlushCache?
Thanks and regards,
Tushar A.
It depends from implementation.
Also there is second case for the noncached write to a cached file - the
written region is flushed and purged (that results to recursive paging
write) and after that the noncashed write is occurred.
One more question: How is the cache immediately flushed? Does the FSD
call CcFlushCache?
Yes
–
Best regards,
Vladimir Zinin
mailto:xxxxx@gmail.com
xxxxx@gmail.com wrote:
Sir you said “IIRC in this scenario the noncached write will be turned to a cached write
followed by immediate flush.”
Does this mean that it is upto the FSD to decide whether to treat NON-CACHED I/O as PURELY NON-CACHED?
And in this case the FSD will forward the request to the Cache manager. The contents of the cache will be updated. And later, the file contents will be flushed. In short this is similar to what happens when you open the file with FILE_WRITE_THROUGH. First the cache contents are updated and then immediately contents are flushed. Right?
One more question: How is the cache immediately flushed? Does the FSD call CcFlushCache?
Thanks and regards,
Tushar A.
You could also issue a flush to the range to be written, using
CcFlushCache, then write straight to the disk the non-cached write. It
is really up to the file system implementer. I believe FastFat does it
this way.
-Jeff
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Friday, September 07, 2007 7:30 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Can someone explain this scenario?
Sir you said “IIRC in this scenario the noncached write will be turned
to a cached write
followed by immediate flush.”
Does this mean that it is upto the FSD to decide whether to treat
NON-CACHED I/O as PURELY NON-CACHED?
And in this case the FSD will forward the request to the Cache manager.
The contents of the cache will be updated. And later, the file contents
will be flushed. In short this is similar to what happens when you open
the file with FILE_WRITE_THROUGH. First the cache contents are updated
and then immediately contents are flushed. Right?
One more question: How is the cache immediately flushed? Does the FSD
call CcFlushCache?
Thanks and regards,
Tushar A.
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: xxxxx@emc.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
> You could also issue a flush to the range to be written, using
CcFlushCache, then write straight to the disk the non-cached write.
Indeed, you can do it this way. However, as they say, if you can easily put peas up your nose, it does not necessarily mean that you should. First you have to wait until CcFlushCache() returns, and then send an IRP, i.e. do 2 disk operations, instead of one. Is not it better to update the cache first and then immediately flush it???
Anton Bassov
Thank you everybody,
This means that in MOST of the FSDs first the cache will be updated and then the cache will be flushed to disk.
If i am correct, the FSD identifies the fact that it has to update the cache by looking into the value of SharedCacheMap. If it is not NULL, this means that there is some File Object via which the file has been opened in cached mode and the cache needs to be updated.
Right?
One more question:
What happens in the same scenario, if instead of Write, an application issues a request to increase file size by SetFilePointer and SetEndOfFile?
> One more question:
What happens in the same scenario, if instead of Write, an application issues a
request to increase file size by SetFilePointer and SetEndOfFile?
Think about it yourself…
Just ask yourself a question - how does a given operation affect the cache coherency??? At this point
you will understand what is the most reasonable approach for FSD to take (although it does not mean that FSD will *actually*take it). SetFilePointer() and increase of file size by SetEndOfFile()
don’t affect the coherency of the existing cache at all, so that it just does not make sense to do anything here. If file gets truncated as a result of SetEndOfFile() calls, a part of the cache becomes invalid, so that it makes sense just to discard it with CcPurgeCacheSection()…
Anton Bassov
Why cache empty space? The CacheManager has to be informed but that
interface between the CC and the FSD is well defined. How the FSD knows
there is caching present is easy for the FSD because it called CC in the
beginning. It may be on any file object for that file stream. There are
many variations and someone in this field must have test programs that use
the Win32 and Native APIs to create, read, and write files using varied
options. There is a test tool available that allows you to do this
manually, but after you see what needs testing you should write your own to
automate the tests. You can use the Process Explorer to see how files are
accessed as you try various programs. Start with the Office line being sure
to use several different versions. Make sure you use Notepad too, but since
it limits the file size, you will need your own program to test with larger
files.
Look at CreateFile in MSDN. Write down every different option for each
parameter. Think and think some more. Match your tracing output against
all the options. You will find some that cannot be mapped because they are
only available from the Native API. A driver can use them by calling
ZwCreateFile and there are books that describe the Native API. This stuff
isn’t easy unless you have OS source code access and even then it won’t be
quick because the options and how they combine are so numerous.
–
David J. Craig
Engineer, Sr. Staff Software Systems
Broadcom Corporation
wrote in message news:xxxxx@ntfsd…
> Thank you everybody,
> This means that in MOST of the FSDs first the cache will be updated and
> then the cache will be flushed to disk.
> If i am correct, the FSD identifies the fact that it has to update the
> cache by looking into the value of SharedCacheMap. If it is not NULL, this
> means that there is some File Object via which the file has been opened in
> cached mode and the cache needs to be updated.
> Right?
> One more question:
> What happens in the same scenario, if instead of Write, an application
> issues a request to increase file size by SetFilePointer and SetEndOfFile?
>
Thank you Anton,
This means that when the application ‘B’ (which had opened the file in non-cached mode) increases file size by SetFilePointer and SetEndOfFile, the FSD just allocates extra disk space and does not overwrite the contents present at that location… Right?
And when application ‘A’( which has opened the file in cached mode) tries to access that data, a page fault occurs and that data is read from the disk…
Right?
> This means that when the application ‘B’ (which had opened the file in
non-cached mode) increases file size by SetFilePointer and SetEndOfFile, the FSD
just allocates extra disk space and does not overwrite the contents present at
that location… Right?
And when application ‘A’( which has opened the file in cached mode) tries to
access that data, a page fault occurs and that data is read from the disk…
Right?
Please note everything is up to FSD - not-so-well designed FSD may invalidate the cache at the time
when it is not really needed; do 2 disk operations when only one is needed; etc,etc,etc…
In any case, you should not make any assumption about the underlying FSD when writing your filters - it may cost you dearly…
Anton Bassov