I am seeing that if an application has a file opened non-buffer (SQL)
and doing reads and writes and another application opens the file
buffered and does reads. It slows the operation SQL operations way down.
Can anyone explain why and how to prevent this from happening?
Thanks,
Ken
If you have both non-buffered and buffered opens on the same file, then any
non-buffered operation has to flush and/or invalidate the cache to keep the
file consistent. If you look at the read and write paths in the FAT sample
in the WDK, you will see a lot of effort and code to handle these cases.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
wrote in message news:xxxxx@ntfsd…
I am seeing that if an application has a file opened non-buffer (SQL)
and doing reads and writes and another application opens the file
buffered and does reads. It slows the operation SQL operations way down.
Can anyone explain why and how to prevent this from happening?
Thanks,
Ken
Every non-cached write causes a cache purge; the next cached read will
cause a cache miss and will force paging I/O to satisfy the cached read.
Thus, cached I/O effectively has the same cost as non-cached I/O, with
the additional overhead of managing the cache. Even the non-cached path
must now pay the cost of constantly tearing down the cache.
Tearing down the cache impacts all CPUs on the system because those
addresses have to be invalidated on all CPUs. Thus, this further
creates a series of cross-processor messages to force this cache
invalidation.
Mixing cached and non-cached I/O does not generally have good
performance characteristics, but it is correct (coherent.) Note that
mixing non-cached and memory mapped I/O is not coherent.
Tony
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@emc.com
Sent: Monday, June 18, 2007 7:24 AM
To: ntfsd redirect
Subject: [ntfsd] Buffering vs. Non-Buffering
I am seeing that if an application has a file opened non-buffer (SQL)
and doing reads and writes and another application opens the file
buffered and does reads. It slows the operation SQL operations way down.
Can anyone explain why and how to prevent this from happening?
Thanks,
Ken
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
>Every non-cached write causes a cache purge; the next cached read will
cause a cache miss and will force paging I/O to satisfy the cached read.
From what I remember, if the FCB has at least 1 cached file object, then any
noncached write goes to the cache too and flushes it immediately. Is it not so?
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> I am seeing that if an application has a file opened non-buffer (SQL)
and doing reads and writes and another application opens the file
buffered and does reads. It slows the operation SQL operations way down.
Can anyone explain why and how to prevent this from happening?
When you use non-buffered IO, every read and write goes directly to the disk in context of a calling thread. When you use a buffered one, the system checks whether the operation can be accomplished without accessing the disk, and if not, read or write goes to the disk. However, it does not go directly to the disk in context of a calling thread, but, instead, is done by Memory Manager’s worker thread. Memory Manager works synchronously, so that you have to spend some time in a queue, which gives you an extra overhead. Therefore, the cost of disk access is higher for buffered IO. This is why specifying FILE_FLAG_NO_BUFFERING in CreateFile() call may give you a significant improvement on large reads and writes - by specifying this flag you avoid overhead, associated with the synchronous nature of Memory Manager’s operations.
Now look what happens in your case. As Tony and Don already explained to you, non-cached write
causes cache purge, so that the next cached read is going to require disk access. However, it is not going to get done straight away - instead, it will be done by Memory Manager’s worker thread, so that you have to wait in a queue. As a result, cached IO effectively turns into non-cached one
in terms of frequency of disk access, but still remains cached one in terms of costs of accessing the disk - as I told you already, it is higher for cached IO because of overhead, associated with the synchronous nature of Memory Manager’s operations. In other words, by combining buffered IO with non-buffered one, you get the worst features of each…
Anton Bassov