Hi all,
Windows 7 NTFS does a lot of extra activities for meta data before and after read/write files. That causes poor performance. Is there anyway that we can distiguish and filter out the un-needed packages in our driver so that we can improve read/write performance? Would it hurt the files integrity if we do that?
Thanks.
xxxxx@gmail.com wrote:
Windows 7 NTFS does a lot of extra activities for meta data before and after read/write files. That causes poor performance. Is there anyway that we can distiguish and filter out the un-needed packages in our driver so that we can improve read/write performance? Would it hurt the files integrity if we do that?
What on earth leads you to imagine that the meta data is “un-needed?”
Every file system has a certain amount of bookkeeping to do. You have
to locate and track the used and free sectors. You have to maintain
file length and timestamps. You have to check and update permissions.
You have to maintain the journal so you can recover in case of disaster.
You can’t just eliminate that stuff and hope for the best. The file
system wouldn’t work. NTFS is not the bottleneck. We have a telemetry
system that records 100MB/sec to a striped set without even breaking a
sweat.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I tried to use fsutil to improve ntfs performance as suggestions on some articals, however, they don’t seem to help a bit.
On Fri, 13 May 2011 11:55:11 -0700
Tim Roberts wrote:
> What on earth leads you to imagine that the meta data is “un-needed?”
> Every file system has a certain amount of bookkeeping to do. You have
> to locate and track the used and free sectors. You have to maintain
> file length and timestamps.
There is certain meta-data that can be disabled: if you don’t have any
applications that depend on it, turning off last access timestamps can
reduce the amount of IO. It’s commonly done on unix, and can be changed
on Windows too, though it looks like it’s off by default on Windows 7.
–
Bruce Cran
xxxxx@gmail.com wrote:
I tried to use fsutil to improve ntfs performance as suggestions on some articals, however, they don’t seem to help a bit.
How are you testing the performance? What leads you to think that NTFS
is the bottleneck, as opposed to, say, the disk itself or the bus?
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I tried that, didn’t help much. I did some more sata traces. There are some traces showing 4K bytes transfers off all 0s, or all FF; there are some that have 1st Dword being FF00FFFF, 2nd Dword being 3FFF, all other are 0s. All of them (except when I tried to copy files bigger than 4KB) show 4KB transfer length.
Is there anyway, anything I can do in the driver to optimize read/write from NTFS? I have the queuedepth set that I beleieve it it the proper number. Running iometer on raw drive, the IOPS number is 150K, while running it on the volumn, the number goes down to 70K.
Thanks.
Do you have an antivirus app on your SUT that does live analysis, by any chance? That’s the biggest performance killer.
Before you ask the same question 5 times in the same thread (is there any way I can improve performance), you should answer Tim’s question: why do you think NTFS is the bottleneck?
Or in other words, what are you doing that needs more performance, and can you do it on another file system considerably faster?
Dejan.
xxxxx@gmail.com wrote:
I tried that, didn’t help much. I did some more sata traces. There are some traces showing 4K bytes transfers off all 0s, or all FF; there are some that have 1st Dword being FF00FFFF, 2nd Dword being 3FFF, all other are 0s. All of them (except when I tried to copy files bigger than 4KB) show 4KB transfer length.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.
Sorry, I replied to the wrong post back there 
What exact IOMeter settings are you using? If you get 70K IOPS you are either using cached I/O or you have quite a few SSDs which are more likely bottlenecked elsewhere.
xxxxx@gmail.com wrote:
Is there anyway, anything I can do in the driver to optimize read/write from NTFS? I have the queuedepth set that I beleieve it it the proper number. Running iometer on raw drive, the IOPS number is 150K, while running it on the volumn, the number goes down to 70K.
Thanks.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.
>beleieve it it the proper number. Running iometer on raw drive, the IOPS number is 150K
150K IOs per second is too much for modern hard disk.
200 or maybe 300 is realistic.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
“150K IOs per second is too much for modern hard disk.”
For sequential, 1 block-read asynchronous (multiple requests) I/O it’s normal.