File writes (fwrite) take very long time sometimes

NTDEV
1

Hi all,
I am performing a scatter gather DMA from FPGA to PC on windows XP, x86 architecture. The FPGA is continuously sending video data and I am allocating 8kb of buffer from my driver end. Once I get 8kb chunk from FPGA, I am transferring the data into a file using fwrite ( fwrite(buffer, 8k , 1 , file pointer) in append mode ) and I repeat this 8kb DMA transfer + file writes continuously.

Problem regarding performance:

  1. For the performance to be fine, DMA of 8kb + file writes should happen within 250 microseconds. Ideally dma of 8kb takes around 50 microseconds and file writes around 20 microseconds for every 8kb. This will be ok, but sometime after many such 8kb transfers and file writes, the file write (fwrite) takes upto 1000 - 10000 microseconds to write 8kb (instead of 20 microseconds) suddenly, due to which I lose my data coming from the FPGA continuously. I really can’t control the data from the FPGA end since it is a high performance video data coming at a faster rate.

So I would like to know the reason why fwrite takes so much time sometimes? Is there a way I can handle this file write issue?

Thanks

2

Run WPA and see what is happening . I would guess the system is flushing tlbs or memory because it has reached a starvation threshold, but that is just a guess. Instead of sending solitary 8k chunks, sent many 8k chunks simultaneously so that when one finishes, you have the next in the queue ready to go without the app needing to immediately respond with a chunk

d

Bent from my phone

From: xxxxx@tataelxsi.co.inmailto:xxxxx
Sent: ?5/?5/?2013 9:48 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: [ntdev] File writes (fwrite) take very long time sometimes

Hi all,
I am performing a scatter gather DMA from FPGA to PC on windows XP, x86 architecture. The FPGA is continuously sending video data and I am allocating 8kb of buffer from my driver end. Once I get 8kb chunk from FPGA, I am transferring the data into a file using fwrite ( fwrite(buffer, 8k , 1 , file pointer) in append mode ) and I repeat this 8kb DMA transfer + file writes continuously.

Problem regarding performance:

1. For the performance to be fine, DMA of 8kb + file writes should happen within 250 microseconds. Ideally dma of 8kb takes around 50 microseconds and file writes around 20 microseconds for every 8kb. This will be ok, but sometime after many such 8kb transfers and file writes, the file write (fwrite) takes upto 1000 - 10000 microseconds to write 8kb (instead of 20 microseconds) suddenly, due to which I lose my data coming from the FPGA continuously. I really can’t control the data from the FPGA end since it is a high performance video data coming at a faster rate.

So I would like to know the reason why fwrite takes so much time sometimes? Is there a way I can handle this file write issue?

Thanks

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

3

> Hi all,

I am performing a scatter gather DMA from FPGA to PC on windows XP, x86
architecture. The FPGA is continuously sending video data and I am
allocating 8kb of buffer from my driver end. Once I get 8kb chunk from
FPGA, I am transferring the data into a file using fwrite (
fwrite(buffer, 8k , 1 , file pointer) in append mode ) and I repeat this
8kb DMA transfer + file writes continuously.

I am not aware of any hard drive that can guarantee the write happens
within the time span you give. Even with a cache, there can be a problem.
One issue here is that “many” is a meaningless word. Some cultures count
“1, 2, 3, many”, and some of use think that “many” doesn’t lose meaning
until it exceeds tens of millions. So the “many” could be the point where
you filled up the disk drive cache, and it now has to start delaying while
it writes the cached data to the magnetic surface. Have a quantity of
“many” would help.

Also, fwrite is a lousy way to handle it; the C library does a lot of
internal buffering to handle transport. Just go for the raw WriteFile.

Then, you should be handling it asynchronously, so the requests can queue
in the kernel. The important thing is that you get control back quickly;
when it actually completes is not nearly as important as getting back
fast. When I was confronted by a similar performance problem, I solved it
by queuing up a bunch of ReadFile requests to the device, so with 50 I was
able to keep it within its realtime window to read data. You should
expect a write to take between 5000 and 10000 microseconds (typical disk
delays including rotational delays and small-step seek times; sometimes
+/-1 cylinder is very fast, 2ms for some disks I looked at; sometimes a
larger range can also be fast, such as up to +/-8 or similar. Oh, yes, if
your disk is badly fragmented, you’re screwed already)

Also, since your app is a passive-level thread, it is subject to
preemption by other threads of higher priority, such as the file system’s
writeback thread(s), so you really need to expect that when your 8K buffer
is ready, your app may not be. Which is why the queue of ReadFiles helps
a lot.

A warning, though: according to the experts on this NG, the order in which
asynchronous requests are sent back to the app is not necessarily the
order in which the driver completed them. Therefore, you will need to put
a “sequence number” in as a “packet header” (32 bits would be great), and
you may need to “reassemble” packets. Everything has a price.

For time-sensitive high-performance devices, synchronous I/O is usually
the kiss of death; you are at the mercy of the app-to-kernel transition
loop, and if you place another synchronous I/O in this path, forget it.
So you do a read from your device, and you must immediatly re-queue the
read (to a different buffer; you need a buffer ring with as many buffers
as you have concurrent ReadFiles), THEN do the write, asynchronously. And
you are STILL at the mercy of the scheduler, so it pays to have a lot of
ReadFiles down there to keep up with the data rate. So your first plan
should be to ditch all synchronous I/O to the device, and all synchronous
I/O to the disk.
joe

You have not said what the incoming data rate is. How much data does 8K
represent? How many 8K buffers are required to get a single frame in?
Maybe you should be thinking of your driver in terms of being able to
buffer an entire frame, where a frame is some large multiple of 8K.

You did not tell us what priority boost you used when you completed the
request. It might be easier to suggest answers if we had the necessary
information.

If you are running on Vista or later, look into the Multimedia Scheduling
Service (MMSC is at least a substring of the acronym, which I no longer
recall). This allows you to run a user level thread in a “normal” process
in the “time sensitive” priority ranges, 16…26 (the service itself runs a
priority 27, so it blocks you from any higher priority).

Problem regarding performance:

  1. For the performance to be fine, DMA of 8kb + file writes should happen
    within 250 microseconds. Ideally dma of 8kb takes around 50 microseconds
    and file writes around 20 microseconds for every 8kb. This will be ok, but
    sometime after many such 8kb transfers and file writes, the file write
    (fwrite) takes upto 1000 - 10000 microseconds to write 8kb (instead of 20
    microseconds) suddenly, due to which I lose my data coming from the FPGA
    continuously. I really can’t control the data from the FPGA end since it
    is a high performance video data coming at a faster rate.

So I would like to know the reason why fwrite takes so much time
sometimes? Is there a way I can handle this file write issue?

Thanks

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

4

I made an error in my explanation; your buffer ring must have enough
buffers to handle all the outstanding ReadFile requests AND enough to do
“packet reassembly” of the packets, so it may need a much larger number of
available buffers.

Note that “a whole whopping lot” is bigger than “many”. Neither is a
useful description when you need to know the actual numbers. But once
“many” seems inadequate, in informal discussions I will use the first
term. But I wouldn’t plug it into a formula.

Note that the time to switch buffers in the buffer queue is on the order
of a small integer number of microseconds in the worst case, whereas you
can assume that the app-kernel-app-kernel loop (read device, write to
disk) is going to have time constants somewhere in the tens of
milliseconds under optimal conditions and with occasional forays into
small number of hundreds of milliseconds under nearly-worst-case
conditions, and tens of hours under absolutely-worst-case conditions (I’m
not kidding). You can write for optimal conditions, dismiss
absolutely-worst-case scenarios, and that still leaves you with a lot of
timings that exceed 250usec. Async I/O can help, but it may not solve all
the problems (my app had a menu item that chose the number of ReadFiles to
send down; at 40, we still saw the occasional problem; at 50, we could run
for a whole day without a problem. The device was a high-performance
streaming data source (NDA) which would lose, unrecoverably, if it overran
the onboard buffer. And we could not lose any data).
joe

> Hi all,
> I am performing a scatter gather DMA from FPGA to PC on windows XP, x86
> architecture. The FPGA is continuously sending video data and I am
> allocating 8kb of buffer from my driver end. Once I get 8kb chunk from
> FPGA, I am transferring the data into a file using fwrite (
> fwrite(buffer, 8k , 1 , file pointer) in append mode ) and I repeat
> this
> 8kb DMA transfer + file writes continuously.
>
I am not aware of any hard drive that can guarantee the write happens
within the time span you give. Even with a cache, there can be a problem.
One issue here is that “many” is a meaningless word. Some cultures count
“1, 2, 3, many”, and some of use think that “many” doesn’t lose meaning
until it exceeds tens of millions. So the “many” could be the point where
you filled up the disk drive cache, and it now has to start delaying while
it writes the cached data to the magnetic surface. Have a quantity of
“many” would help.

Also, fwrite is a lousy way to handle it; the C library does a lot of
internal buffering to handle transport. Just go for the raw WriteFile.

Then, you should be handling it asynchronously, so the requests can queue
in the kernel. The important thing is that you get control back quickly;
when it actually completes is not nearly as important as getting back
fast. When I was confronted by a similar performance problem, I solved it
by queuing up a bunch of ReadFile requests to the device, so with 50 I was
able to keep it within its realtime window to read data. You should
expect a write to take between 5000 and 10000 microseconds (typical disk
delays including rotational delays and small-step seek times; sometimes
+/-1 cylinder is very fast, 2ms for some disks I looked at; sometimes a
larger range can also be fast, such as up to +/-8 or similar. Oh, yes, if
your disk is badly fragmented, you’re screwed already)

Also, since your app is a passive-level thread, it is subject to
preemption by other threads of higher priority, such as the file system’s
writeback thread(s), so you really need to expect that when your 8K buffer
is ready, your app may not be. Which is why the queue of ReadFiles helps
a lot.

A warning, though: according to the experts on this NG, the order in which
asynchronous requests are sent back to the app is not necessarily the
order in which the driver completed them. Therefore, you will need to put
a “sequence number” in as a “packet header” (32 bits would be great), and
you may need to “reassemble” packets. Everything has a price.

For time-sensitive high-performance devices, synchronous I/O is usually
the kiss of death; you are at the mercy of the app-to-kernel transition
loop, and if you place another synchronous I/O in this path, forget it.
So you do a read from your device, and you must immediatly re-queue the
read (to a different buffer; you need a buffer ring with as many buffers
as you have concurrent ReadFiles), THEN do the write, asynchronously. And
you are STILL at the mercy of the scheduler, so it pays to have a lot of
ReadFiles down there to keep up with the data rate. So your first plan
should be to ditch all synchronous I/O to the device, and all synchronous
I/O to the disk.
joe

You have not said what the incoming data rate is. How much data does 8K
represent? How many 8K buffers are required to get a single frame in?
Maybe you should be thinking of your driver in terms of being able to
buffer an entire frame, where a frame is some large multiple of 8K.

You did not tell us what priority boost you used when you completed the
request. It might be easier to suggest answers if we had the necessary
information.

If you are running on Vista or later, look into the Multimedia Scheduling
Service (MMSC is at least a substring of the acronym, which I no longer
recall). This allows you to run a user level thread in a “normal” process
in the “time sensitive” priority ranges, 16…26 (the service itself runs a
priority 27, so it blocks you from any higher priority).
> Problem regarding performance:
>
> 1. For the performance to be fine, DMA of 8kb + file writes should
> happen
> within 250 microseconds. Ideally dma of 8kb takes around 50 microseconds
> and file writes around 20 microseconds for every 8kb. This will be ok,
> but
> sometime after many such 8kb transfers and file writes, the file write
> (fwrite) takes upto 1000 - 10000 microseconds to write 8kb (instead of
> 20
> microseconds) suddenly, due to which I lose my data coming from the FPGA
> continuously. I really can’t control the data from the FPGA end since it
> is a high performance video data coming at a faster rate.
>
> So I would like to know the reason why fwrite takes so much time
> sometimes? Is there a way I can handle this file write issue?
>
> Thanks
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

5

@Doran & Joseph:

The input rate from FPGA is 250 microsecond/8kb.

Should I use WriteFile() and ReadFile(), instaed of fwrite() and fread()? Will it improve performance?

Thanks

6

Yes, the underlying API will be a little faster, but not enough to explain the long delay. You need to have many simultaneous ios pending in the driver, I don’t think fwrite affords async overlapped io

d

Bent from my phone

From: xxxxx@tataelxsi.co.inmailto:xxxxx
Sent: ?5/?6/?2013 8:56 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] File writes (fwrite) take very long time sometimes

@Doran & Joseph:

The input rate from FPGA is 250 microsecond/8kb.

Should I use WriteFile() and ReadFile(), instaed of fwrite() and fread()? Will it improve performance?

Thanks

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

7

@Doran:
1 more thing. Since it was a video data coming in, I used to store the file in binary mode (example: fopen("“filename.264)”,“wb”). I am not having any such provisions for WriteFile() and Readfile() operations since CreateFile() call does not have a field to specify binary mode. I tried saving the file in ‘.264’ mode, but unable to read the saved file. So is there any provision to save it in binary mode (filename.264)?

Thanks

8

ReadFile/WriteFile are only concerned with raw bytes (“binary” mode), they don’t care about file contents.

If the saved file is corrupted, you (your device) may have dropped data.

9

xxxxx@tataelxsi.co.in wrote:

The input rate from FPGA is 250 microsecond/8kb.

In other words, 32 MB/s. Most (but not all) disks from the last few
years should be able to keep up with that rate, at least for a while.

Should I use WriteFile() and ReadFile(), instaed of fwrite() and fread()? Will it improve performance?

Do you understand the difference between them? WriteFile and ReadFile
are the operating system primitives. They go directly to the driver.
fread and fwrite are part of the C run-time library. When you call
fopen, it creates a buffer in memory. fwrite copies data into that
buffer. When the buffer fills, it calls WriteFile to send the buffer down.

1 more thing. Since it was a video data coming in, I used to store the file in binary mode (example: fopen("“filename.264)”,“wb”). I am not having any such provisions for WriteFile() and Readfile() operations since CreateFile() call does not have a field to specify binary mode.

Right. The binary/text mode thing is just a convention of the C
run-time library. The operating system primitives don’t worry about
lines. They just write the bytes – binary mode all the time.

I tried saving the file in ‘.264’ mode, but unable to read the saved file. So is there any provision to save it in binary mode (filename.264)?

What do you mean by “unable to read”? Did you actually look at the
bytes of the file? Are you absolutely sure you aren’t dropping some of
your data packets? That would make the file unreadable by media players.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

10

> @Doran & Joseph:

The input rate from FPGA is 250 microsecond/8kb.

Should I use WriteFile() and ReadFile(), instaed of fwrite() and fread()?
Will it improve performance?

As I pointed out, fwrite and fread do a lot up in user space. So I see no
reason to even consider them as an option. But more importantly, you need
to move to asynchronous I/O ASAP. It is not something easily retrofitted
into a large app, you have to design it in from the beginning. But if you
put a WriteFile to disk in the loop of read-write, you are doomed, because
WriteFile to disk is always synchronous, even if you asked for async,
until you have set the FILE_NO_BUFFERING flag (I think: check out my async
explorer from www.flounder.com/mvp_tips). Note that the use of this flag
requires you to write 512-byte sequences. There is no way a synchronous
WriteFile is going to complete in 250usec unless it is purely writing to
the cache, and that can’t be guaranteed. Actually, I’m not sure this
timing window can be met even if it IS writing to the cache. I would have
a separate thread for writing data, and I would post a message to it to do
the writing. I would use an IOCP for my interthread queue, because
PostMessage can lose data and will interfere with the GUI; see my essay on
using IOCPs on my MVP Tips site. Essentially, you have two
badly-mismatched I/O transports in your app, and I cannot imagine why the
failure to keep up surprises you. To meet that requirement, you need to
go around the transport loop every 250usec, which is impossible if it
involves a disk write, or your thread is preempted (guess what: disk
writes probably activate a much-higher-priority file system thread). You
have to prime the pump with a whole lot of ReadFiles to your realtime
device, and move everything out of the “loop” (which is now distributed
across a couple threads) that can slow that loop down. In principle, you
need as many ReadFiles as it will take to handle the longest delay in that
loop, under optimal conditions. And if you make the reader or writer
thread Priority 15, you will kill the GUI response unless you limit the
cores that thread runs on. I would partition that thread to a small
number of cores, and set the writer thread and GUI thread to the
complement of that set.

You are essentially asking Windows to have a real-time responsiveness that
it was never designed to support, and there is always the possibility that
no matter what you do, you are going to lose data. One important design
criterion for hardware is that it should have onboard buffers for several
hundred milliseconds of incoming data. Your hardware is badly mismatched
to Windows’ ability to deal with that data rate.
joe

joe

Thanks

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

11

Also, if you write while expanding the file, file expansion (append) is ALWAYS synchronous. Move WriteFile to a separate thread and write in BIG chunks (4MB), with FILE_FLAG_NO_BUFFERING. You don’t have to use FILE_FLAG_OVERLAPPED because file append is synchronous.

12

I jumped into this conversation a little late, so this may have already been suggested. The OP is apparently using an FPGA-based hardware to bring in data at 32 MB (or is it 32 Mb) per second in bursts every 250 usec.

A potential design option (no thought all the way through):

  1. the FPGA is the interface directly onto the PCIe (?) why not do it with DMA into a known buffer location.
  2. Have the FPGA/ buffer enough data to only burst the input at 2 or 4 ms intervals.
  3. Use a raw formatted disk which you control completely, ie, no file system, just raw bytes.
  4. Have the ISR for the FPGA device trigger the IST that queues the buffer to be written to the disk.
  5. Keep a circular buffer so that as soon as data come in, the next buffer is handed to the FPGA. This could possibly be done in the ISR since it is not much more than writting a register to tell the FPGA where to go next.

It’s not simple to accomplish. Tis scheme may not even work under Windows. it works under other (more RTOS-ish) OSes. The problem is not the data rate. Windows can easily handle 32 MB (or is it 32 Mb) per second without breaking a sweat on modern hardware. The problem is the 250uS burst intervals.

Another though: Perhaps the ISR could batch these bursts and only fire every millisecond or 4 milliseconds. It would have to keep buffers ready to hand off. This is getting close to the edge of what is kosher to do in an ISR so no guarantees.

Greg

— xxxxx@flounder.com wrote:

From: xxxxx@flounder.com
To: “Windows System Software Devs Interest List”
Subject: RE:[ntdev] File writes (fwrite) take very long time sometimes
Date: Tue, 7 May 2013 19:54:00 -0400 (EDT)

> @Doran & Joseph:
>
> The input rate from FPGA is 250 microsecond/8kb.
>
> Should I use WriteFile() and ReadFile(), instaed of fwrite() and fread()?
> Will it improve performance?
>
As I pointed out, fwrite and fread do a lot up in user space. So I see no
reason to even consider them as an option. But more importantly, you need
to move to asynchronous I/O ASAP. It is not something easily retrofitted
into a large app, you have to design it in from the beginning. But if you
put a WriteFile to disk in the loop of read-write, you are doomed, because
WriteFile to disk is always synchronous, even if you asked for async,
until you have set the FILE_NO_BUFFERING flag (I think: check out my async
explorer from www.flounder.com/mvp_tips). Note that the use of this flag
requires you to write 512-byte sequences. There is no way a synchronous
WriteFile is going to complete in 250usec unless it is purely writing to
the cache, and that can’t be guaranteed. Actually, I’m not sure this
timing window can be met even if it IS writing to the cache. I would have
a separate thread for writing data, and I would post a message to it to do
the writing. I would use an IOCP for my interthread queue, because
PostMessage can lose data and will interfere with the GUI; see my essay on
using IOCPs on my MVP Tips site. Essentially, you have two
badly-mismatched I/O transports in your app, and I cannot imagine why the
failure to keep up surprises you. To meet that requirement, you need to
go around the transport loop every 250usec, which is impossible if it
involves a disk write, or your thread is preempted (guess what: disk
writes probably activate a much-higher-priority file system thread). You
have to prime the pump with a whole lot of ReadFiles to your realtime
device, and move everything out of the “loop” (which is now distributed
across a couple threads) that can slow that loop down. In principle, you
need as many ReadFiles as it will take to handle the longest delay in that
loop, under optimal conditions. And if you make the reader or writer
thread Priority 15, you will kill the GUI response unless you limit the
cores that thread runs on. I would partition that thread to a small
number of cores, and set the writer thread and GUI thread to the
complement of that set.

You are essentially asking Windows to have a real-time responsiveness that
it was never designed to support, and there is always the possibility that
no matter what you do, you are going to lose data. One important design
criterion for hardware is that it should have onboard buffers for several
hundred milliseconds of incoming data. Your hardware is badly mismatched
to Windows’ ability to deal with that data rate.
joe

joe
> Thanks
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

13

>…until you have set the FILE_NO_BUFFERING flag…

…Note that the use of this flag requires you to write 512-byte sequences…

We no longer live in a world where disk sector size is always 512 bytes. Unbuffered I/O will need to query the sector size and do I/O in sector size chunks.

Jan

14

>>…until you have set the FILE_NO_BUFFERING flag…

>…Note that the use of this flag requires you to write 512-byte
> sequences…

We no longer live in a world where disk sector size is always 512 bytes.
Unbuffered I/O will need to query the sector size and do I/O in sector
size chunks.

I believe the API states that the write must be multiples of 512. What
happens below that is unspecified. However, this subject is addressed in
detail, including 4K sectors, in

http://msdn.microsoft.com/en-us/library/windows/desktop/cc644950(v=vs.85).aspx

I should have said “multiples of 512-byte”. If you assume 4K sectors, you
will lose. Yes, you should query.

As already pointed out, the data rate as such is not the problem, but the
250usec interrupt rate is. What it imposes is that synchronous I/O is Not
Your Friend.
joe

Jan

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

15

>3. Use a raw formatted disk which you control completely, ie, no file system,
just raw bytes.

32 MB/s was not a big deal for a ATA disk 5 years ago on NTFS. No need to use raw disk.

If you’re concerned with metadata update overhead, you can preallocate a big file.

16

On 08-May-2013 06:00, Jan Bottorff wrote:

> …until you have set the FILE_NO_BUFFERING flag…
> …Note that the use of this flag requires you to write 512-byte sequences…

We no longer live in a world where disk sector size is always 512 bytes. Unbuffered I/O will need to query the sector size and do I/O in sector size chunks.

Jan

Not only this, but the alignment of items in data files written
with unbuffered i/o should be considered, if the data is “structured”.
Suppose you create a file on a disk with 512-byte sector,
then move it to 4K sector, and some offsets that were sector-aligned
are no longer. So it may be worth to simply use 4K units always (which
is the page size as well.) Until the next sector size change :wink:

– pa