Is there some reason you just don’t open a file with buffering turned off,
which will perform almost the same as the direct write you talk about. If
you preallocate the file, an unbuffered write will essentially be some
calculations to determine the correct disk address and then a direct DMA of
the data from your application buffer.
Your second question seems like you WANT to OS to do buffering, undoing all
the trouble you want to go through to do direct writes.
How to get best performance will depend on what your I/O patterns look like.
If you do a number of small writes, you would want to let the Windows
buffering system do its job. If you have large writes, then you might get
better performance without buffering. Either way, you should almost
certainly open the file for overlapped I/O, and initiate multiple I/O’s in
parallel. These will get queued down into the disk drive or RAID controller,
and it can optimize physical I/O.
Fast I/O will often look something like:
- open the file with CreateFile, possible unbuffered, definitely overlapped
- get setup for async completion, like with I/O completion ports
- initiate a bunch of I/O’s, specifying the offset in each I/O
- do other stuff while the I/O’s run
- when each of those I/O completes, in the completion handing, initiate
another I/O, to keep the pipeline busy
- if you need to have a sync point, you need to wait for (a) outstanding
I/O to complete, as they will complete in an arbitrary order
- when done, wait for all the outstanding I/O to finish and close the file
You never would use SetFilePointer, you would specify the correct offset in
each overlapped request.
It’s likely a bad idea to not use a file on a file system, and almost
certainly a bad idea to think about writing any kind of driver. Performance
will almost totally be limited by controller/disk transfer rates and request
processing time, none of which will change if the I/O comes from a driver vs
an application. If you’re continually extending the file on every write, you
should preallocate some big chuck, and prune it back when you close it if
needed. Using a queue of overlapped I/O, you can easily load the storage
hardware to I/O saturation. Your time would be WAY better spend working on
caching algorithms for your specific data access patterns (assuming the OS’s
buffering is inappropriate), than writing a driver to do direct disk I/O.
You have done tests using all the normal ways of doing fast application I/O,
right? And assuming so, can you tell us what didn’t perform like you wanted.
Based on you question, it doesn’t sound like you have done enough
investigation to really know if or where a performance problem might be. You
might also look at adjusting your hardware environment, solid-state disks
can do wonders at improving random I/O.
Jan
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-414906-
xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, June 16, 2010 10:21 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] DASD - raw direct drive access in win2k8 restricted?
we’ve a need to write and read app data onto a raw disk. it’s beyond me
why
not just use a file system but whatever. The raw disk is most likely a
RAID LU,
i.e. not just a direct attached disk. At any rate, i imagine it’ll be
shown in
Windows Server 2008 as \.\physicaldrive[n], let us just say n=9. i am
picturing the app will do these:
- createfile physicaldrive9
- writefile nSectorsBytes
- readfile mSectorsBytes
- closehandle
question 1: in msdn createfile manual, it states Windows Server 2008
family
OSes restrict this kind of direct access - does this mean i can’t really
do the
above? someone mentioned a special kernel driver needs to be created to
work with the app for this to work in vista+ family OSes.
http://support.microsoft.com/kb/942448
question 2: if i do a writefile for 10 sectors followed by another
writefile of 5
sectors, does it write 15 sectors (assuming both succeeded)? i.e. even for
direct disk access (raw), will windows remember the last position of
writefile
and continue from there in the next write? i would think it would.
question 3: once i’ve written some data to the raw disk, i open it again
some
other time with createfile. if i want to overwrite sector 5, do i use
setfilepointer to get there and issue the writefile?
This may be a little off topic, apologize - i can repost to the right
forum if
advised. thanks.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer