Re: [ntdev] How to verify file write to a physical hard disk

NTDEV
1

Use FILE_FLAG_WRITE_THROUGH | FILE_FLAG_NO_BUFFERING with OVERLAPPED IO

If the hardware supports this, and since we are now finally rid of IDE drives generally it does, the IOP will not return until you can rely on persistent storage (or a broken driver you shout sue the makers of )

Sent from Surface Pro

From: xxxxx@rafael.co.il
Sent: ‎Tuesday‎, ‎February‎ ‎10‎, ‎2015 ‎11‎:‎28‎ ‎AM
To: Windows System Software Devs Interest List

Hi all,

I have a user mode process (written in C++) that needs to write a file to a local hard disk partition
and then verify that the file has been written to the physical media without errors.

Is this doable under Windows? How?

I ask the question in NTDEV because I know that hard disks have their own internal cache, so I
am worried that if I use the following sequence: [write - flush - read - compare], then the
read may return (at least some) data from the on-disk cache, and not from the physical media.
In case the physical media has errors, I will not be able to detect the corruption.

I think that if I can purge the on-disk cache then all the required data will be read from the
physical media. But I don’t know how to do that.

I am aware that Windows has its own caching as well.

Thank you,
Itai Handler

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

2

And what is FUA on read useful for? Buggy hardware caches perhaps?

Event Windows clustering shared quorum disks on a shared SCSI bus don’t need that

FUA on write however, it vital for applications like SQL server or anything else needing ACID properties

Sent from Surface Pro

From: Peter Wieland
Sent: ‎Tuesday‎, ‎February‎ ‎10‎, ‎2015 ‎6‎:‎42‎ ‎PM
To: Windows System Software Devs Interest List

I think the answer is that it was. I went to look since I couldn’t remember what we used to do, but it doesn’t look classpnp only sets FUA for WRITE requests. There is not an option, short of rolling your own CDBs, to force unit access on a read.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Tuesday, February 10, 2015 3:30 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to verify file write to a physical hard disk

I traced this once, and it IS set correctly. Or, well, it WAS.

Shove a SATA disk on a drive analyzer sometime, and sent it some FILE_FLAG_WRITE_THROUGH reads and writes. You *will* see FUA set, using standard Windows drivers and a commodity AHCI controller.

I can also tell you with certainty that there are disks/controllers/drivers that do not honor the FUA bit correctly. But, as I said in my initial reply, that’s really beyond your ability to control at that point and getting FUA set is the best you can do.

Peter
OSR
@OSRDrivers

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

Unless you can power down the disk, you really have no control and have to make that assumption.

Sent from Surface Pro

From: xxxxx@rafael.co.il
Sent: ‎Wednesday‎, ‎February‎ ‎11‎, ‎2015 ‎4‎:‎56‎ ‎AM
To: Windows System Software Devs Interest List

Hi all,

Assuming I use FILE_FLAG_WRITE_THROUGH, which results in a FUA sent to the disk
after the file write operation, how can I verify that the physical media contains a good
copy of the file?
For example, if I try to write “ABC” to the file, and somehow “ACC” gets to the media,
how can I detect this kind of error?
If I try to read the file from the disk for this verification purpose, will the read return
the data from the physical media (“ACC”) or from the on-disk cache (“ABC”)?
I understand from Peter’s message that FUA is not set during read operations so I believe that
the disk is allowed to return the data from its cache.
I am very curious how do backup software ensure that data is written correctly to the media.
Do they assume that the disk cache is small enough? I saw that some disk drives have pretty big caches.

Thank you,
Itai Handler

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer