Serialization of I/O operations

NTDEV
1

Hello,

It is important for certain applications to maintain the execution
order of I/O operations, e.g. database servers. Our concern is related
to the execution order of I/O requests issued by such an application,
and, as a result, which implementations are valid at the storage
driver level.

Example: say an application (from within a single thread) issues 3
consecutive overlapped I/O requests: write(A), read, write(B). Both
the writes and the read refer to the same location in the same file,
and access to the file is done using the same handle. If the requests
are performed in the order issued, the read is guaranteed to return A.
However, if they are handled out of order, it may be possible for the
read to return B, A, or the value that was present in that location
before write(A) (all up to the specific non-deterministic timing
scenario that occurred when the specific sequence was issued).

Clearly, any application that requires proper I/O operation ordering
has to find a resolution for the above issue:

a. It is possible that the application itself will avoid any
possible contention, i.e. in the above example, detect the reference
to the same location in file of the 3 requests, and delay the issuance
of the read operation until after write(A) returns, and defer the
issuance of write(B) until the read returns.

b. It is also possible that the operating system will take care
of the execution order:

a. The file system could resolve the contentions and synchronize
calls whenever inconsistent results may be returned due to out of
order execution.

b. The file system may rely on the I/O manager to do the work.

c. The entire set of upper layers (application, FS, I/O manager)
may assume that the driver processes the I/O requests in the order
received (by the driver).

d. ?and maybe there are other modules that take care of
synchronization along this stack?

We are implementing a storage device driver, and would like to take
advantage of some performance enhancements that we can achieve if we
execute I/O operations out of order. We want to know if this is
“allowed” and has absolutely no risk of causing any unexpected
behavior on the application level (which may lead to data corruption,
e.g. in database servers).

Alex

2

Applications doing overlapped I/O operations are responsible for managing collisions between I/O operations. If an application issues two writes to the same resource at the same time the result is indeterminate. Likewise with a read and a write.

So as long as the operations are all being initiated by clients you can reorder them. If you interject your own operations into the mix then you should probably make sure that your operations don’t collide with those of the application. For example if you are caching data and delaying writes to the underlying persistent store it becomes your responsibility to ensure that if you cache the data from write A and then receive write B that you don’t write A to the backing store after you write B. but that’s basic cache coherency.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Krinov
Sent: Wednesday, March 05, 2008 12:39 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Serialization of I/O operations

Hello,

It is important for certain applications to maintain the execution
order of I/O operations, e.g. database servers. Our concern is related
to the execution order of I/O requests issued by such an application,
and, as a result, which implementations are valid at the storage
driver level.

Example: say an application (from within a single thread) issues 3
consecutive overlapped I/O requests: write(A), read, write(B). Both
the writes and the read refer to the same location in the same file,
and access to the file is done using the same handle. If the requests
are performed in the order issued, the read is guaranteed to return A.
However, if they are handled out of order, it may be possible for the
read to return B, A, or the value that was present in that location
before write(A) (all up to the specific non-deterministic timing
scenario that occurred when the specific sequence was issued).

Clearly, any application that requires proper I/O operation ordering
has to find a resolution for the above issue:

a. It is possible that the application itself will avoid any
possible contention, i.e. in the above example, detect the reference
to the same location in file of the 3 requests, and delay the issuance
of the read operation until after write(A) returns, and defer the
issuance of write(B) until the read returns.

b. It is also possible that the operating system will take care
of the execution order:

a. The file system could resolve the contentions and synchronize
calls whenever inconsistent results may be returned due to out of
order execution.

b. The file system may rely on the I/O manager to do the work.

c. The entire set of upper layers (application, FS, I/O manager)
may assume that the driver processes the I/O requests in the order
received (by the driver).

d. …and maybe there are other modules that take care of
synchronization along this stack…

We are implementing a storage device driver, and would like to take
advantage of some performance enhancements that we can achieve if we
execute I/O operations out of order. We want to know if this is
“allowed” and has absolutely no risk of causing any unexpected
behavior on the application level (which may lead to data corruption,
e.g. in database servers).

Alex

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer