assume we have:
- The file is opened for asynchronous I/O.
- A I/O completion port is associated with the file.
- we invoke asynchronous I/O operation on file (ApcContext != 0)
so how detect - will be queued a packet to the IOCP or not ?
this always need exactly know, because we can not free/dereference IO_STATUS_BLOCK or OVERLAPPED (and may be other resources) until I/O not finished.
usually this done in callback called when packet queued to IOCP (we can handle IOCP yourself, or use CreateThreadpoolIo or BindIoCompletionCallback)
if we know that will no packet queued to IOCP - need manually just call this callback with error status
if we use CreateThreadpoolIo callback, need call CancelThreadpoolIo, if will be no IOCP notification
look like documentation for StartThreadpoolIo and CancelThreadpoolIo give exactly answer, when will be no notification to IOCP (and we must call the CancelThreadpoolIo)
-
An overlapped (asynchronous) I/O operation fails (that is, the asynchronous I/O function call returns failure with an error code other than ERROR_IO_PENDING).
-
An asynchronous I/O operation returns immediately with success and the file handle associated with the I/O completion object has the notification mode FILE_SKIP_COMPLETION_PORT_ON_SUCCESS. The file handle will not notify the I/O completion port and the associated I/O callback function will not be called.
because win32 layer usually set error code and return false when NT_ERROR(status) (except special case STATUS_PENDING converted to ERROR_IO_PENDING) returned from Zw* api, for NT layer this mean:
- if api return NT_ERROR(status) ( status in range [0xC0000000, 0xFFFFFFFF] ) - no notification
- otherwise, if returned status in range [0, 0xC0000000) - will be
the range [0x80000000, 0xC0000000) bit problematic really
if such status returned from I/O manager, due invalid parameters and before actual call driver, will be no notification.
only one case which i know here - this is STATUS_DATATYPE_MISALIGNMENT 0x80000002 returned for example from ZwNotifyChangeDirectoryFile(ReadDirectoryChangesW ) if lpBuffer not DWORD-aligned.
in this case will be no completion.
interesting that win32 call
OVERLAPPED ov{};
ReadDirectoryChangesW(0, (void*)1, 1, TRUE, FILE_NOTIFY_VALID_MASK, 0, &ov, 0);
return TRUE, because !NT_ERROR(STATUS_DATATYPE_MISALIGNMENT) - win32 layer simply lost STATUS_DATATYPE_MISALIGNMENT error
from another case, if say NtQueryDirectoryFile return STATUS_NO_MORE_FILES (0x80000006) will be notification
but this is wrong in general case. really can be notification to IOCP even in case NT_ERROR(status) !
this is because FastIo which can be invoked before IRP create - can return TRUE with error final status.
say FastIoDeviceControl or FastIoLock can return TRUE (I/O finished) and final status is error
for example we can call LockFileEx or ZwLockFile on directory file - I/O manager call Fs FastIoLock implementation (say NtfsFastLock) and it just return TRUE with STATUS_INVALID_PARAMETER
so after such call we got STATUS_INVALID_PARAMETER. by documented rules - must not be notification in this case. but in real word it will be !
of course this is rarely situation, but it break general rule.
also in case SetFileCompletionNotificationModes - if we test it (FILE_SKIP_COMPLETION_PORT_ON_SUCCESS) with LockFileEx - will be no notification in this case.
so FILE_SKIP_COMPLETION_PORT_ON_SUCCESS prevent notification not only success return, but any synchronous return.
so more correct name it FILE_SKIP_COMPLETION_PORT_ON_SYNCHRONOUS and instead
A request returns success immediately without returning ERROR_PENDING
must be
A request returns immediately without returning ERROR_PENDING
POC of code with LockFileEx - https://pastebin.com/qWrMYvy4 , unfortunately too long for post here
so really correct detect will be or not notification in general case 100% reliable ?
i view next solution (but may be i mistake or exist better ?- in this and main question)
set IO_STATUS_BLOCK.Status = STATUS_PENDING; before api call (win32 layer always do this how i know)
check IO_STATUS_BLOCK.Status == STATUS_PENDING after api return code other than STATUS_PENDING (ERROR_IO_PENDING)
sense here in next - notification to IOCP will be when and only when I/O manager write back status to user mode IO_STATUS_BLOCK (if no FILE_SKIP_COMPLETION_PORT_ON_SYNCHRONOUS)
but we can not simply access IO_STATUS_BLOCK(or OVERLAPPED) after api call - because other thread can in concurrent execute callback with this IO_STATUS_BLOCK and can already free it
(this like I/O manager can not more access IRP after call driver, if no IRP_DEFER_IO_COMPLETION in flags - IRP can be already completed and free/reused)
solution here use reference counting on structure which incapsulate IO_STATUS_BLOCK(or OVERLAPPED).
we always create this structure (let name it user mode IRP) with 2 reference.
one reference we release after check result/IO_STATUS_BLOCK of API call (for detect - are we need manually invoke callback, call CancelThreadpoolIo )
another we release in callback
class NT_IRP : public IO_STATUS_BLOCK
{
//...
LONG m_dwRefCount;//=2 on init
NT_IRP() : m_dwRefCount(2)
{
Status = STATUS_PENDING, Information = 0;
}
void Release()
{
if (!InterlockedDecrement(&m_dwRefCount)) delete this;
}
VOID IOCompletionRoutine(NTSTATUS status, ULONG_PTR dwNumberOfBytesTransfered)
{
//..
Release();
}
static VOID CALLBACK _IOCompletionRoutine(NTSTATUS status, ULONG_PTR dwNumberOfBytesTransfered, PVOID ApcContext)
{
// we must pass NT_IRP pointer in place ApcContext in I/O call
reinterpret_cast<NT_IRP*>(ApcContext)->IOCompletionRoutine(status, dwNumberOfBytesTransfered);
}
void CheckNtStatus(NTSTATUS status, BOOL bSkippedOnSynchronous = FALSE)
{
// api completed synchronous (status != STATUS_PENDING)
// and
// bSkippedOnSynchronous or iosb not modified (Status == STATUS_PENDING)
if (status != STATUS_PENDING && (bSkippedOnSynchronous || Status == STATUS_PENDING))
{
IOCompletionRoutine(status, Information);
}
Release();
}
};
or for win32 case
class WIN32_IRP : public OVERLAPPED
{
//...
LONG m_dwRefCount;//=2 on init
WIN32_IRP() : m_dwRefCount(2)
{
Internal = STATUS_PENDING, InternalHigh = 0;
}
void Release()
{
if (!InterlockedDecrement(&m_dwRefCount)) delete this;
}
VOID IOCompletionRoutine(NTSTATUS status, ULONG_PTR dwNumberOfBytesTransfered)
{
//...
Release();
}
static VOID CALLBACK _IOCompletionRoutine(NTSTATUS status, ULONG_PTR dwNumberOfBytesTransfered, PVOID ApcContext)
{
// we must pass NT_IRP pointer in place ApcContext in I/O call
reinterpret_cast<NT_IRP*>(ApcContext)->IOCompletionRoutine(status, dwNumberOfBytesTransfered);
}
void CheckErrorCode(ULONG dwErrorCode, BOOL bSkippedOnSynchronous = FALSE)
{
// api completed synchronous (dwErrorCode != ERROR_IO_PENDING)
// and
// bSkippedOnSynchronous or iosb not modified (Internal == STATUS_PENDING)
if (dwErrorCode != ERROR_IO_PENDING && (bSkippedOnSynchronous || Internal == STATUS_PENDING))
{
IOCompletionRoutine(status, Information);
}
Release();
}
};
but solution look like too complex… are microsoft forget Fast Io case ?
also why say FastIoRead and FastIoWrite called only in case synchronous I/O (so not make problems here), but FastIoLock and FastIoDetachDevice called for any I/O ?
more logic from my look - or call FastIoLock and FastIoDetachDevice also only for synchronous file, FastIoRead and FastIoWrite - for any I/O too