ReadFile Overlapped

Hi All,

This query is similar to recent thread : “Overlapped I/O problem”.
http://www.osronline.com/showthread.cfm?link=215367

Maybe a filter driver which will force sync completion can help here.
–pa

  • put a filter above your KMDF driver to handle the pending status.

As suggested by Pavel and Mark, I am using a filter driver (modified serenum as upper filter and KMDF as function) for usb-serial devices.

Using this approach, I am able to support old VB applications (which uses MSCOMM32.OCX).

However, in one of the other application, which uses overlapped i/o the same driver HANGS at the very first ReadFile call itself.
Below is part of the application code:


commtimeouts.ReadIntervalTimeout = MAXDWORD;
commtimeouts.ReadTotalTimeoutMultiplier = MAXDWORD;
commtimeouts.ReadTotalTimeoutConstant = MAXDWORD - 1;

overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);

ReadFile(handle, buffer, bytes, &received, &overlapped);

Here the application never comes out of “ReadFile”.

After further analysis: For the case of max “ReadIntervalTimeout”, function driver sets cancellation routine using WdfRequestMarkCancelable. And EvtRequestCancel function never gets called.

Later I was able to see the differential behavior. If I comment-out the part of upper filter changes (means commenting out: IoCopyCurrentIrpStackLocationToNext, IoSetCompletionRoutine and ReadCompletion at the upper filter), the cancellation callback “EvtRequestCancel” at the function driver gets called and application returns to the normal state.

Below is the code part of upper filter changes:


if(IrpStack->MajorFunction == IRP_MJ_READ) { //upper filter modification starts
IoCopyCurrentIrpStackLocationToNext(Irp);
KeInitializeEvent(&completionEvent, SynchronizationEvent, FALSE);
IoSetCompletionRoutine(Irp, ReadCompletion, &completionEvent, TRUE, TRUE, TRUE);
rval = IoCallDriver(pFdoData->TopOfStack, Irp);
if(rval == STATUS_PENDING) {
rval = KeWaitForSingleObject(&completionEvent, Executive, KernelMode, FALSE, NULL);
}
}
else {
IoSkipCurrentIrpStackLocation (Irp); //default behaviour
rval = IoCallDriver(pFdoData->TopOfStack, Irp); //default behaviour
}

//And the completion routine is:
ReadCompletion(IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp, IN PKEVENT ReadEvent)
{
KeSetEvent(ReadEvent, IO_NO_INCREMENT, FALSE);
return STATUS_SUCCESS;
}

Now the query is: By retaining the filter driver changes how to support the Overlapped IO Read call.
Means if there is no pending read data or actual device read happened, KMDF driver should come out with read cancelled state (EvtRequestCancel should get called, which is not happening!)

Please advice. Thanks in advance.

Your logic which makes the read synchronous needs to only make it synchronous when there is data available. Otherwise you are just blocking the whole time. That means you can’t add the “make it synchronous” code to an upper filter, it needs to be in the fdo.

d

debt from my phone


From: xxxxx@gmail.com
Sent: 11/8/2011 8:16 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] ReadFile Overlapped

Hi All,

This query is similar to recent thread : “Overlapped I/O problem”.
http://www.osronline.com/showthread.cfm?link=215367

Maybe a filter driver which will force sync completion can help here.
–pa

  • put a filter above your KMDF driver to handle the pending status.

As suggested by Pavel and Mark, I am using a filter driver (modified serenum as upper filter and KMDF as function) for usb-serial devices.

Using this approach, I am able to support old VB applications (which uses MSCOMM32.OCX).

However, in one of the other application, which uses overlapped i/o the same driver HANGS at the very first ReadFile call itself.
Below is part of the application code:


commtimeouts.ReadIntervalTimeout = MAXDWORD;
commtimeouts.ReadTotalTimeoutMultiplier = MAXDWORD;
commtimeouts.ReadTotalTimeoutConstant = MAXDWORD - 1;

overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);

ReadFile(handle, buffer, bytes, &received, &overlapped);

Here the application never comes out of “ReadFile”.

After further analysis: For the case of max “ReadIntervalTimeout”, function driver sets cancellation routine using WdfRequestMarkCancelable. And EvtRequestCancel function never gets called.

Later I was able to see the differential behavior. If I comment-out the part of upper filter changes (means commenting out: IoCopyCurrentIrpStackLocationToNext, IoSetCompletionRoutine and ReadCompletion at the upper filter), the cancellation callback “EvtRequestCancel” at the function driver gets called and application returns to the normal state.

Below is the code part of upper filter changes:


if(IrpStack->MajorFunction == IRP_MJ_READ) { //upper filter modification starts
IoCopyCurrentIrpStackLocationToNext(Irp);
KeInitializeEvent(&completionEvent, SynchronizationEvent, FALSE);
IoSetCompletionRoutine(Irp, ReadCompletion, &completionEvent, TRUE, TRUE, TRUE);
rval = IoCallDriver(pFdoData->TopOfStack, Irp);
if(rval == STATUS_PENDING) {
rval = KeWaitForSingleObject(&completionEvent, Executive, KernelMode, FALSE, NULL);
}
}
else {
IoSkipCurrentIrpStackLocation (Irp); //default behaviour
rval = IoCallDriver(pFdoData->TopOfStack, Irp); //default behaviour
}

//And the completion routine is:
ReadCompletion(IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp, IN PKEVENT ReadEvent)
{
KeSetEvent(ReadEvent, IO_NO_INCREMENT, FALSE);
return STATUS_SUCCESS;
}

Now the query is: By retaining the filter driver changes how to support the Overlapped IO Read call.
Means if there is no pending read data or actual device read happened, KMDF driver should come out with read cancelled state (EvtRequestCancel should get called, which is not happening!)

Please advice. Thanks in advance.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Thanks Doron for suggestions.

That means you can’t add the “make it synchronous” code to an upper filter, it
needs to be in the fdo.

d

The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

Maxim S. Shatskih

I did changes at the function driver level, now I can see Requests are not landing from EvtIoRead, instead I can see the requests from ReadEvtIoInCallerContext.
In the ReadEvtIoInCallerContext, for Read requests: I did changes to process the requests.

However still I can see the “STATUS_PENDING” coming and my MSCOMM32.OCX application failing with “READ ERROR”

Below is the code part containing changes:
VOID ReadEvtIoInCallerContext(IN WDFDEVICE Device, IN WDFREQUEST Request)

switch (params.Type) {
case WdfRequestTypeRead:

status = WdfRequestRetrieveOutputBuffer (Request, reqContext->Length, &reqContext->SystemBuffer, &bufLen);

UsbStartOrQueue(extension, Request, extension->ReadQueue, &extension->CurrentReadRequest, UsbStartRead);
}
break;

default:
status = WdfDeviceEnqueueRequest(Device, Request);

} //switch

I was able to see using IRP tracker, READ at the fdo level is STATUS_SUCCESS but
READ at the filter level (modified serenum) is STATUS_PENDING !?!?! Which I am not able to understand.

Also to compare this behaviour with default serial driver, I found interesting result.

This is a known issue with KMDF :frowning: It seems to always return
STATUS_PENDING and do async completion, even if the request is completed
without delay. Maybe a filter driver which will force sync completion
can help here.
–pa

KMDF returns STATUS_PENDING if the request was passed via the queue.
EvtIoRead/Write are below the queue, so, if the IRP got there, STATUS_PENDING was returned for it for sure.
The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

Maxim S. Shatskih

Here, I switched the default serial.sys with the KMDF version supplied in winddk - wdfserial.sys (from the Winddk 7600.16385.1) and loaded inbox serenum.sys

Using this MSCOMM32.OCX application nevers returns “READ ERROR”. And works perfectly well !?!?

So I put the IRPtracker and saw all the reads with STATUS_SUCCESS !?!?

Guess! wdfserial.sys is a KMDF driver, and should return STATUS_PENDING right?

IRPTracker log for wdfserial.sys:===================================

Call 0x8936E680-5550778 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550778 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Call 0x8936E680-5550779 (UNKNOWN) (0x8945E940) serenum DEVICE_CONTROL
Call 0x8936E680-5550779 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550779 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Comp 0x89FDF250-5550092 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x4
Call 0x8936E680-5550780 (UNKNOWN) (0x8945E940) serenum READ NORMAL
Call 0x8936E680-5550780 (0x8945E940) serenum \Device\Serial5 READ NORMAL
Comp 0x8936E680-5550780 \Device\Serial5 READ NORMAL SUCCESS, Info = 0x8

IRPTracker log for wdfserial.sys:===================================

re-posting as updates are rejected…

Thanks Doron for suggestions.

That means you can’t add the “make it synchronous” code to an upper filter, it
needs to be in the fdo.

d

The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

I did changes at the function driver level, now I can see Requests are not landing from EvtIoRead, instead I can see the requests from ReadEvtIoInCallerContext
In the ReadEvtIoInCallerContext, for Read requests: I did changes to process the requests.

However still I can see the “STATUS_PENDING” coming and my MSCOMM32.OCX application failing with “READ ERROR”

Below is the code part containing changes:
VOID ReadEvtIoInCallerContext(IN WDFDEVICE Device, IN WDFREQUEST Request)

switch (params.Type) {
case WdfRequestTypeRead:

status = WdfRequestRetrieveOutputBuffer (Request, reqContext->Length, &reqContext->SystemBuffer, &bufLen);

UsbStartOrQueue(extension, Request, extension->ReadQueue, &extension->CurrentReadRequest, UsbStartRead);
}
break;

default:
status = WdfDeviceEnqueueRequest(Device, Request);

} //switch

I was able to see using IRP tracker, READ at the fdo level is STATUS_SUCCESS but
READ at the filter level (modified serenum) is STATUS_PENDING !?!?!
Which I am not able to understand.

Also to compare this behaviour with default serial driver, I found interesting result.

This is a known issue with KMDF :frowning: It seems to always return
STATUS_PENDING and do async completion, even if the request is completed
without delay. Maybe a filter driver which will force sync completion
can help here.
–pa

KMDF returns STATUS_PENDING if the request was passed via the queue.
EvtIoRead/Write are below the queue, so, if the IRP got there, STATUS_PENDING was returned for it for sure.
The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

Maxim S. Shatskih

Here, I switched the default serial.sys with the KMDF version supplied in winddk - wdfserial.sys (from the Winddk 7600.16385.1) and loaded inbox serenum.sys

Using this MSCOMM32.OCX application nevers returns “READ ERROR”. And works perfectly well !?!?
So I put the IRPtracker and saw all the reads with STATUS_SUCCESS !?!?

Guess! wdfserial.sys is a KMDF driver, and should return STATUS_PENDING right?

IRPTracker log for wdfserial.sys:===================================

Call 0x8936E680-5550778 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550778 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Call 0x8936E680-5550779 (UNKNOWN) (0x8945E940) serenum DEVICE_CONTROL
Call 0x8936E680-5550779 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550779 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Comp 0x89FDF250-5550092 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x4
Call 0x8936E680-5550780 (UNKNOWN) (0x8945E940) serenum READ NORMAL
Call 0x8936E680-5550780 (0x8945E940) serenum \Device\Serial5 READ NORMAL
Comp 0x8936E680-5550780 \Device\Serial5 READ NORMAL SUCCESS, Info = 0x8

IRPTracker log for wdfserial.sys:===================================

Awaiting for comments on my updates…

Ya know, given the number of posts to this forum in a 24 hour period, with usually several active threads, I would never ever post such an silly comment without indicating just exactly what those updates are that are so important. Not everyone reads the thread from the OSR website, and since you decided NOT to include the thread history, I have absolutely no idea what updates are so very important to you.

Gary Little
H (952) 223-1349
C (952) 454-4629
xxxxx@comcast.net

On Nov 13, 2011, at 11:37 PM, xxxxx@gmail.com wrote:

Awaiting for comments on my updates…


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Thanks Gary,

I am using a PC from a network where only osr site is allowed.
Later when I saw reply to my mails from another network, it says my updates are rejected.

I am re-posting again with thread history (hoping this time wont see any “excess quoted lines suppressed” messages).

===================================
Thanks Doron for suggestions.

That means you can’t add the “make it synchronous” code to an upper filter, it
needs to be in the fdo.

d

The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

I did changes at the function driver level, now I can see Requests are not landing
from EvtIoRead, instead I can see the requests from ReadEvtIoInCallerContext
In the ReadEvtIoInCallerContext, for Read requests: I did changes to process the
requests.

However still I can see the “STATUS_PENDING” coming and my MSCOMM32.OCX application
failing with “READ ERROR”

Below is the code part containing changes:
VOID ReadEvtIoInCallerContext(IN WDFDEVICE Device, IN WDFREQUEST Request)

switch (params.Type) {
case WdfRequestTypeRead:

status = WdfRequestRetrieveOutputBuffer (Request, reqContext->Length,
&reqContext->SystemBuffer, &bufLen);

UsbStartOrQueue(extension, Request, extension->ReadQueue,
&extension->CurrentReadRequest, UsbStartRead);
}
break;

default:
status = WdfDeviceEnqueueRequest(Device, Request);

} //switch

I was able to see using IRP tracker, READ at the fdo level is STATUS_SUCCESS but
READ at the filter level (modified serenum) is STATUS_PENDING !?!?!
Which I am not able to understand.

Also to compare this behaviour with default serial driver, I found interesting result.

This is a known issue with KMDF :frowning: It seems to always return
STATUS_PENDING and do async completion, even if the request is completed
without delay. Maybe a filter driver which will force sync completion
can help here.
–pa

KMDF returns STATUS_PENDING if the request was passed via the queue.
EvtIoRead/Write are below the queue, so, if the IRP got there, STATUS_PENDING was
returned for it for sure.
The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

Here, I switched the default serial.sys with the KMDF version supplied in winddk -
wdfserial.sys (from the Winddk 7600.16385.1) and loaded inbox serenum.sys

Using this MSCOMM32.OCX application nevers returns “READ ERROR”. And works perfectly
well !?!?

So I put the IRPtracker and saw all the reads with STATUS_SUCCESS !?!?

Guess! wdfserial.sys is a KMDF driver, and should return STATUS_PENDING right?

IRPTracker log for wdfserial.sys:===================================

Call 0x8936E680-5550778 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550778 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Call 0x8936E680-5550779 (UNKNOWN) (0x8945E940) serenum DEVICE_CONTROL
Call 0x8936E680-5550779 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550779 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Comp 0x89FDF250-5550092 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x4
Call 0x8936E680-5550780 (UNKNOWN) (0x8945E940) serenum READ NORMAL
Call 0x8936E680-5550780 (0x8945E940) serenum \Device\Serial5 READ NORMAL
Comp 0x8936E680-5550780 \Device\Serial5 READ NORMAL SUCCESS, Info = 0x8

IRPTracker log for wdfserial.sys:===================================

No idea what UsbStartOrQueue does. We have no idea what you are returning from ReadEvtIoInCallerContext either in the read case. Looks like you have a working sample with a log which means you have everything you need to be self sufficient

d

debt from my phone


From: xxxxx@gmail.com
Sent: 11/14/2011 6:05 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] ReadFile Overlapped

Thanks Gary,

I am using a PC from a network where only osr site is allowed.
Later when I saw reply to my mails from another network, it says my updates are rejected.

I am re-posting again with thread history (hoping this time wont see any “excess quoted lines suppressed” messages).

===================================
Thanks Doron for suggestions.

That means you can’t add the “make it synchronous” code to an upper filter, it
needs to be in the fdo.

d

The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

I did changes at the function driver level, now I can see Requests are not landing
from EvtIoRead, instead I can see the requests from ReadEvtIoInCallerContext
In the ReadEvtIoInCallerContext, for Read requests: I did changes to process the
requests.

However still I can see the “STATUS_PENDING” coming and my MSCOMM32.OCX application
failing with “READ ERROR”

Below is the code part containing changes:
VOID ReadEvtIoInCallerContext(IN WDFDEVICE Device, IN WDFREQUEST Request)

switch (params.Type) {
case WdfRequestTypeRead:

status = WdfRequestRetrieveOutputBuffer (Request, reqContext->Length,
&reqContext->SystemBuffer, &bufLen);

UsbStartOrQueue(extension, Request, extension->ReadQueue,
&extension->CurrentReadRequest, UsbStartRead);
}
break;

default:
status = WdfDeviceEnqueueRequest(Device, Request);

} //switch

I was able to see using IRP tracker, READ at the fdo level is STATUS_SUCCESS but
READ at the filter level (modified serenum) is STATUS_PENDING !?!?!
Which I am not able to understand.

Also to compare this behaviour with default serial driver, I found interesting result.

This is a known issue with KMDF :frowning: It seems to always return
STATUS_PENDING and do async completion, even if the request is completed
without delay. Maybe a filter driver which will force sync completion
can help here.
–pa

KMDF returns STATUS_PENDING if the request was passed via the queue.
EvtIoRead/Write are below the queue, so, if the IRP got there, STATUS_PENDING was
returned for it for sure.
The only way to avoid STATUS_PENDING in KMDF is to do all work in
EvtInCallerContext.

Here, I switched the default serial.sys with the KMDF version supplied in winddk -
wdfserial.sys (from the Winddk 7600.16385.1) and loaded inbox serenum.sys

Using this MSCOMM32.OCX application nevers returns “READ ERROR”. And works perfectly
well !?!?

So I put the IRPtracker and saw all the reads with STATUS_SUCCESS !?!?

Guess! wdfserial.sys is a KMDF driver, and should return STATUS_PENDING right?

IRPTracker log for wdfserial.sys:===================================

Call 0x8936E680-5550778 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550778 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Call 0x8936E680-5550779 (UNKNOWN) (0x8945E940) serenum DEVICE_CONTROL
Call 0x8936E680-5550779 (0x8945E940) serenum \Device\Serial5 DEVICE_CONTROL
Comp 0x8936E680-5550779 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x14
Comp 0x89FDF250-5550092 \Device\Serial5 DEVICE_CONTROL SUCCESS, Info = 0x4
Call 0x8936E680-5550780 (UNKNOWN) (0x8945E940) serenum READ NORMAL
Call 0x8936E680-5550780 (0x8945E940) serenum \Device\Serial5 READ NORMAL
Comp 0x8936E680-5550780 \Device\Serial5 READ NORMAL SUCCESS, Info = 0x8

IRPTracker log for wdfserial.sys:===================================


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Thanks,
UsbStartOrQueue internally queues Request in ReadQueue, which later completed by UsbStartRead

I will try to capture each read request completion status using irp tracker and update.

However, one point is not clear: how wdfserrial.sys not showing STATUS_PENDING ?