Reading from umdf virtualserial port

I am working on VirtualSerial at the following link.

https://github.com/microsoft/Windows-driver-samples/tree/master/serial/VirtualSerial

This sample demonstrates A simple virtual serial driver (ComPort). This driver supports sending and receiving AT commands or Echo strings using the ReadFile and WriteFile calls or via a TAPI interface using an application such as, HyperTerminal.

This project working good with HyperTerminal. But not working with other serial applications like Ex. TereTerm.

Actually inside driver, CMyQueue::OnWrite called, but CMyQueue::OnRead never get called.

How can handle ReadQueue to work with other serial applications ?

Why are you developing a UMDF v1 driver? You are far better off going with umdf v2.

@Doron_Holan said:
Why are you developing a UMDF v1 driver? You are far better off going with umdf v2.

I did a little search. I think,
There is no filter driver. The driver can not complete a request without lower driver. If it was designed as filter driver, the driver filters can write requests as they arrive and filters read requests after lower drivers have completed them. The driver implements the default I/O handler interface so that it can set an I/O completion callback for read requests.

So, Does UMDF v2 support completion request at same layer ?

I am confused by your mention of a filter driver. The original driver sample was not written as a filter driver, it was written as an FDO. Is your current implementation of the driver modified to be a filter?

@goodfriend said:
So, Does UMDF v2 support completion request at same layer ?

Both versions of UMDF allow the driver to complete requests at the same layer without sending them down the stack as appropriate for the operation being processed.

@Doron_Holan said:
The original driver sample was not written as a filter driver, it was written as an FDO. Is your current implementation of the driver modified to be a filter?

Yes i mean that as you said, this is not filter driver. There is no read requests , CMyQueue::OnRead never get called with serial applications.
For this, to call OnRead method, i think, there must be a lower drivers. Forward the request down the stack. The device below completes the request.
Am i right ?

Actually, i did not see echo loopback when using virtualserial driver trying with serial application. I want to see echo. I am trying to find out how can i do ?
For this, i did some changes at the following :

VOID
CMyQueue::ForwardFormattedRequest(
    _In_ IWDFIoRequest* pRequest,
    _In_ IWDFIoTarget* pIoTarget
)
{
    //
    //First set the completion callback
    //

    IRequestCallbackRequestCompletion* pCompletionCallback = NULL;
    HRESULT hrQI = this->QueryInterface(IID_PPV_ARGS(&pCompletionCallback));
    WUDF_TEST_DRIVER_ASSERT(SUCCEEDED(hrQI) && (NULL != pCompletionCallback));

    pRequest->SetCompletionCallback(
        pCompletionCallback,
        NULL
    );

    pCompletionCallback->Release();
    pCompletionCallback = NULL;

    //
    //Send down the request
    //
    HRESULT hrSend = S_OK;
    hrSend = pRequest->Send(pIoTarget,
        0,  //flags
        0); //timeout

    if (FAILED(hrSend))
    {
        pRequest->CompleteWithInformation(hrSend, 0);
    }

    return;
}

VOID CMyQueue::FlushRx()
{
    IWDFIoRequest* FxRequest = NULL;
    IWDFMemory* FxMemory = NULL;
    IWDFIoTarget* pwdfIoTarget;
    HRESULT hr = S_OK;
    IWDFDriver* FxDriver = NULL;

    m_FxDevice->GetDefaultIoTarget(&pwdfIoTarget);
    if (pwdfIoTarget == NULL) {
        return;
    }

    hr = m_FxDevice->CreateRequest(NULL, NULL, &FxRequest);

    if (SUCCEEDED(hr))
    {
        m_FxDevice->GetDriver(&FxDriver);

        hr = FxDriver->CreatePreallocatedWdfMemory((PBYTE)&m_Buffer,
            sizeof(m_Buffer),
            NULL, //pCallbackInterface
            FxRequest, //pParetObject
            &FxMemory);
    }
    if (SUCCEEDED(hr))
    {
        hr = pwdfIoTarget->FormatRequestForRead(FxRequest,
            NULL, //pFile - IoTarget would apply its file
            FxMemory,
            NULL, //Memory offset
            NULL);  //Device offset                                                                                   
    }
    if (SUCCEEDED(hr))
    {
        ForwardFormattedRequest(FxRequest, pwdfIoTarget);
    }
}

And i called in the OnWrite at the following :

    //
    // Get the amount of data available in the ring buffer
    //
    if (availableData > 0)
    {
        FlushRx();
        .......

Then i noticed that, there is no lower device. Am i right ?
Do we have to modify virtual serial as a filter driver ?

For this, to call OnRead method, i think, there must be a lower drivers.

No. I/O requests in Windows ALWAYS go downward, from user-mode toward the hardware. CMyQueue::OnRead will get called when there is an IRP_MJ_READ request from above you, usually from a ReadFile call in an application.

I’m not sure why OnWrite would care how much data is in the ring buffer. OnWrite means you are receiving new data for the device. You’re not returning existing data, that’s for OnRead.

Then i noticed that, there is no lower device. Am i right ?

You are the only one who would know that. If you are a virtual driver and there is no hardware, then of course there is no lower device.

No matter if your driver is a filter or the FDO, you should see the read request come into the driver if you are the top of the stack (IOW if there are no filters above your driver). There is a 99.99% certainty your driver is the top of the stack. Since the driver works with hyperterm and you get a read request but not other serial consoles, the driver is not responding correctly to previously sent requests and the app is not seeing the results it needs to send the read . I suggest you look at the logs to see what IOCTLs are sent by the app and see if you are fully and correctly supporting all of the necessary IOCTL semantics, these can get complicated and the sample you are using is quite old, so there could be either mistakes/gaps in the sample itself or with the subsequent changes you have made to the driver.

Hi, sorry for late answer.

@Tim_Roberts said:
CMyQueue::OnRead will get called when there is an IRP_MJ_READ request from above you

@Doron_Holan said:

No matter if your driver is a filter or the FDO, you should see the read request come into the driver if you are the top of the stack (IOW if there are no filters above your driver). There is a 99.99% certainty your driver is the top of the stack.
the app is not seeing the results it needs to send the read
what IOCTLs are sent by the app and see if you are fully and correctly supporting all of the necessary IOCTL semantics

Yes you are all right. Application must request this. I thought about IOCTLs , what is the wrong, then research around this.
I find out what is the problem. Shortly, the problem is about case IOCTL_SERIAL_WAIT_ON_MASK.

In the orginal source code at the following, code return and there is no completing request for m_FxWaitMaskQueue.
Serial application stay waiting on IOCTL_SERIAL_WAIT_ON_MASK.

        case IOCTL_SERIAL_WAIT_ON_MASK:
        {
            //
            // NOTE: the contract is that this ioctl should be marked pending
            // and not to be completed until some wait event happens. Therefore
            // it is incorrect for the driver to complete the ioctl right away,
            // no matter whether success or failure is returned. In either case
            // most likely the app will send down another iocl of wait-on-mask
            // in a tight loop, and that gets completed again, and again.
            // The end result will be high CPU utilization in task manager.
            //
            // The correct way would be to mark the ioctl as pending, or as in
            // WDF world, keep it in a manual queue. Since this is a driver for
            // a virtual serial port and there is no actual hardware, there will
            // be no wait event happening. The ioctl stays in the queue until
            // the calling app decides to no longer wait for it by means of
            // sending down a set-wait-mask request.
            //

            //
            // At most one pending wait-on-mask request is expected
            //
            IWDFIoRequest *pSavedRequest;
            hr = m_FxWaitMaskQueue->RetrieveNextRequest(&pSavedRequest);
            if (SUCCEEDED(hr))
            {
                pSavedRequest->Complete(E_FAIL);

                //
                // RetrieveNextRequest from a manual queue increments the reference
                // counter by 1. We need to decrement it, otherwise the request will
                // not be released and there will be an object leak.
                //
                SAFE_RELEASE(pSavedRequest);
            }

            //
            // Keep the request in a manual queue and the framework will take
            // care of cancelling them when the app exits.
            //
            pWdfRequest->ForwardToIoQueue(m_FxWaitMaskQueue);

            //
            // Instead of "break" out of the switch statement and complete the
            // request at the end of this function, use "return" directly.
            //
            return;
        }

I think, m_FxWaitMaskQueue must be complete in other thread, but i change this state like at the following.
And Echo is now working correctly.

        case IOCTL_SERIAL_WAIT_ON_MASK:
        {
            //
            // NOTE: the contract is that this ioctl should be marked pending
            // and not to be completed until some wait event happens. Therefore
            // it is incorrect for the driver to complete the ioctl right away,
            // no matter whether success or failure is returned. In either case
            // most likely the app will send down another iocl of wait-on-mask
            // in a tight loop, and that gets completed again, and again.
            // The end result will be high CPU utilization in task manager.
            //
            // The correct way would be to mark the ioctl as pending, or as in
            // WDF world, keep it in a manual queue. Since this is a driver for
            // a virtual serial port and there is no actual hardware, there will
            // be no wait event happening. The ioctl stays in the queue until
            // the calling app decides to no longer wait for it by means of
            // sending down a set-wait-mask request.
            //
            ULONG IsrWaitMask = { 0 };

            pWdfRequest->GetOutputMemory(&outputMemory);
            if (NULL != outputMemory)
            {
                hr = outputMemory->CopyFromBuffer(0,
                    (void*)&IsrWaitMask,
                    sizeof(IsrWaitMask));

                if (SUCCEEDED(hr))
                {
                    reqCompletionInfo = sizeof(ULONG);
                }
            }

            hr = S_OK;

            break;
        }

To find out problem , I test and debug device at the following application code :

#include <windows.h>
#include <tchar.h>
#include <assert.h>
#include <stdio.h>

void _tmain(
    int argc,
    TCHAR* argv[]
)
{
    HANDLE hCom;
    BOOL fSuccess;
    DWORD dwEvtMask;

    hCom = CreateFile(TEXT("\\\\.\\COM3"),
        GENERIC_READ | GENERIC_WRITE,
        0,    // exclusive access 
        NULL, // default security attributes 
        OPEN_EXISTING,
        FILE_FLAG_OVERLAPPED,
        NULL
    );

    if (hCom == INVALID_HANDLE_VALUE)
    {
        // Handle the error. 
        printf("CreateFile failed with error %d.\n", GetLastError());
        return;
    }

    DCB* dcb = new DCB();
    if (GetCommState(hCom, dcb)) {
        dcb->BaudRate = 19200;
        dcb->ByteSize = 8;
        dcb->StopBits = 1;
        dcb->Parity = 0;

        //since 0.8 ->
        dcb->fRtsControl = RTS_CONTROL_ENABLE;
        //dcb->fRtsControl = RTS_CONTROL_DISABLE;
        dcb->fDtrControl = DTR_CONTROL_ENABLE;
        //dcb->fDtrControl = DTR_CONTROL_DISABLE;
        dcb->fOutxCtsFlow = FALSE;
        dcb->fOutxDsrFlow = FALSE;
        dcb->fDsrSensitivity = FALSE;
        dcb->fTXContinueOnXoff = TRUE;
        dcb->fOutX = FALSE;
        dcb->fInX = FALSE;
        dcb->fErrorChar = FALSE;
        dcb->fNull = FALSE;
        dcb->fAbortOnError = FALSE;
        dcb->XonLim = 2048;
        dcb->XoffLim = 512;
        dcb->XonChar = (char)17; //DC1
        dcb->XoffChar = (char)19; //DC3
        //<- since 0.8

        if (SetCommState(hCom, dcb)) {

            //since 2.1.0 -> previously setted timeouts by another application should be cleared
            COMMTIMEOUTS* lpCommTimeouts = new COMMTIMEOUTS();
            lpCommTimeouts->ReadIntervalTimeout = 0;
            lpCommTimeouts->ReadTotalTimeoutConstant = 0;
            lpCommTimeouts->ReadTotalTimeoutMultiplier = 0;
            lpCommTimeouts->WriteTotalTimeoutConstant = 0;
            lpCommTimeouts->WriteTotalTimeoutMultiplier = 0;
            if (SetCommTimeouts(hCom, lpCommTimeouts)) {
                printf("SetCommTimeouts\n");
            }
            delete lpCommTimeouts;
            //<- since 2.1.0
        }
    }
    delete dcb;

    // Set the event mask. 

    fSuccess = SetCommMask(hCom, EV_RXCHAR | EV_RXFLAG | EV_TXEMPTY);

    if (!fSuccess)
    {
        // Handle the error. 
        printf("SetCommMask failed with error %d.\n", GetLastError());
        return;
    }

    while (1) {
        OVERLAPPED* overlapped = new OVERLAPPED();
        // Initialize the rest of the OVERLAPPED structure to zero.
        overlapped->Internal = 0;
        overlapped->InternalHigh = 0;
        overlapped->Offset = 0;
        overlapped->OffsetHigh = 0;
        
        // Create an event object for use by WaitCommEvent. 
        overlapped->hEvent = CreateEvent(
            NULL,   // default security attributes 
            TRUE,   // manual-reset event 
            FALSE,  // not signaled 
            NULL    // no name
        );

        if (overlapped->hEvent == NULL) {
            printf("Create an event object for use by WaitCommEvent FAIL.\n");
            break;
        }

        if (WaitCommEvent(hCom, &dwEvtMask, overlapped))
        {
            if (dwEvtMask & EV_RXCHAR)
            {
                // To do.
                printf("I/O is EV_RXCHAR\n");
            }

            if (dwEvtMask & EV_RXFLAG)
            {
                // To do. 
                printf("I/O is EV_RXFLAG...\n");
            }
        }
        else
        {
            DWORD dwRet = GetLastError();
            if (ERROR_IO_PENDING == dwRet)
            {
                DWORD lpNumberOfBytesTransferred = 0;
                printf("I/O is pending...\n");

                if (WaitForSingleObject(overlapped->hEvent, INFINITE) == WAIT_OBJECT_0) {
                    if (GetOverlappedResult(hCom, overlapped, &lpNumberOfBytesTransferred, false)) {
                        printf("function Successful...%d\n", lpNumberOfBytesTransferred);
                    }
                    else {
                        printf("OverlappedResult failed with error %d.\n", GetLastError());
                    }
                }
            }
            else {
                printf("Wait failed with error %d.\n", GetLastError());
            }
        }
        delete overlapped;
    }
}