Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

ISOCH USB Transfer: USBD_STATUS_CANCELED 0xC0010000

K_WellerK_Weller Member Posts: 6

Hi all,

I have a WDM USB driver doing USB ISOCH Hi-Speed streaming.

I have sometimes the issue that an URB (Urb->UrbHeader.Status) will return with the error message:

  • USBD_STATUS_CANCELED 0xC0010000: The USB stack reports this error whenever it completed a transfer because of an AbortPipe request from the client driver.

The "Irp->IoStatus.Status" is:

  • 0xC0000120 STATUS_CANCELLED: The I/O request was canceled.

I know that I do not call the AbortPipe request (I guess it is the Windows USB driver) and I have no idea what goes wrong at all.
I setup a sample which will do nothing in my asynchronous callback only resend the IRP, to check if I'm doing something wrong here but the stream can work for longer time and all of the sudden one URB will return with this error.

Is there a way to check out in more detail why and where from this error appears or any other suggestion ?

Many thank and best regards,
K. Weller

Comments

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,509

    You could try enabling tracing for the usb stack and see what’s up...

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • K_WellerK_Weller Member Posts: 6

    Thanks Peter,

    I will do this in the next couple of days. Btw as additional information: I do have a HW Analyzer and I do not see any issues with this one.
    But I will do the usb stack-tracing in more detail like you suggest. Until now I have't found any helpfull yet.

    Thanks
    K. Weller

  • Eric_WittmayerEric_Wittmayer Member Posts: 31

    You could also check the USBD_STATUS of the individual isoch packet descriptors in the URB. That might also give you more of a clue.

    Are you testing with your device connected to a USB 2 or USB 3 port? Even when debugging a USB 2 device I find it helpful to use a USB 3 root port because you will only see traffic for your device is the HW analyzer trace.

    @Peter_Viscarola_(OSR) I somehow missed that log tracing had been added to the USB 2 driver stack in Win 7. That's nice to know. Having the inbox driver logging for USB 3 has been very helpful.

    Eric

  • K_WellerK_Weller Member Posts: 6

    Hi,

    I needed to do some other stuff first so sorry for my late reply.

    Eric thanks for your suggestion. I did that before but I missunderstood the error code.
    The IsoPacket[i].Status == USBD_STATUS_ISO_NOT_ACCESSED_BY_HW.
    Because it was the output I though the irp was cancled so it was of course not accessed by the hardware. But this error code leads to this:

    https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/transfer-data-to-isochronous-endpoints

    When the USB driver stack processes the URB, the driver discards all isochronous packets in the URB whose frame numbers are lower than the current frame number. The driver stack sets the Status member of the packet descriptor for each discarded packet to USBD_STATUS_ISO_NA_LATE_USBPORT, USBD_STATUS_ISO_NOT_ACCESSED_BY_HW, or USBD_STATUS_ISO_NOT_ACCESSED_LATE. Even though some packets in the URB are discarded, the driver stack attempts to transmit only those packets whose frame numbers are higher than the current frame number.

    So I guess I have some kinde of racecondition in my IRP/URB pairs I queue up !?
    What I also noticed that I can reproduce and force this issue in debug mode if I do some debug printout messages with a DeviceIOControll call beside the streaming.

    But how does it come that I queue for example 4 asynchronous IRP/URB pairs, send them to the driver and wait for the completion callbacks.
    Then I fill the transfere buffers and send them again and then they arrive in a different order in the USB driver ?
    Any suggestions ?

    Or do I missunderstand something here ?

    Best regards,
    K.Weller

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,160

    Are you specifying a frame number in your isochronous requests? If so, why?

    How many packets in each URB? What's your endpoint's interval?

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,160

    Debug prints can have a significant impact on real-time actions. Depending on how you are connected, you have to be very stingy about including debug prints in the real-time path.

    Your URBs will always be completed in the same order in which you submit them, unless there is an error. If an URB fails validation, it will completed immediately.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • K_WellerK_Weller Member Posts: 6

    Thanks Tim,

    No I do not set frame numbers. I submit them always with those flags:

    urb->UrbIsochronousTransfer.TransferFlags |= USBD_START_ISO_TRANSFER_ASAP; urb->UrbIsochronousTransfer.TransferFlags |= USBD_SHORT_TRANSFER_OK;

    I send 8 microframes in each URB and the EP interval is 1. So I get callbacks every ms.

    About the debug prints. I debug over network. Would it be more straight if I would do the streaming realtime debugging with DbgView on the target ?

    Beside this, the issue will some times also appear without the printout. This will only force it some how.

    About your URBs.
    I noticed that if I give my URBs an index and send them down to the driver, the callbacks some times arrive in different order. If so the issue will most of the time happen but not always. In Release build it happens very rare and very hard to catch. Is a just connected debugger also impacting the real time actions significant so that the whole issue is just a debug side effect ?

    On question on how to repeat and reuse the IRP if the above error happens:
    I do it like this and call:
    IoReuseIrp(Irp, STATUS_SUCCESS);
    and then I prepare my URB and submitt it like allways.

    pNextStack = IoGetNextIrpStackLocation(pIrp);

    // Prepare Stack Location
    pNextStack->Parameters.Others.Argument1 = pTransferObject->Urb;
    pNextStack->Parameters.DeviceIoControl.IoControlCode = IoControlCode;                    
    pNextStack->MajorFunction = IRP_MJ_INTERNAL_DEVICE_CONTROL;
    
    // Register Completion Rountine
    ntStatus = IoSetCompletionRoutineEx(myPdo,
            pIrp,
            pAsyncCompletion,
            pTransferObject,
            TRUE,
            TRUE,
            TRUE);
    
    if (!NT_SUCCESS(ntStatus))
    {
        DBG_PRINT("UsbCallAsync: IoSetCompletionRoutineEx failed: %X", ntStatus);
    }
    else
    {
    // Call the driver 
    ntStatus = IoCallDriver(myPdo, pIrp);
    }
    

    Is this a way how it could be done ?

    Thanks and best regards,
    K.Weller

  • Eric_WittmayerEric_Wittmayer Member Posts: 31

    Hi @K_Weller,

    @K_Weller said:
    ...
    The IsoPacket[i].Status == USBD_STATUS_ISO_NOT_ACCESSED_BY_HW.
    ....

    That's very interesting, when the URBs that aren't cancelled complete, check the values of _URB_ISOCH_TRANSFER->ErrorCount and StartFrame. If ErrorCount != 0, look at the IsoPacket[i].Status of all 8 packets in URB and see if any are USBD_STATUS_ISO_NOT_ACCESSED_BY_HW or if some other non success status.

    My first thought is to make sure you have enough URBs in flight to keep up with the endpoint when completions are delayed by system load. Think about if processing the URB data really needs to happen every ms or if you could increase the number of packets in each URB.

    When you submit a URB with USBD_START_ISO_TRANSFER_ASAP, then somewhere either in the host driver stack or in the host hardware, a start frame is decided for your packets. When the isoch EP is already streaming data, the frame # of the last submitted packet is tracked and the next frame # is used as URBs are received. If the URBs aren't getting all the way to the HW in time to process the start frame needed, USBD_STATUS_ISO_NOT_ACCESSED_BY_HW or one of the other status values from the MSDN doc you quoted get returned. Thats why I'm asking about how many URBs/packets you have in flight and if any of the packets in your completed URBs have USBD_STATUS_ISO_NOT_ACCESSED_BY_HW.

    This is also a time when the windows USB driver tracing @Peter_Viscarola_(OSR) linked to above may be more useful than a HW Analyzer. It can show you the ISO URB/packet values both on submission and on completion.

    What version of Windows are you testing with and are you connecting to a USB2 or USB3 port. Also, is this Isoch IN or OUT?

    Eric

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,160

    Eric's description is quite right, but all of that processing happens inside the host controller driver. Your debugger activity shouldn't have an effect on that. His point about the number of URBs is and the number of packets per URB is also good; if you are able to send more than 8 packets in each URB, that will reduce overhead.

    USB_SHORT_TRANSFER_OK has no meaning for an isochronous pipe. It won't hurt, but it serves no purpose.

    From your discussion, I gathered you were doing isochronous OUT. Is that right? Are you quite sure your device can keep up with the data rate? If your device doesn't acknowledge one of the packets, that packet will be dropped with an error like that. Is this custom IP, or are you using an off-the-shelf USB chip?

    What does your device do? There are really very, very few circumstances where isochronous is the right answer. Live video and live audio are just about it.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • K_WellerK_Weller Member Posts: 6

    Eric and Tim thanks for your suggestions and help!

    I will answer top down:

    Most of the time all 8 packages have the USBD_STATUS_ISO_NOT_ACCESSED_BY_HW.
    Very rare I had the first microFrame with USBD_STATUS_ISO_TD_ERROR, all others USBD_STATUS_ISO_NOT_ACCESSED_BY_HW.
    But if one fails all other of the frame will fail to. Well make sense somehow from what you explained. But I saw never the first success or other in between succeeded.
    So this makes sense too I guess.

    About do I have enough URB in flight:
    Well I'm dealing with Audio so I have IN and OUT streaming (@Tim this answer your last two question). I try to lower the latency as much as possible.
    Well my customer wishes no latency the best :#
    So I'm experimenting with 2 IRP/URB pairs and 8 microFrames which is the lowest possible setting I guess.
    So probably there is no why to handle this without dropouts.

    Do I understand this correct. Each URB in flight will increase my latency for 1ms*Frames
    A Frame == 8 microFrames == 1 ms audio.
    So if I pass down 4 frames, each with 8 microFrames I have on the output side 4 ms latency ?
    If I double the frames in each URB because I always need to deal with multiple of 8 microFrames, I double the latency ?

    Input is handled different because I get the data which just arrived. So I can queue up many URB and do not increase the latency. Correct ?

    So at the end of the day I guess I need to increase the URB frame size.

    I will also follow up again the suggestion from Peter and digg more into this.
    BTW Is there a way to checkout what the Microsoft USBAudio driver uses for URB sizes ?

    @Tim_Roberts

    USB_SHORT_TRANSFER_OK has no meaning for an isochronous pipe. It won't hurt, but it serves no purpose.

    Yes you right of course. That was a left over from some copy past sample code I guess. Thanks for the hint.

    Best regards,
    K. Weller

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,160

    Why on earth are you writing a USB audio driver? That's lunacy. The Microsoft driver has had dozens of man years of development effort over the past 20 years, and probably thousands of man years of in-the-field operation. You'll never compete.

    The balance between latency and buffering is very delicate. The problem is that a USB frame is all scheduled in advance. If you do not have a new request submitted and in the queue when the host controller driver is scheduling the frame, you will miss that frame. It can't change the request chain once it is executing.

    USBAudio.sys used to aim for 10ms of buffering. I believe it now aims for 4ms. The difference between 4ms latency and 2ms latency is not detectable.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • K_WellerK_Weller Member Posts: 6

    Why on earth are you writing a USB audio driver?

    Well there are still reasons like ASIO and customized HW. I like it ;-)

  • Eric_WittmayerEric_Wittmayer Member Posts: 31

    Hi @K_Weller,

    Do I understand this correct. Each URB in flight will increase my latency for 1ms*Frames

    Yes, that's correct.

    Input is handled different because I get the data which just arrived. So I can queue up many URB and do not increase the latency. Correct ?

    Yes, queue up a bunch of URBs.
    This also begs the question, are the errors only on the OUT EP or on both IN and OUT?

    So at the end of the day I guess I need to increase the URB frame size.

    You should increase some combination of URB frame size and/or the number of URBs you keep submitted.
    I would start by increasing the number of URBs and make sure it all works without errors. Then start tuning the size and number of URBs on the OUT EP.

    BTW Is there a way to checkout what the Microsoft USBAudio driver uses for URB sizes ?

    You can use Microsoft Message Analyzer and the information from the link @Tim_Roberts gave to capture and view that information. Find when the USBAudio driver starts streaming and see how many URBs it submits. In Message Analyzer, there is an option to flatten the list, use that so the URB submissions and completions are shown separately.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,509

    Well there are still reasons like ASIO and customized HW

    Huh? ASIO latency of 4ms would be considered very acceptable in any use to which I've been exposed.

    Having said that, I am aware of drivers that claim to provide ASIO latencies less than 1ms... But, I don't believe it.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Eric_WittmayerEric_Wittmayer Member Posts: 31

    @Peter_Viscarola_(OSR) said:
    Having said that, I am aware of drivers that claim to provide ASIO latencies less than 1ms... But, I don't believe it.

    Certainly on windows with USB Isoch on inbox driver stack that seems impossible, considering the minimum 1ms URB size imposed by windows. Was the transport USB Isoch or some other protocol?
    Eric

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Writing WDF Drivers 21 Oct 2019 OSR Seminar Space & ONLINE
Internals & Software Drivers 18 Nov 2019 Dulles, VA
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 27 Apr 2020 OSR Seminar Space & ONLINE