Fragmentet USB Interrupt Data on High Systemload

NTDEV
1

Hi there

I got a problem with my usb driver. It uses a system thread which continuously creates IRP’s and calls the underlaying usb driver. There are always three outstanding Interrupt IN transfer requests. So the system polls my device as it should be. No problem, everything works fine until my system is under heavy load (and especially the device defines a polling interval of 4ms or less). In this case I got fragmentet, invalid data from the underlaying driver again and again. The lower the polling intervall is, the more data is invalid. Mostly it contains some data from the next transfer which will complete after the current one. I don’t have a clue what the problem is. On a low-powered notebook up to every 10th (or more) transfer contains invalid data. On my highend 2Core PC only a few transfers are fragmentet and invalid.

Within the completion routine the driver stores the data into a buffer. The usermode application accesses this buffer through an DeviceIoControll request. Every access to buffer is protected by a SpinLook.

So I is there maybe a timing / synchronization problem?!?

Btw: My Usb-Transfer-Logger application (I think it’s filter driver lays under my driver) also sees the fragmented data as my drivers receives it.

May someone has an idea? Thank you!

Patric

CODE:

VOID
ThreadInterruptTransfer (
IN PVOID pDeviceExtension )

{
PURB urb;
PIRP irp;
LONG counter;
NTSTATUS ntStatus;
PDEVICE_EXTENSION deviceExtension;
PIO_STACK_LOCATION pNextStack;
IO_STATUS_BLOCK ioStatus;
LONG DATAPACK_SIZE;
CHAR buffer[sizeof(DataPackS)][CONCURRENT_INTERUPT_TRANSFERS];

DATAPACK_SIZE = sizeof(DataPackS);
deviceExtension = pDeviceExtension;
KeSetPriorityThread(KeGetCurrentThread(), LOW_REALTIME_PRIORITY);
KeInitializeSemaphore(&deviceExtension->InterruptTransferSemaphore, CONCURRENT_INTERUPT_TRANSFERS, CONCURRENT_INTERUPT_TRANSFERS);
counter = 0;

while(!deviceExtension->bShutdownInterruptTransferThread)
{
KeWaitForSingleObject(&deviceExtension->InterruptTransferSemaphore,
Executive, KernelMode, TRUE, NULL);

if(deviceExtension->bShutdownInterruptTransferThread)
break;

ntStatus = SubmitInterruptReadIRP(deviceExtension->TopOfStackDeviceObject,&buffer[0][counter],DATAPACK_SIZE,deviceExtension);

counter = (++counter) % (CONCURRENT_INTERUPT_TRANSFERS);
}

PsTerminateSystemThread(STATUS_SUCCESS);
}

NTSTATUS
SubmitInterruptReadIRP(
PDEVICE_OBJECT TopOfDeviceStack,
PVOID WriteBuffer,
ULONG NumBytes,
PDEVICE_EXTENSION deviceExtension
)
{
NTSTATUS ntStatus;
PURB urb;
NTSTATUS status;
PIRP irp;
LARGE_INTEGER startingOffset;
KEVENT event;
PIO_STACK_LOCATION nextStack;
PBULKUSB_RW_CONTEXT rwContext;

rwContext = NULL;

startingOffset.QuadPart = (LONGLONG) 0;

irp = IoAllocateIrp( TopOfDeviceStack->StackSize, FALSE );
if (NULL == irp) {

return STATUS_INSUFFICIENT_RESOURCES;
}

irp->AssociatedIrp.SystemBuffer = WriteBuffer;
irp->MdlAddress = NULL;

urb = ExAllocatePool(NonPagedPool,
sizeof(struct _URB_BULK_OR_INTERRUPT_TRANSFER));
if(urb == NULL) {

BulkUsb_DbgPrint(1, (“Failed to alloc mem for urb\n”));
return STATUS_INSUFFICIENT_RESOURCES;
}

UsbBuildInterruptOrBulkTransferRequest(
urb,
sizeof(struct _URB_BULK_OR_INTERRUPT_TRANSFER),
deviceExtension->InterruptPipeHandle,
irp->AssociatedIrp.SystemBuffer,
irp->MdlAddress,
NumBytes,
USBD_TRANSFER_DIRECTION_IN | USBD_SHORT_TRANSFER_OK,
NULL);

rwContext = ExAllocatePool(NonPagedPool,
sizeof(BULKUSB_RW_CONTEXT));
if(rwContext == NULL) {
return STATUS_INSUFFICIENT_RESOURCES;
}

rwContext->Urb = urb;
rwContext->Mdl = irp->MdlAddress;
rwContext->Length = NumBytes;
rwContext->Numxfer = 0;
rwContext->VirtualAddress = NULL;
rwContext->DeviceExtension = deviceExtension;

nextStack = IoGetNextIrpStackLocation(irp);
nextStack->MajorFunction = IRP_MJ_INTERNAL_DEVICE_CONTROL;
nextStack->Parameters.Others.Argument1 = (PVOID) urb;
nextStack->Parameters.DeviceIoControl.IoControlCode =
IOCTL_INTERNAL_USB_SUBMIT_URB;

IoSetCompletionRoutine(irp,
InterruptCompletion,
rwContext,
TRUE,
TRUE,
TRUE);

ntStatus = IoCallDriver(TopOfDeviceStack, irp);

return STATUS_SUCCESS;
}

NTSTATUS
InterruptCompletion(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp,
IN PVOID Context
)
{
NTSTATUS status;
PVOID buffer;
PMDL mdl, nextMdl;
KIRQL oldIrql;
PDEVICE_EXTENSION deviceExtension;
PBULKUSB_RW_CONTEXT rwContext;
LARGE_INTEGER timeout;
rwContext = Context;
deviceExtension = rwContext->DeviceExtension;

if(NT_SUCCESS(Irp->IoStatus.Status))
{
buffer = Irp->AssociatedIrp.SystemBuffer;

KeAcquireSpinLock(&deviceExtension->InterruptTransferBufferSpinLock, &oldIrql);
CarSmUsbFifoInsertData(deviceExtension->InterruptTransferBuffer, (unsigned char*)(buffer), sizeof(DataPackS));
KeReleaseSpinLock(&deviceExtension->InterruptTransferBufferSpinLock, oldIrql);
}

ExFreePool(rwContext->Urb);
ExFreePool(rwContext);

BulkUsb_DbgPrint(3, (“IoFreeIrp - Interrupt Transfer Completed\n”));
RemoveOutstandingInterruptTransferIrp(deviceExtension, Irp);
IoFreeIrp(Irp);

try {
KeReleaseSemaphore(&deviceExtension->InterruptTransferSemaphore,
0,
1,
FALSE);
} except (STATUS_SEMAPHORE_LIMIT_EXCEEDED) {}

return STATUS_MORE_PROCESSING_REQUIRED;
}

2

xxxxx@carag.com wrote:

I got a problem with my usb driver. It uses a system thread which continuously creates IRP’s and calls the underlaying usb driver. There are always three outstanding Interrupt IN transfer requests. So the system polls my device as it should be. No problem, everything works fine until my system is under heavy load (and especially the device defines a polling interval of 4ms or less). In this case I got fragmentet, invalid data from the underlaying driver again and again…
Within the completion routine the driver stores the data into a buffer. The usermode application accesses this buffer through an DeviceIoControll request. Every access to buffer is protected by a SpinLook.

So I is there maybe a timing / synchronization problem?!?

Nope, I think it’s a C problem…

CODE:

VOID
ThreadInterruptTransfer (
IN PVOID pDeviceExtension )

Why do you use a thread for this? Why don’t you just resubmit the URBs
in the completion routine? I just don’t see what the thread is buying
you, and it’s yet another asynchronous player to interfere.

CHAR buffer[sizeof(DataPackS)][CONCURRENT_INTERUPT_TRANSFERS];

I think you are confused about the layout of multidimensional arrays in
C. buffer[0][0] is adjacent to buffer[0][1], and it looks like you are
assuming buffer[0][0] is adjacent to buffer[1][0]. I suspect that’s
your fundamental issue. Any time you have a request complete before you
have finished the one before it, you will get garbage.

How big is DataPackS? You’re chewing up a lot of stack space here. You
should be allocating these out of pool memory.

irp = IoAllocateIrp( TopOfDeviceStack->StackSize, FALSE );
if (NULL == irp) {
return STATUS_INSUFFICIENT_RESOURCES;
}

irp->AssociatedIrp.SystemBuffer = WriteBuffer;
irp->MdlAddress = NULL;

You shouldn’t be setting AssociatedIrp.SystemBuffer or MdlAddress. Just
set the address into TransferBuffer in the URB (as you do below, and get
it out of there in the completion routine.

NTSTATUS
InterruptCompletion(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp,
IN PVOID Context
)
{
…

I don’t see that you ever free the URB structure. That’s going to chew
up memory pretty fast. You ought to be able to allocate one set of IRPs
and URBs and just reuse them in the completion routine.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

Hi Tim, thanks for your response.

I’m ashamed :frowning: of course I assinged my buffer array not correctly to the function. To your other comments:

I used the thread because I read something like that in this forum. But it’s obvious to reuse the IRP’s and resend them to the underlaying driver. I will change this.
My DataPackS structure is 18Bytes long. Not that much. I will also change this and allocate pool memory especially when I change the whole thing to the IRP reuse way.

Thanks again!
Patric

4

Patric Bucher wrote:

I used the thread because I read something like that in this forum. But
it’s obvious to reuse the IRP’s and resend them to the underlaying
driver. I will change this.

Like many before you, you’ve totally failed in implementing the USB continuous reader pattern. If you’re going to “change” something, I would start by switching to KMDF.