Hello,
I have built a driver that compresses the data in a TDI IRP for selected processes and resets the size to send then passes to next driver (TCP). In the completion routine if the send was successful then I reset the size to send and the bytes sent (Information variable) to reflect the original values. The receiving application recognizes the compressed data and uncompresses. This seems to work partially, but frequently results in flakyness. The receiving application gets an unknown unrecognizable stream of data, like too much was sent or a packet not sent at all. Something like that. And I have seen blue screens as well for certain builds.
To finish my question (hit enter accidentally), is there something inheritently wrong with doing what I did…compressing the data in the existing IRP, resetting the size to send and then resetting both back to original in the completion routine. Here are segments of the code:
if( Irp->MdlAddress ) {
userBuffer = MmGetSystemAddressForMdlSafe( Irp->MdlAddress, NormalPagePriority );
} else {
userBuffer = Irp->UserBuffer;
}
bufLen = requestKernelSend->SendLength;
// now compress data in userbuffer…
requestKernelSend->SendLength = (ULONG) nNewLen;
ntstat = IPNitroPassOnWithTailProcessing(HookDevice, Irp, bufLen);
NTSTATUS IPNitroPassOnWithTailProcessing (PDEVICE_OBJECT HookDevice, IN PIRP Irp, int nOriginalLen)
{
PHOOK_EXTENSION hookExt = HookDevice->DeviceExtension;
DbgPrint((“Tail logging\n”));
//IoMarkIrpPending( Irp );
// Copy parameters down to next level in the stack for the driver below us
IoCopyCurrentIrpStackLocationToNext(Irp);
DbgPrint((“set IPNormalCompletionRoutine”));
// But make sure not to copy the completion routine!
IoSetCompletionRoutine(Irp, IPNitroNormalCompletionRoutine, (void *) nOriginalLen,
TRUE, TRUE, TRUE);
return IoCallDriver(hookExt->attachedDevice, Irp);;
}
NTSTATUS IPNitroNormalCompletionRoutine (IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp, IN PVOID Context)
{
KIRQL oldirql;
PCHAR eventBuffer;
PIO_STACK_LOCATION IrpSp;
CHAR errorBuf[ERRORLEN];
ULONG nOrigLen = (ULONG) Context;
// Log the return values.
IrpSp = IoGetCurrentIrpStackLocation( Irp );
DbgPrint((“IPNitroNormalCompletionRoutine\n”));
if ((IrpSp->MajorFunction==IRP_MJ_INTERNAL_DEVICE_CONTROL) && IrpSp->MinorFunction==TDI_SEND) {
// if something was sent then set size to what user passed in to send
DbgPrint((“if success reset size to original\n”));
if (Irp->IoStatus.Information) {
DbgPrint((“reset size\n”));
if (Irp->IoStatus.Information != ((PTDI_REQUEST_KERNEL_SEND)&IrpSp->Parameters.Others)->SendLength) {
KeBugCheckEx(9,Irp->IoStatus.Information,((PTDI_REQUEST_KERNEL_SEND)&IrpSp->Parameters.Others)->SendLength,0,74);
}
*((long *) &Irp->IoStatus.Information) = nOrigLen;
}
((PTDI_REQUEST_KERNEL_SEND)&IrpSp->Parameters.Others)->SendLength = nOrigLen;
}
else if ((IrpSp->MajorFunction==IRP_MJ_INTERNAL_DEVICE_CONTROL) && IrpSp->MinorFunction==TDI_RECEIVE) {
}
if (Irp->PendingReturned) {
IoMarkIrpPending(Irp);
}
return STATUS_CONTINUE_COMPLETION;
}
Are you determining the ‘process’ on a request by request basis or by
correlating the PFILE_OBJECT in the IRP IO_STACK_LOCATION to some context?
Hopefully the later since you cannot assume that send operations will come
to you in any particular process context.
Do you have some rudimentary test mechanism that sends a recognizable stream
of data (like the Gettysburg Address, my favorite, but any work of
Shakespear will do too) so that, for instance, in a network trace it would
be very easy to detect that it was *not* compressed? Your description of
flakeyness does not narrow down the type of failure you are getting to one
of:
- uncompressed data inserted into the compressed stream.
- out of order data
- missing data
- duplicated data
I assume that this is TCP only since it would not seem to make sense for
non-stream endpoints.
Basically, you need to instrument both ends and the middle and figure out
what is going on in enough (symptom) detail to know where to look next. If,
for instance, you determine that every once and a while an uncompressed
segment shows up in the stream in the correct place, perhaps, you are not
intercepting send operations correctly.
It is also very helpful to ‘disable’ the actual compression so that you can
find insertion/deletion errors more easily.
Good Luck,
-dave
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@swbell.net
Sent: Thursday, July 19, 2007 11:11 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] compressing data in place in TDI IRP and resetting size
causing flakyness - why?
Hello,
I have built a driver that compresses the data in a TDI IRP for
selected processes and resets the size to send then passes to next driver
(TCP). In the completion routine if the send was successful then I reset the
size to send and the bytes sent (Information variable) to reflect the
original values. The receiving application recognizes the compressed data
and uncompresses. This seems to work partially, but frequently results in
flakyness. The receiving application gets an unknown unrecognizable stream
of data, like too much was sent or a packet not sent at all. Something like
that. And I have seen blue screens as well for certain builds.
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
David, Thanks for the input!
Our outside application issues an IOCTL to our driver sending the driver the pid of the process to filter. The driver then passes through all other traffic for other processes. But if the send operations for that process can actually arrive in the dispatch methods under another processes pid, then while I was hoping that was not the case, the driver can still actually on the fly determine to not compress given blocks anyhow. The receiving application recognizes that the data is not compressed in that case and accepts the data as is.
This is a TDS stream which has an 8 byte header where we can alter control codes and TDS lengths. That is how we designate whether the following TDS packet data is compressed or not and how the receiving app knows if that packet is compressed or not as well as the length.
The “flakyness” I meniton actually comes in 3 forms.
- really incredibly slow transmittal of data. On the order of at least 5 times slower before I get impatient and cancel it.
- an invalid TDS stream is received by the client so connection dropped.
- blue screen, although the blue screen could be just mistakes made in my mad code change attempts to get past the real problem.
So mainly I just wanted to know if changing data right inthe IRP and altering the length and then resetting in the completion routine was a valid thing to do. Anything wrong with that in particular? Liek what if the higher up driver created the mdl for read access only. Could that be the problem?
If I do one simple thing to the driver everything works great. If I go through the compression and set up the IRP for calling my completion routine were I reset the length all as normal, but just make one change…don’t memcpy the compressed data into the IRP and set nNewLen to the original length anyhow, then it all works. Data is not sent compressed of course, but all the same code is executed short of the memcpy.
Sorry, I missed the really important part and the one question you really
asked in the second post - can you safely modify the data. The answer is
no. You must treat the data as read-only. You have no way of knowing what
the ‘owner’ of that block of memory/virtual address is planning to do with
it. The filter should only consider itself permitted to ‘read’ the data.
You need to compress into another buffer and *substitute* that buffer in
the call down-stack.
Good Luck,
-dave
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@swbell.net
Sent: Thursday, July 19, 2007 1:52 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] compressing data in place in TDI IRP and resetting size
causing flakyness - why?
David, Thanks for the input!
Our outside application issues an IOCTL to our driver sending the driver the
pid of the process to filter. The driver then passes through all other
traffic for other processes. But if the send operations for that process can
actually arrive in the dispatch methods under another processes pid, then
while I was hoping that was not the case, the driver can still actually on
the fly determine to not compress given blocks anyhow. The receiving
application recognizes that the data is not compressed in that case and
accepts the data as is.
This is a TDS stream which has an 8 byte header where we can alter control
codes and TDS lengths. That is how we designate whether the following TDS
packet data is compressed or not and how the receiving app knows if that
packet is compressed or not as well as the length.
The “flakyness” I meniton actually comes in 3 forms.
- really incredibly slow transmittal of data. On the order of at least 5
times slower before I get impatient and cancel it. - an invalid TDS stream is received by the client so connection dropped.
- blue screen, although the blue screen could be just mistakes made in my
mad code change attempts to get past the real problem.
So mainly I just wanted to know if changing data right inthe IRP and
altering the length and then resetting in the completion routine was a valid
thing to do. Anything wrong with that in particular? Liek what if the higher
up driver created the mdl for read access only. Could that be the problem?
If I do one simple thing to the driver everything works great. If I go
through the compression and set up the IRP for calling my completion routine
were I reset the length all as normal, but just make one change…don’t
memcpy the compressed data into the IRP and set nNewLen to the original
length anyhow, then it all works. Data is not sent compressed of course, but
all the same code is executed short of the memcpy.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Most excellent. Thank you David. another thought occurred to me along that vein. I suppose if the device is not ready and the send did not go through so that the app or higher driver has to resend, the data was left still compressed in the IRP. I coded to recognize that to then just pass it through and just reset the send length to compressed size again…but that’s only if the driver sees the send come in on the same pid context again. If it comes in on a different process’s pid then I bypass it, thereby passing through compressed data but with the uncompressed length specified inthe TDS. That coudl be the primary problem, one which will be avoided if I take your advice and use a substitute buffer. Thanks again David.
I would suggest to use the WinSock layered service provider instead for
such a job.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> Hello,
> I have built a driver that compresses the data in a TDI IRP for selected
processes and resets the size to send then passes to next driver (TCP). In the
completion routine if the send was successful then I reset the size to send and
the bytes sent (Information variable) to reflect the original values. The
receiving application recognizes the compressed data and uncompresses. This
seems to work partially, but frequently results in flakyness. The receiving
application gets an unknown unrecognizable stream of data, like too much was
sent or a packet not sent at all. Something like that. And I have seen blue
screens as well for certain builds.
>
>