TDI_SEND / TDI_RECEIVE data format

Hi guys!

I’m writing a TDI (passive) filter driver and came to the point when I can intercept send and receive requests / events. Can anybody tell what is the format of the actual data I see in send / receive. I would like to be able to detect and log HTTP requests and for that (I suppose) I have to (somehow) parse that sent / received data buffer for HTTP headers. I was thinking that these TSDU that I see are formatted one way or another so I can just jump through the layer of headers till I reach (or not) HTTP, but I’m having a hard time figuring out how TSDU are formated at TDI level (if they are formatted at all).
Sorry if this Q seem to be to broad / silly. I’m new to the TDI business and I would appreciate any response.

TIA,

Vladimir

Not sure whether you intend for the driver to run on client machines or web
servers, but make sure you factor in https(assuming you want to capture all
HTTP traffic). The data may still be encrypted depending on how you
implement the TDI driver(i.e. if you’re a filter for the TDI transport).

If you were intending that it run with IIS, you may want to consider an
ISAPI filter/[wildcard] extension to capture all HTTP traffic, after
decryption has taken place.

From: xxxxx@gmail.com
Reply-To: “Windows System Software Devs Interest List”

>To: “Windows System Software Devs Interest List”
>Subject: [ntdev] TDI_SEND / TDI_RECEIVE data format
>Date: Wed, 18 Apr 2007 18:43:45 -0400 (EDT)
>
>Hi guys!
>
>I’m writing a TDI (passive) filter driver and came to the point when I can
>intercept send and receive requests / events. Can anybody tell what is the
>format of the actual data I see in send / receive. I would like to be able
>to detect and log HTTP requests and for that (I suppose) I have to
>(somehow) parse that sent / received data buffer for HTTP headers. I was
>thinking that these TSDU that I see are formatted one way or another so I
>can just jump through the layer of headers till I reach (or not) HTTP, but
>I’m having a hard time figuring out how TSDU are formated at TDI level (if
>they are formatted at all).
>Sorry if this Q seem to be to broad / silly. I’m new to the TDI business
>and I would appreciate any response.
>
>TIA,
>
>Vladimir
>
>—
>Questions? First check the Kernel Driver FAQ at
>http://www.osronline.com/article.cfm?id=256
>
>To unsubscribe, visit the List Server section of OSR Online at
>http://www.osronline.com/page.cfm?name=ListServer

_________________________________________________________________
Get a FREE Web site, company branded e-mail and more from Microsoft Office
Live! http://clk.atdmt.com/MRT/go/mcrssaub0050001411mrt/direct/01/

The data in the TSDU for a TCP ‘stream’ TDI endpoint is simply the octet
stream of the the TCP connection. There is no formatting. If you are
trying to extract HTTP information, you need to parse the data stream and
deal with the possiblity that an HTTP message may stradle multiple TSDUs and
that a TSDU may straddle multpiple HTTP messages.

Good luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, April 18, 2007 6:44 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] TDI_SEND / TDI_RECEIVE data format

Hi guys!

I’m writing a TDI (passive) filter driver and came to the point when I can
intercept send and receive requests / events. Can anybody tell what is the
format of the actual data I see in send / receive. I would like to be able
to detect and log HTTP requests and for that (I suppose) I have to (somehow)
parse that sent / received data buffer for HTTP headers. I was thinking that
these TSDU that I see are formatted one way or another so I can just jump
through the layer of headers till I reach (or not) HTTP, but I’m having a
hard time figuring out how TSDU are formated at TDI level (if they are
formatted at all).
Sorry if this Q seem to be to broad / silly. I’m new to the TDI business and
I would appreciate any response.

TIA,

Vladimir


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Thank you Ron and David!

So what would be the best way to monitor HTTP traffic on the driver level? I bet this is very common task and there should be many products that do this. I just need some hints on the approach. Could filtering afd.sys be better?
So, any hints are very welcome.

Thanks again,

Vladimir

No format at all. Just chunks of data, which were passed to send() call by
the app. Format depends on the app.

TSDU boundaries for TCP carry no semantics, the data flow can switch to the
next TSDU in any possible place, regardless of what the application-level
format (like HTTP) is.

For application-level format, read the HTTP RFCs. There is nothing TDI
specific there.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
> Hi guys!
>
> I’m writing a TDI (passive) filter driver and came to the point when I can
intercept send and receive requests / events. Can anybody tell what is the
format of the actual data I see in send / receive. I would like to be able to
detect and log HTTP requests and for that (I suppose) I have to (somehow) parse
that sent / received data buffer for HTTP headers. I was thinking that these
TSDU that I see are formatted one way or another so I can just jump through the
layer of headers till I reach (or not) HTTP, but I’m having a hard time
figuring out how TSDU are formated at TDI level (if they are formatted at all).
> Sorry if this Q seem to be to broad / silly. I’m new to the TDI business and
I would appreciate any response.
>
> TIA,
>
> Vladimir
>

>need some hints on the approach. Could filtering afd.sys be better?

I think yes. TDI filters is same undocumented and unsupported as AFD filtering,
and AFD filtering is easier.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

If you’re looking to achieve this at the driver level, it would still depend
on whether it’s for a client or server platform. For the client, you could
filter AFD, but I don’t believe decryption takes place until you hit
user-mode. Therefore, for HTTPS scenarios, you would not be able to parse
anything.

If you’re talking about the server(specifically IIS 6.0), you could layer a
device over one of the HTTP.sys devices(I don’t remember the name) and get
the payloads after decryption has taken place. IIS 6.0 uses either LSASS or
ksecdd.sys for decryption, depending on whether kernel-mode SSL is enabled.
Although this might work, technically it would be considered a poor design
because the details of how HTTP.sys was implemented are not documented
outside of Microsoft. I did a proof-of-concept for this a couple of years
ago, and it was not as clean as I would have liked.

From: xxxxx@gmail.com
Reply-To: “Windows System Software Devs Interest List”

>To: “Windows System Software Devs Interest List”
>Subject: RE:[ntdev] TDI_SEND / TDI_RECEIVE data format
>Date: Thu, 19 Apr 2007 01:09:42 -0400 (EDT)
>
>Thank you Ron and David!
>
>So what would be the best way to monitor HTTP traffic on the driver level?
>I bet this is very common task and there should be many products that do
>this. I just need some hints on the approach. Could filtering afd.sys be
>better?
>So, any hints are very welcome.
>
>Thanks again,
>
>Vladimir
>
>—
>Questions? First check the Kernel Driver FAQ at
>http://www.osronline.com/article.cfm?id=256
>
>To unsubscribe, visit the List Server section of OSR Online at
>http://www.osronline.com/page.cfm?name=ListServer

_________________________________________________________________
Exercise your brain! Try Flexicon.
http://games.msn.com/en/flexicon/default.htm?icid=flexicon_hmemailtaglineapril07

xxxxx@gmail.com wrote:

So what would be the best way to monitor HTTP traffic on the driver level? I bet this is very common task and there should be many products that do this.

Why would you think that? You should never, ever, ever put anything in
kernel mode unless it absolutely, positively HAS to be in kernel mode.
Most HTML filtering tasks are much more sensible as a user-mode proxy.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Tim, the reason I thought so was that I’ve got this task into my hands. Given amount of driver devs out there I thought that chances of finding somebody with a related experience are far from being zero :wink:
As for the general rule, I’m 100% with you. If something can be accomplished entirely in the UM it must be accomplished entirely in the UM. Unfortunately, this is not my case.