cisco vpn and multiple buffers per packet

This is the same sort of problem I’ve mentioned previously but I have investigated a bit further now.

Normally I start off investigating obvious network errors (eg ping < 100 bytes works but ping >= 100 bytes doesn’t) with tools like tcpdump and wireshark, looking for errors that way. In this case though, the ESP (ip proto 50) packets never show up in wireshark, which says to me that the cisco vpn driver grabs them at a fairly low level in the network stack.

My driver often receives packets from xen with the eth+ip(+udp/tcp/etc) header in one buffer and the rest of the data in another. Is it possible that cisco vpn (testing with latest version) simply cannot handle packets with more than one attached buffer? Do other network adapters never do this? I see under Linux that packets appear to come from the hardware like this indicating that the hardware is doing header/data split, but maybe they never split non-tcp/udp packets.

I’m really hoping that it’s just something else dumb I’m doing but I can’t think what. NDISTest doesn’t tell me anything is wrong.

I won’t be able to support packets > 4096 bytes if cisco vpn can’t handle the split either, although I suspect that jumbo frames end-to-end between the vpn client and the concentrator would be an unlikely scenario.

I’m testing with NDIS5 under 2003 but have been told that the same problem exists with my NDIS6 driver, which would make sense as the cisco vpn client is fundamentally the same afaik.

Any suggestions appreciated, especially someone who can tell me authoritatively that cisco vpn cannot handle packets than span more than one buffer!

Thanks

James

I can’t speak to the Cisco VPN specifics as I have never worked on that VPN
product. However I would suggest that you identify what version of “Cisco
VPN” you are talking about as there have been a few. You might also right
up front identify if the particular Cisco VPN you are having fun with is one
that uses the DNE IM Driver as its packet interception/injection facility.

You mention header-data-split but say you are testing with NDIS5 on 2K3. I
am a bit confused. The miniport must obey the look-a-head specification
set in OID_GEN_CURRENT_LOOKAHEAD.

I am also confused by what the symptom is. Are you saying that having the
VPN product (driver) installed causes networking to fail or are you saying
that the VPN does not work but everything else is fine?

If it is the later then can you whip up a test version of your miniport that
flattens the frames?

You might also try indicating multi-buffer frames with NDIS_STATUS_RESOURCES
(as a test) to force the bound protocol/IM to copy the frame; again, as a
test only.

If you are dealing with the DNE based VPN, makes sure you go off and get the
latest DNE (Citrix, last I knew, was the current owner of that).

Good Luck,
Dave Cattley

>

I can’t speak to the Cisco VPN specifics as I have never worked on that VPN
product. However I would suggest that you identify what version of “Cisco
VPN” you are talking about as there have been a few. You might also right
up front identify if the particular Cisco VPN you are having fun with is one
that uses the DNE IM Driver as its packet interception/injection facility.

Hmmm… I think it is just “Cisco VPN Client”, I didn’t know there were different versions. It does have a DNE service though.

You mention header-data-split but say you are testing with NDIS5 on 2K3. I
am a bit confused. The miniport must obey the look-a-head specification
set in OID_GEN_CURRENT_LOOKAHEAD.

My reasoning was that afaik the ndis5 and ndis6 cisco vpn products appear to be the same, so if the ndis6 product can handle header-data-split then in theory the ndis5 product should be capable of dealing with my multi-buffer packets.

I am also confused by what the symptom is. Are you saying that having the
VPN product (driver) installed causes networking to fail or are you saying
that the VPN does not work but everything else is fine?

The symptom is that esp packets < current_lookahead work fine, but larger ones just get dropped.

If it is the later then can you whip up a test version of your miniport that
flattens the frames?

Each packet buffer is already PAGE_SIZE bytes long, and the populated data is always at the start, so flattening the packets up to PAGE_SIZE bytes is easy and doesn’t appear to have a measurable impact on performance (and may be better as the subsequent buffers that held the now-consolidated fragments are now free to be reused), but once I get a jumbo packet of PAGE_SIZE bytes such flattening would be a lot more work. Given that a VPN would typically work across a WAN link, and WAN links would normally not be using jumbo frames I’m probably safe in this instance, I just have this nagging feeling I might be doing something else wrong.

You might also try indicating multi-buffer frames with
NDIS_STATUS_RESOURCES
(as a test) to force the bound protocol/IM to copy the frame; again, as a
test only.

Hmm… I did see a note in the docs that I should do this for Win2K and XP (or turn off some copy lookahead function) but didn’t think of it as a test.

If you are dealing with the DNE based VPN, makes sure you go off and get
the latest DNE (Citrix, last I knew, was the current owner of that).

I’ll get the customer to try that too.

Thanks

James

Be aware also that ESP is only one of the IP protocols that Cisco routers
and concentrators can be configured to use for VPN. GRE and AH can also be
used as well as ISAKMP. I don’t know anything about their products for
Windows, but between firewalls, routers and concentrators there are several
different types of VPN that broadly break down into peer-to-peer and dialup;
and both categories can use any of the several protocols so it is important
to know what exactly you are working with.

“James Harper” wrote in message news:xxxxx@ntdev…

I can’t speak to the Cisco VPN specifics as I have never worked on that
VPN
product. However I would suggest that you identify what version of
“Cisco
VPN” you are talking about as there have been a few. You might also
right
up front identify if the particular Cisco VPN you are having fun with is
one
that uses the DNE IM Driver as its packet interception/injection facility.

Hmmm… I think it is just “Cisco VPN Client”, I didn’t know there were
different versions. It does have a DNE service though.

You mention header-data-split but say you are testing with NDIS5 on 2K3.
I
am a bit confused. The miniport must obey the look-a-head specification
set in OID_GEN_CURRENT_LOOKAHEAD.

My reasoning was that afaik the ndis5 and ndis6 cisco vpn products appear to
be the same, so if the ndis6 product can handle header-data-split then in
theory the ndis5 product should be capable of dealing with my multi-buffer
packets.

I am also confused by what the symptom is. Are you saying that having
the
VPN product (driver) installed causes networking to fail or are you saying
that the VPN does not work but everything else is fine?

The symptom is that esp packets < current_lookahead work fine, but larger
ones just get dropped.

If it is the later then can you whip up a test version of your miniport
that
flattens the frames?

Each packet buffer is already PAGE_SIZE bytes long, and the populated data
is always at the start, so flattening the packets up to PAGE_SIZE bytes is
easy and doesn’t appear to have a measurable impact on performance (and may
be better as the subsequent buffers that held the now-consolidated fragments
are now free to be reused), but once I get a jumbo packet of PAGE_SIZE bytes
such flattening would be a lot more work. Given that a VPN would typically
work across a WAN link, and WAN links would normally not be using jumbo
frames I’m probably safe in this instance, I just have this nagging feeling
I might be doing something else wrong.

You might also try indicating multi-buffer frames with
NDIS_STATUS_RESOURCES
(as a test) to force the bound protocol/IM to copy the frame; again, as a
test only.

Hmm… I did see a note in the docs that I should do this for Win2K and XP
(or turn off some copy lookahead function) but didn’t think of it as a test.

If you are dealing with the DNE based VPN, makes sure you go off and get
the latest DNE (Citrix, last I knew, was the current owner of that).

I’ll get the customer to try that too.

Thanks

James

> > You might also try indicating multi-buffer frames with

> NDIS_STATUS_RESOURCES
> (as a test) to force the bound protocol/IM to copy the frame; again, as a
> test only.

Hmm… I did see a note in the docs that I should do this for Win2K and XP (or
turn off some copy lookahead function) but didn’t think of it as a test.

I just tried that and it hasn’t made any difference. I guess the cisco vpn client just doesn’t like packets with multiple buffers.

James

>

Be aware also that ESP is only one of the IP protocols that Cisco routers
and concentrators can be configured to use for VPN. GRE and AH can also be
used as well as ISAKMP. I don’t know anything about their products for
Windows, but between firewalls, routers and concentrators there are
several different types of VPN that broadly break down into peer-to-peer and
dialup;
and both categories can use any of the several protocols so it is important
to know what exactly you are working with.

My proposed fix is going to be to coalesce buffers together when the protocol is not UDP or TCP. My only concern is that it’s also possible to run IPSEC over UDP and there may be some limitation there.

Thanks

James

Again, all caveats that this is not my driver nor product included by
reference …

To the best of my knowledge there is not an NDIS6 version of this client,
only an NDIS5 version. In other words, it includes an NDIS5 virtual
Ethernet adapter for the VPN interface and an NDIS5 IM Driver (DNE + Cisco
plugin) for the IPSec engine & other data-plane processing. So the idea
that it can handle header-data-split is purely because the NDIS6->NDIS5 ship
copies multi-MDL receive NBs into flattened single NDIS_BUFFER NDIS_PACKET
indications. Or so it seems to me.

I’m not sure that matters.

Cheers,
Dave Cattley