Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

cisco vpn and multiple buffers per packet

James_HarperJames_Harper Member Posts: 1,615
This is the same sort of problem I've mentioned previously but I have investigated a bit further now.

Normally I start off investigating obvious network errors (eg ping < 100 bytes works but ping >= 100 bytes doesn't) with tools like tcpdump and wireshark, looking for errors that way. In this case though, the ESP (ip proto 50) packets never show up in wireshark, which says to me that the cisco vpn driver grabs them at a fairly low level in the network stack.

My driver often receives packets from xen with the eth+ip(+udp/tcp/etc) header in one buffer and the rest of the data in another. Is it possible that cisco vpn (testing with latest version) simply cannot handle packets with more than one attached buffer? Do other network adapters never do this? I see under Linux that packets appear to come from the hardware like this indicating that the hardware is doing header/data split, but maybe they never split non-tcp/udp packets.

I'm really hoping that it's just something else dumb I'm doing but I can't think what. NDISTest doesn't tell me anything is wrong.

I won't be able to support packets > 4096 bytes if cisco vpn can't handle the split either, although I suspect that jumbo frames end-to-end between the vpn client and the concentrator would be an unlikely scenario.

I'm testing with NDIS5 under 2003 but have been told that the same problem exists with my NDIS6 driver, which would make sense as the cisco vpn client is fundamentally the same afaik.

Any suggestions appreciated, especially someone who can tell me authoritatively that cisco vpn cannot handle packets than span more than one buffer!

Thanks

James

Comments

  • David_R._CattleyDavid_R._Cattley Member - All Emails Posts: 2,112
    I can't speak to the Cisco VPN specifics as I have never worked on that VPN
    product. However I would suggest that you identify what version of "Cisco
    VPN" you are talking about as there have been a few. You might also right
    up front identify if the particular Cisco VPN you are having fun with is one
    that uses the DNE IM Driver as its packet interception/injection facility.

    You mention header-data-split but say you are testing with NDIS5 on 2K3. I
    am a bit confused. The miniport must obey the look-a-head specification
    set in OID_GEN_CURRENT_LOOKAHEAD.

    I am also confused by what the symptom is. Are you saying that having the
    VPN product (driver) installed causes networking to fail or are you saying
    that the VPN does not work but everything else is fine?

    If it is the later then can you whip up a test version of your miniport that
    flattens the frames?

    You might also try indicating multi-buffer frames with NDIS_STATUS_RESOURCES
    (as a test) to force the bound protocol/IM to copy the frame; again, as a
    test only.

    If you are dealing with the DNE based VPN, makes sure you go off and get the
    latest DNE (Citrix, last I knew, was the current owner of that).

    Good Luck,
    Dave Cattley
  • James_HarperJames_Harper Member Posts: 1,615
    >
    > I can't speak to the Cisco VPN specifics as I have never worked on that VPN
    > product. However I would suggest that you identify what version of "Cisco
    > VPN" you are talking about as there have been a few. You might also right
    > up front identify if the particular Cisco VPN you are having fun with is one
    > that uses the DNE IM Driver as its packet interception/injection facility.

    Hmmm... I think it is just "Cisco VPN Client", I didn't know there were different versions. It does have a DNE service though.

    >
    > You mention header-data-split but say you are testing with NDIS5 on 2K3. I
    > am a bit confused. The miniport must obey the look-a-head specification
    > set in OID_GEN_CURRENT_LOOKAHEAD.

    My reasoning was that afaik the ndis5 and ndis6 cisco vpn products appear to be the same, so if the ndis6 product can handle header-data-split then in theory the ndis5 product should be capable of dealing with my multi-buffer packets.

    > I am also confused by what the symptom is. Are you saying that having the
    > VPN product (driver) installed causes networking to fail or are you saying
    > that the VPN does not work but everything else is fine?
    >

    The symptom is that esp packets < current_lookahead work fine, but larger ones just get dropped.

    > If it is the later then can you whip up a test version of your miniport that
    > flattens the frames?

    Each packet buffer is already PAGE_SIZE bytes long, and the populated data is always at the start, so flattening the packets up to PAGE_SIZE bytes is easy and doesn't appear to have a measurable impact on performance (and may be better as the subsequent buffers that held the now-consolidated fragments are now free to be reused), but once I get a jumbo packet of PAGE_SIZE bytes such flattening would be a lot more work. Given that a VPN would typically work across a WAN link, and WAN links would normally not be using jumbo frames I'm probably safe in this instance, I just have this nagging feeling I might be doing something else wrong.

    > You might also try indicating multi-buffer frames with
    > NDIS_STATUS_RESOURCES
    > (as a test) to force the bound protocol/IM to copy the frame; again, as a
    > test only.

    Hmm... I did see a note in the docs that I should do this for Win2K and XP (or turn off some copy lookahead function) but didn't think of it as a test.

    >
    > If you are dealing with the DNE based VPN, makes sure you go off and get
    > the latest DNE (Citrix, last I knew, was the current owner of that).

    I'll get the customer to try that too.

    Thanks

    James
  • MBondMBond Member - All Emails Posts: 846
    Be aware also that ESP is only one of the IP protocols that Cisco routers
    and concentrators can be configured to use for VPN. GRE and AH can also be
    used as well as ISAKMP. I don't know anything about their products for
    Windows, but between firewalls, routers and concentrators there are several
    different types of VPN that broadly break down into peer-to-peer and dialup;
    and both categories can use any of the several protocols so it is important
    to know what exactly you are working with.

    "James Harper" wrote in message news:xxxxx@ntdev...

    >
    > I can't speak to the Cisco VPN specifics as I have never worked on that
    > VPN
    > product. However I would suggest that you identify what version of
    > "Cisco
    > VPN" you are talking about as there have been a few. You might also
    > right
    > up front identify if the particular Cisco VPN you are having fun with is
    > one
    > that uses the DNE IM Driver as its packet interception/injection facility.

    Hmmm... I think it is just "Cisco VPN Client", I didn't know there were
    different versions. It does have a DNE service though.

    >
    > You mention header-data-split but say you are testing with NDIS5 on 2K3.
    > I
    > am a bit confused. The miniport must obey the look-a-head specification
    > set in OID_GEN_CURRENT_LOOKAHEAD.

    My reasoning was that afaik the ndis5 and ndis6 cisco vpn products appear to
    be the same, so if the ndis6 product can handle header-data-split then in
    theory the ndis5 product should be capable of dealing with my multi-buffer
    packets.

    > I am also confused by what the symptom is. Are you saying that having
    > the
    > VPN product (driver) installed causes networking to fail or are you saying
    > that the VPN does not work but everything else is fine?
    >

    The symptom is that esp packets < current_lookahead work fine, but larger
    ones just get dropped.

    > If it is the later then can you whip up a test version of your miniport
    > that
    > flattens the frames?

    Each packet buffer is already PAGE_SIZE bytes long, and the populated data
    is always at the start, so flattening the packets up to PAGE_SIZE bytes is
    easy and doesn't appear to have a measurable impact on performance (and may
    be better as the subsequent buffers that held the now-consolidated fragments
    are now free to be reused), but once I get a jumbo packet of PAGE_SIZE bytes
    such flattening would be a lot more work. Given that a VPN would typically
    work across a WAN link, and WAN links would normally not be using jumbo
    frames I'm probably safe in this instance, I just have this nagging feeling
    I might be doing something else wrong.

    > You might also try indicating multi-buffer frames with
    > NDIS_STATUS_RESOURCES
    > (as a test) to force the bound protocol/IM to copy the frame; again, as a
    > test only.

    Hmm... I did see a note in the docs that I should do this for Win2K and XP
    (or turn off some copy lookahead function) but didn't think of it as a test.

    >
    > If you are dealing with the DNE based VPN, makes sure you go off and get
    > the latest DNE (Citrix, last I knew, was the current owner of that).

    I'll get the customer to try that too.

    Thanks

    James
  • James_HarperJames_Harper Member Posts: 1,615
    > > You might also try indicating multi-buffer frames with
    > > NDIS_STATUS_RESOURCES
    > > (as a test) to force the bound protocol/IM to copy the frame; again, as a
    > > test only.
    >
    > Hmm... I did see a note in the docs that I should do this for Win2K and XP (or
    > turn off some copy lookahead function) but didn't think of it as a test.
    >

    I just tried that and it hasn't made any difference. I guess the cisco vpn client just doesn't like packets with multiple buffers.

    James
  • James_HarperJames_Harper Member Posts: 1,615
    >
    > Be aware also that ESP is only one of the IP protocols that Cisco routers
    > and concentrators can be configured to use for VPN. GRE and AH can also be
    > used as well as ISAKMP. I don't know anything about their products for
    > Windows, but between firewalls, routers and concentrators there are
    > several different types of VPN that broadly break down into peer-to-peer and
    > dialup;
    > and both categories can use any of the several protocols so it is important
    > to know what exactly you are working with.

    My proposed fix is going to be to coalesce buffers together when the protocol is not UDP or TCP. My only concern is that it's also possible to run IPSEC over UDP and there may be some limitation there.

    Thanks

    James
  • David_R._CattleyDavid_R._Cattley Member - All Emails Posts: 2,112
    <quote>
    My reasoning was that afaik the ndis5 and ndis6 cisco vpn products appear to
    be the same, so if the ndis6 product can handle header-data-split then in
    theory the ndis5 product should be capable of dealing with my multi-buffer
    packets.
    </quote>

    Again, all caveats that this is not my driver nor product included by
    reference ...

    To the best of my knowledge there is not an NDIS6 version of this client,
    only an NDIS5 version. In other words, it includes an NDIS5 virtual
    Ethernet adapter for the VPN interface and an NDIS5 IM Driver (DNE + Cisco
    plugin) for the IPSec engine & other data-plane processing. So the idea
    that it can handle header-data-split is purely because the NDIS6->NDIS5 ship
    copies multi-MDL receive NBs into flattened single NDIS_BUFFER NDIS_PACKET
    indications. Or so it seems to me.

    I'm not sure that matters.

    Cheers,
    Dave Cattley
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Developing Minifilters 29 July 2019 OSR Seminar Space
Writing WDF Drivers 23 Sept 2019 OSR Seminar Space
Kernel Debugging 21 Oct 2019 OSR Seminar Space
Internals & Software Drivers 18 Nov 2019 Dulles, VA