Subject: Using virtual PPP for IPv6 tunneling.

From: Peter Tattam <[email-masked]>
Date: Mon, 5 Jul 1999 13:21:29 +1000 (EST)

While the solution I knocked together for dynamic tunneling is ok, while
developing it, I became aware that there is another way to do the dynamic
tunneling that would work much better, and this is by setting up a virtual PPP
session over an TCP session.   I believe this approach has been applied to one
of the tunnel broker instances.

Could those involved let me know how it's done so I can implement a compatible
solution in Trumpet Winsock. As I already have working PPP code, I can easily
port this to go across a TCP connection, and I even have an opprotunity to test
out IPv6 header compression.

I believe a PPP solution may be the most efficent one for the deployment of
tunnels over IPv4 as TCP header compression would aply to the IPv4 component,
and the new draft standard for compression can be applied to the IPv6
component.  It would also be able to tunnel reliably through firewalls and
NAT's as it would not require the use of a IPv6 over IP tunnels, but rather
would directly interface as a PPP connection to the IPv6 subsystem at both
ends.

From my point of view, the implementation could be *very* simple.  Merely
connect to a well known port on the PPP tunnel server and negotiate all PPP
functions over the virtual circuit.  Getting a usable IPv6 address would be
accomplished via ND (router advertise) from the tunnel server.

While I have reviewed the version that is being used across Linux, I have not
tried it as it requires ssh do do its job.  I believe a solution could be found
which avoids this.  I am also unclear as to whether it does actually implement
a virtual PPP connection over TCP or whether the control connection is just to
manage the configuration of a traditional tunnel at both ends.

If a successful compromise can be found, I will be releasing a Win32 capable
Tunnel server which would complement the client.  I would als be incorporating
the client directly into our client TCP/IP package.

Peter

--
Peter R. Tattam                            [email-masked]
Managing Director,    Trumpet Software International Pty Ltd
Hobart, Australia,  Ph. +61-3-6245-0220,  Fax +61-3-62450210

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: "Perry E. Metzger" <[email-masked]>
Date: 04 Jul 1999 23:36:36 -0400

Peter Tattam <[email-masked]> writes:
> session over an TCP session.

That ends up producing Bad Mojo. The fight between the two layers of
congestion control algorithms is frightening to watch. Better to use
other mechanisms that don't have TCP at the bottom.

Perry

---

From: Craig Metz <[email-masked]>
Date: Mon, 05 Jul 1999 00:03:48 -0400

In message <[email-masked]>, yo
>of the tunnel broker instances.

  This sounds like a way to accomplish the same thing as IPv6-in-IPv4 tunnels
but with much greater protocol overhead (read: waste).

									-Craig

---

From: Bill Sommerfeld <[email-masked]>
Date: Mon, 05 Jul 1999 00:10:20 -0400

While I have no direct experience with IPv6 over PPP over TCP for
tunnelling, I have been regularly using IP over PPP over SSH over TCP
tunnels for quite some time as a "poor man's ipsec".

> While the solution I knocked together for dynamic tunneling is ok,
> while developing it, I became aware that there is another way to do
> the dynamic tunneling that would work much better, and this is by
> setting up a virtual PPP session over an TCP session.

This is very easy to configure, but is likely to have poorer
performance than a "regular" tunnel, particularly for TCP traffic
through the tunnel.

TCP trades latency for reliability, but doesn't deal well with link
layers which involve variable latency.  The short answer is that TCP
doesn't run over TCP very well.

other gotchas: 
 1) Unless you're careful configuring the MTU's of the PPP
link, maximum-sized PPP frames will probably be fragmented into
multiple TCP segments, which will subject you to additional latency
from slow start and the Nagle algorithm.

 2) In the presence of packet loss on the path between the tunnel
endpoints, both TCP's will end up retransmitting.  

Packets for the inner tcp will end up queued in the tcp send queue at
the tunnel endpoints as well as backed up in the PPP send queue.

Once the packets get through, the inner tcp will likely see a flood of
duplicate acks, which may prod the tcp fast retransmission algorithm
into action, provoking a few more duplicate packets which might by
themselves cause congestion and loss.  Bad news..

					- Bill

---

From: Bill Sommerfeld <[email-masked]>
Date: Mon, 05 Jul 1999 00:19:14 -0400

A couple additional notes..

Since you brought up NATs and firewalls as obstacles to work around:

 - if tunnelling through NAT is desired, an IPv6 over *UDP* tunnel is
likely to be able to get through a NAT without too much trauma.

 - tunnelling IPv6 through a firewall without the knowledge and
consent of the firewall's manager has great potential to be a career
limiting move.

					- Bill

---

From: Craig Metz <[email-masked]>
Date: Mon, 05 Jul 1999 00:32:44 -0400

>likely to be able to get through a NAT without too much trauma.

  Somewhere in my todo pile is a draft on using UDP as a chained header. I use
it every day in order to do a secure tunnel through the highly broken NAT
implementation I'm stuck behind. The same problem, and the same solution,
should work just as well for IPv6. So you'd build your packets like:

  [ IPv4 p=UDP ] [ UDP dport=whatever ] [pad, nexthdr=41 ] [ IPv6 ... ]

  If this is something that people would be interested in building, I can
dequeue this particular task and post a draft.

									-Craig

---

From: Peter Tattam <[email-masked]>
Date: Mon, 5 Jul 1999 15:23:15 +1000 (EST)

> but with much greater protocol overhead (read: waste).

Boy, did my idea cop it or what.  I forgot to don my flame proof clothing :)

I forgot to mention that I am talking in terms of using this only for real
client PPP connections, not necessarily for normal tunnels.

When I crunched the numbers and am not quite so convinced as to wastage.

For an IPv6 tunnel, you would have 

IPv4 header (prot=41),  uncompressible over VJ compression. (20 bytes)
Ipv6 header.  Uncompressible with most tunnels. (40 bytes)

We are looking at minimum 60 bytes per tunnelled packet, more often than not
carrying TCP or UDP traffic.  PPP framing and overhead would still apply as we
are already PPP over IPv4.

Now if virutal PPP was running over a TCP session, you would have VJ affecting
the tunnel traffic directly - our 20 bytes of prot 41 down to around 4-5 bytes. 

There would not be the IPv4 encapsulation so one can apply the latest IPv6
compression techniques to the raw IPv6 packet. For TCP traffic, this would
equate to around 4-5 bytes or so.

I would see on average an improvement of around 4 to compression on the headers
if this approach were taken for live PPP connections.

I accept your comments about TCP latency & data aggregation, which I believe
could be dealt with by TCP_NODELAY options if the stacks at either end are
suffiently flexible. Since we know that the underlying data is already
packetized, I don't see any performance hits over using a normal tunnel except
for reliable delivery getting in the way.  With TCP_NODELAY off, you may
actually get some kind of improvement in overheads as many small IPv6 packets
could get aggregated into one lump.  As long as you kept your packetizing
routines under the MTU, I think it could sort itself out, but I could be
totally wrong.  As far as TCP reties are concerned, if the tunnel server is
realtively close to the client, I think the extent of retries over the local
IPv4 link would be minimal.  If the server is much more remote, I admit there
would be problems.  The only way of alleviating this would be adding some
intelligence to the protocol to prevent congestion - i.e. short circuit the
stream when it gets too many packets.

The other idea of using UDP results in slightly more overhead over using normal
tunneling, but has the advantage of being able to get through a NAT.  It does
however suffer in that the native PPP connection will not be able to compress
the headers.  Tunneling overhead in this case would be 28 bytes.

My primary motive for using a TCP stream is to gain advantage of the widespread
use of VJ compression in regular PPP terminal servers.

Important in the whole discussion however is that the biggest overhead savings
actually are made with the Ipv6 header compression.  Perhaps we should be
pushing to make this standard on any dynamic IPv6 tunnel server.  In fact I
thought about this for other reasons, but without the ability to specify
different framing for the different compressed IPv6 packet types, I'm sunk.  If
I'm allowed to use the IP version to specify framing for the other compression
packet types, it could be done....  (unless we allow the next_header to have
specific values for the compressed frames - a bad idea as routing filters would
need to be extended past prot=41) 

Anyway, I could put something together and test the performance between the two
as a benchmark.  Who knows, it might still work.  Nothing ventured, nothing
gained.

Peter

---

From: Craig Metz <[email-masked]>
Date: Mon, 05 Jul 1999 01:36:53 -0400

In message <[email-masked]>, you
 write:
[[email-masked] snipped; this is a protocol discussion now ;)]
>My primary motive for using a TCP stream is to gain advantage of the widesprea

  Most modern terminal servers also support some LZ-ish compression. If you
really want to save bits, that's what you should be using anyway. I believe
that LZ-ish compression will do better with tunnelling in UDP than in TCP, as
the UDP header is very fixed and the TCP header has sequence numbers moving.
VJ compression is and always has been a kluge. A kluge that happens to work
really well in the common case, but it's still at the end of the day a kluge.

  The big issue is that TCP delivers properties that are very different than
what IP does, while UDP delivers properties that are approximately the same as
what IP does. If you run TCP -- or some other reliable-ish transport -- on
top of TCP, odd things probably happen. You may find that the change in
properties confuses the inner TCP enough to wipe out any performance gains you
would otherwise get from header compression. But I bet that it would work fine
in the common case; there are a lot of people out there using TCP-based tunnels
and they do seem to get along. What happens to those tunnels when things aren't
so good on the network isn't so clear, and that's where I'm guessing it would
misbehave.

  It's also inefficient to run this over TCP, and not just in bits, but also in
terms of both ends' stacks. Remember that TCP is a very stateful thing and
you also have TCP segmentation/reassembly and sequencing thrown in there, while
UDP requires no such state at the two ends.

  Doing this over TCP would work. But I don't think it is a good idea. OTOH, I
have enough operational experince running over UDP to know that it works and
doesn't do anything wacky, it just decreases your PMTU a bit.

									-Craig

---

From: Peter Tattam <[email-masked]>
Date: Mon, 5 Jul 1999 15:53:33 +1000 (EST)

> doesn't do anything wacky, it just decreases your PMTU a bit.

Ok, so the general consensus is that there's not a great deal to be gained from
compressing the IPv4 tunnel whether it be over prot 41 or UDP, except that UDP
can get through a NAT, and that LZ-ish compression (e.g. typical modem) will do
a lot for you already.   If that were the case, I would hazard a guess that the
LZ-ish compression would apply to the whole packet obviating the need for IPv6
header compression no?

If that whole approach to header compression is valid then why are we bothering
with the newer header compression protocol at all.  Shoudln't we just let other
link layers do it instead.

Some emperical tests could be made of typical modem connection to see if there
really is a significant difference between IPv6 tunneled telnet session to a
raw Ipv4 telnet session.   Surely there are some good rules of thumb already?

Just to throw another spanner in the works - wait till you hear this one.  I
had a customer requesting how they could open a raw socket on the TCP port so
that they could send their packets to take advantage of VJ compression for
modem PPP traffic.  Sound familiar?

Peter

P.S. in Aus, we are byte counters because we get charged by the megabyte for
internet traffic.  Issues like this could be important if I want to roll out
IPv6 locally.

---

From: Bill Sommerfeld <[email-masked]>
Date: Mon, 05 Jul 1999 23:52:00 -0400

> I would have thought TCP would be more likely to go through NAT cleanly
> than UDP. 

It depends on the NAT and how it sets up the address/port mappings.
The ones I'm familiar with deal about the same with UDP as with TCP.

> After all, TCP is full of well-known connection associations
> that tie all the packets to a part of a known stream. UDP, even
> connected UDP, is (apparently) stateless from the firewall's POV.

UDP has both source and destination ports just like tcp; not that i've
ever implemented the guts of a NAT, but it would seem to be pretty
straightforward to map UDP just like you map TCP..

In comparison, IP-in-IP lacks ports, and AH and ESP have just the SPI,
which is unidirectional (can't be used to figure out where to reply
to).

						- Bill

---

From: Peter Tattam <[email-masked]>
Date: Wed, 7 Jul 1999 00:26:16 +1000 (EST)

> straightforward to map UDP just like you map TCP..

Having implemented a NAT, I can speak from some experience.  TCP is slightly
better than UDP in that it is stateful, and you can reuses resources more
quickly.  With UDP, there's no way to tell how long an address mapping tuple is
going to last for.  At least with TCP, you have SYN/FIN/RST packets to give you
hints.  I haven't actually used the stateful info in my NAT though, thus TCP
and UDP can be treated identically.

Peter

---

From: Dancer <[email-masked]>
Date: Mon, 05 Jul 1999 18:05:35 +1000

I would have thought TCP would be more likely to go through NAT cleanly
than UDP. After all, TCP is full of well-known connection associations
that tie all the packets to a part of a known stream. UDP, even
connected UDP, is (apparently) stateless from the firewall's POV.

I don't suppose you'd care to expand a little further?

> consent of the firewall's manager has great potential to be a career

*amen* to that. It's also bad if the firewall's manager waves a
feather-duster around like a sword.

---

From: Dancer <[email-masked]>
Date: Mon, 05 Jul 1999 17:58:03 +1000

> other mechanisms that don't have TCP at the bottom.

Well, in a pinch you could connect a UDP socket. It mimics the supposed
reliability of the underlying network link, will run point-to-point, and
PPP should be relatively happy with it. You also shouldn't need to
inform anything about the change to the transport layer.

IMO, though, if you want scary: PPP and the error-correction on any
modern modem can do some pretty bleak things to flow and congestion too.
The modem may perform all sorts of guff, while trying to keep the
link-layer ignorant of it all.

D

---

From: Dancer <[email-masked]>
Date: Mon, 05 Jul 1999 18:01:48 +1000

> themselves cause congestion and loss.  Bad news..

Yes, pathological loss bites, rather.

When you add the route, modify the window size to (say) 4096, and set
the txqueuelen down to something low. Like 1. This really blows away udp
traffic (like DNS and ICMP and so forth) but will make your tcp over ppp
over tcp work a lot more smoothly.

D

---

From: Dancer <[email-masked]>
Date: Mon, 05 Jul 1999 17:58:03 +1000

D

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 6 Jul 1999 19:00:10 +0100 (BST)

> link-layer ignorant of it all.

I keep expecting someone to build an 'IP only' modem with a lot of
redundant-with-PPP stuff stripped out.

Doesn't seem to have happened - even with software modems.

L.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>