Subject: [rrg] IRON: SEAL summary

From: Robin Whittle <[email-masked]>
Date: Mon, 08 Feb 2010 20:36:47 +1100

This is a summary of my current probably imperfect understanding of
the parts of Fred Templin's SEAL tunneling protocol which are
important for his scalable routing proposal: RANGER (now IRON).

Please consult any discussion which follows - I wrote this without
checking it first with Fred, so he will no provide corrections.

At the end I compare this understanding of SEAL with my IPTM
arrangement for Ivip, and with the non-stateful and stateful
approaches of LISP.

  - Robin

Fred's "RANGER" scalable routing proposal is now known as IRON:

   The Internet Routing Overlay Network (IRON)
   http://tools.ietf.org/html/draft-templin-iron-00

The proposal as currently described:
   http://tools.ietf.org/html/draft-irtf-rrg-recommendation-04#section-16

is a particular application of Fred's more general-purpose RANGER:

   http://tools.ietf.org/html/draft-templin-ranger-09

which itself is based on ISATAP (RFC 5214).  RANGER and therefore
IRON uses SEAL for tunneling with Path MTU Discovery management
functions.

Next I will write up my understanding of IRON.

The latest SEAL ID is:

  http://tools.ietf.org/html/draft-templin-intarea-seal-08

Perhaps SEAL is capable of tunneling IPv4 packets in IPv6 tunnels and
vice-versa, but I am only interested in SEAL for pure IPv4 and pure
IPv6, since this is how I am trying to understand IRON.

SEAL is intended to be suitable for the extremely ad-hoc tunneling
arrangements found in Core-Edge Separation solutions to the routing
scaling problem, where an ITR may have a sudden need to tunnel one or
more packets to an ETR it has never had anything to do with.

In SEAL, the router (or some function programmed into a server,
rather than a conventional router) which accepts packets and tunnels
them is known as the ITE (Ingress Tunnel Endpoint).  The router or
other device which the tunnel reaches to, and which decapsulates the
inner packet, is the ETE (Egress Tunnel Endpoint).

Please see the discussions between Fred and me in the thread: "SEAL
critique, PMTUD, RFC4821 = vapourware".

  http://www.ietf.org/mail-archive/web/rrg/current/msg05816.html RW
  http://www.ietf.org/mail-archive/web/rrg/current/msg05834.html   FT
  http://www.ietf.org/mail-archive/web/rrg/current/msg05843.html RW
  http://www.ietf.org/mail-archive/web/rrg/current/msg05902.html   FT
  http://www.ietf.org/mail-archive/web/rrg/current/msg05924.html RW
  http://www.ietf.org/mail-archive/web/rrg/current/msg05927.html   FT
  http://www.ietf.org/mail-archive/web/rrg/current/msg05976.html RW

SEAL for IPv4 and IPv6
----------------------

SEAL is capable of "segmenting" (fragmenting, but within the SEAL
protocol, rather than by using IPv4 or IPv4 fragmentation mechanisms)
packets which are known to be too long for the PMTU to a given ETE.
However, this is not intended to be used with IRON.

SEAL is intended to tunnel packets to an ETE, without any need to
establish tunneling arrangements, without expecting acknowledgements
of successful receipt by the ETE and without resending any packets.
The tunnel is a one-way (ITE to ETE), ad-hoc, arrangement - but the
ITE stores some state for each particular ETE it is tunneling to.

SEAL ITEs do not cache any part of the packets they send.  So in
order to generate a PTB to the sending host (which may itself be
another router, if this SEAL tunnel is already within an outer tunnel
- which I think is not the case with IRON) the ITE relies on
receiving enough of the packet from (IPv6-only) a PTB from a limiting
router on the path to the ETE, or (IPv4-only) a "Fragmentation
Experienced" message from the ETE itself.

I assume that for IRON, there will be no need for "mid-level headers"
such as UDP between the outer header and the SEAL (or IPv6
Fragmentation) header - but see msg05927 and msg05976 and search for
"ECMP".

There is some pretty confusing text about the Tunnel Interface MTU -
4.3.1.  One part discusses setting this to 1500 bytes or more.  Later
in that paragraph there is discussion of setting it to smaller values
than this. The next paragraph discusses setting it to an infinite
value.  I hope Fred will provide some guidance on how this would be
done for IRON.

Each ITE maintains some state for each ETE it tunnels packets to (by
each ETE IP address).  I guess there would be a process for deleting
this after a while if no packets are sent to that ETE, since over
time, the state could grow to a considerable size, depending on how
many ETEs there are in the world.

    SEAL-ID most recently used.  A 32 bit value which is initialised
    to some random value, and then incremented modulo 2^32 every time
    a packet is tunneled to this particular ETE.

    Further state as required to implement a window function or
    some other arrangement by which this ITE can test a SEAL-ID
    in an incoming message including at least these:

       PTB from a router in the tunnel path (IPv6 only).
       SEAL Packet Too Big message from ITE (Not needed in IRON?).
       SEAL "Fragmentation Experienced" (IPv4 only).

    Please see the last two messages listed above for Fred's approach
    to doing this and my critique and timer-based suggestion.

Other items of state are listed in 4.3.3:

    MHLEN   Constant mid-level header length.  In this attempt to
            describe SEAL, I will assume there is no need for these
            mid-level headers - between the outer IP header and the
            SEAL (or IPv6 fragmentation) header which precedes the
            encapsulated packet.  So I will assume this = 0.

    HLEN    Constant outer header length: 20 for IPv4 and 40 for
            IPv6 plus the length of the SEAL header (IPv4 only -
            8 bytes) or IPv6 Fragment Header (8 bytes).  So these
            are constants:

               IPv4 28           IPv6 48

    S_MSS   Variable.  SEAL Maximum Segment Size.  I think this is
            initialised to the value of (the ID says "no larger
            than") the MTU of the "underlying IP interface".  I guess
            this means that the ITE has a single interface for
            sending out encapsulated packets on.  (To be pernickety,
            it could be pointed out that the ITE might be a multiple
            interface router, with different MTUs on each, and that
            the best path to a given ETE might change from one
            interface to another.)

            According to the ID, this value may be adjusted upwards
            and downwards based on received SEAL Reassembly Report
            messages.

            However, I think these are not of interest in IRON - and
            that it is other messages which alter this value which
            we need to consider:

              IPv4: SEAL "Fragmentation Experienced" from the ETE.

              IPv6: PTB from a router in the tunnel path.

    S_MRU   Variable.  SEAL Maximum Reassembly Unit.  Initialised to
            "infinity", but the effective value of S_MRU is never
            more than 256 * S_MSS.  (Since S_MSS can rise and fall,
            this means there are really two items of state: one the
            limit and the other the effective value, which may be
            increased or decreased according to S_MSS * 256, as long
            as it remains less than or equal to the limit value.)

SEAL for IPv4
-------------

DF=0 packets will be fragmented with standard IPv4 techniques before
other processing, if they are longer than:

   (MIN(S_MRU, S_MSS) - 28)

The fragments will be of this length and will be tunneled as
described below.

DF=1 packets which are longer than:

   (S_MRU - 28)

will result in a PTB being sent to the Sending Host (SH).  Apart from
being used to generate that PTB, the packet will be dropped.

Encapsulation involves:

  IPv4 outer header   With the 16 Identification bit field set to
                      the 16 most significant bits of SEAL-ID.
                      DF = 0, so the packet is fragmentable by
                      any router which finds the packet is too long
                      for its next-hop MTU.

  SEAL header (32 bits) as described below.

  The original traffic packet.

The SEAL header (only for IPv4) has a 16 bit field "ID-Extension"
which is set to the least significant 16 bits of SEAL-ID.  Section
4.2 discusses this.  I haven't figured out exactly how the other bits
would be set.

If the packet arrives intact, then the ETE decapsulates it and
forwards the traffic packet to wherever it needs to go.  In IRON,
most of the time, there is a single tunnel to the IRON router which
directly connects to the end-user network.  However initial packets
go to an IRON router (the VP router) which is typically not the
router which connects to the end-user network - so then the packet
would be decapsulated and tunneled in the same manner to the IRON
router which connects to the destination network.

If the packet is too long for one router in the ITE -> ETE path, then
that router will fragment the whole packet and the ETE will (usually)
receive the first and other fragments.  The first fragment should be
as long as the limit imposed at that router by the next-hop MTU.  If
there were two or more MTU limits, such as 1400 and then 1300, the
fragment generated at the first limiting router would be 1400 bytes
long, and this would be fragmented at the second router, so the first
fragment would be 1300 bytes.

In this way, the ETE discovers the PMTU of the ITE -> ETE tunnel path.

The ETE does not attempt to deliver the packet, but sends a
"Fragmentation Experienced" message to the ITE.  See 4.3.9.1.2 and
4.4.5.1.2.  This message is defined in Figure 6 and contains as much
of the first fragment as would make the total message 576 bytes in
length.  There are 20 bytes for the IPv4 header and 16 for this
particular SEAL header, so up to 540 bytes of the first fragment will
be sent back to the ITE.  There is no acknowledgement of reception by
the ITE.

The ITE can authenticate this message by looking at the 16 bit ID in
the IPv4 header of the enclosed portion of the first fragment - and
the 16 bits of "ID-Extension" in the SEAL header in that fragment.

The S_MSS field of this message is set to the length of the first
fragment, which is (or should be, and is assumed to be) the PMTU of
the ITE -> ETE path.

The S_MRU field may be set to zero.  As far as I know S_MRU is how
big a packet the ETE is prepared to reconstruct from packets which
are fragmented with IPv4's native fragmentation - but it may also
apply to the use of SEAL's internal segmentation system, which I
understand is not generally used for IRON.

There is a separate SEAL PTB message (4.4.5.1.3) which as far as I
know, is not relevant to IRON - since I think it concerns SEAL's
internal "segmentation" system.

I understand that the reason Fred uses DF=0 packets for IPv4 tunnels
is that IPv4 routers, when sending back a PTB, are required (by RFC
1191) to only return the IPv4 header and the next 8 bytes.  This is
enough for the ITE to authenticate the PTB as genuine, since these 8
bytes are the SEAL header. With the IPv4 header, the ITE can
therefore see the full 32 bit SEAL-ID.  However, there is nothing of
the original traffic packet in the PTB, so the only way the ITE could
generate a PTB to the SH would be to have cached the first 28 bytes
of the original traffic packet.  To avoid the need for caching, and
so as not to rely on PTBs from routers in the ITE -> ETE tunnel, SEAL
ITEs tunnel IPv4 packets using DF=0 packets.

The SEAL header used in this IPv4 process contains a flag which, when
set, signals that the ETE should report the successful reception of
the packet to the ITE.  I am not sure to what extent this would be
used for IRON.

SEAL for IPv6
-------------

DF=1 packets which are longer than:

   (S_MRU - 48)

will result in a PTB being sent to the Sending Host (SH).  Apart from
being used to generate that PTB, the packet will be dropped.

When encapsulating IPv6 packets, the SEAL ITE does not use a SEAL
header.  It uses the IPv6 Fragment Header:

  http://tools.ietf.org/html/rfc2460#section-4.5

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Next Header  |   Reserved    |      Fragment Offset    |Res|M|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Identification                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The single packet does not contain a "fragment" - this is just using
the header for another purpose.  This header contains a 32 bit
"Identification" field, which is set to the SEAL-ID value.

If the packet does not encounter any MTU limits and arrives at the
ETE, then the ETE decapsulates it and does whatever it needs to do
with it - deliver it directly to the end-user network or perhaps (if
the ETE is the VP IRON router) tunnel it again to an IRON router
which will deliver it to the end-user network.

If the packet hits an MTU limit, then the limiting router will send
back a PTB to the ITE, and since this is IPv6, the PTB will contain
enough of the packet for the ITE to construct a valid PTB for the SH.

There is no facility, as there was with IPv4, for the ITE to request
the ETE to confirm that the packet arrived.

Rejecting larger packets directly
---------------------------------

Once the ITE has established a value for S_MSS for a given ETE, then
any packet it needs to tunnel to that ETE will be rejected with a PTB
to the SH if, once it was encapsulated, it would be too long for the
S_MSS of this ETE.   So if a DF=1 IPv4 packet arrives with a length
longer than (S_MSS - 28), it will be rejected with a PTB.  Likewise
an IPv6 packet longer than (S_MSS - 48).

As far as I know, all IPv4 DF=0 packets longer than (S_MSS - 28) will
be fragmented into IPv4 fragments of this length (with the final one
potentially - typically - being shorter) and then the fragments will
be tunnelled.  At the ETE, each fragment will be forwarded to the
destination network and then host (or tunnelled to another IRON
router which will forward them to the destination network and then
host - and the destination host will reassemble the fragments using
the standard IPv4 system.)

The ITE can't reject any DF=0 packets.  It can however learn the PMTU
to the ETE by the ETE reporting the size of any fragments which
result from the tunneled versions of the IPv4 native fragments.  That
will enable the ITE to use IPv4 native fragmentation on subsequent
DF=0 packets to this ITE in order that they will fit within the newly
discovered PMTU limitation, which is stored in this ETE's S_MSS variable.

Adapting to changed PMTUs
-------------------------

I am not sure if or where it is specified, but I understand that SEAL
will allow exploration of increased PMTUs.  According to RFC 1191 and
therefore RFC 1981, the SH can try sending a longer packet than it
has been told to send by a previous PTB, after 10 minutes has elapsed.

I assume that if the ITE is only handling DF=0 packets, or at least
that if there are no DF=1 packets which are longer than the current
limitation (S_MSS - 28) and that if DF=0 packets keep arriving,
requiring native IPv4 fragmentation before encapsulation (that is,
the DF=0 packets are longer than (S_MSS - 28)) then after 10 minutes
or so, the ITE should explore sending larger packets into the tunnel.

Except when lost packets prevent it, the SEAL ITE will instantly
discover a reduced PMTU to a given ETE and so reduce that ETE's S_MSS
value, due to:

  IPv4:    Limiting router fragments the DF=0 tunnel packet and
           the ETE reports to the ITE the length of the first
           fragment, which is the new PMTU limit of this path.

           If the original packet was DF=1, the ITE will generate
           a suitable PTB to the SH.

  IPv6:    A PTB arrives at the ITE, and the ITE sends a PTB to the
           SH.

Jumboframes
-----------

("Jumbograms" refer to IPv6 packets with special formats so they can
be longer than 2^16 bytes, and can be as long as 4 gigabytes.
Neither SEAL, Ivip's IPTM, nor any CES proposal attempts to deal with
these.)

As far as I understand how SEAL would work, SEAL will be able to
smoothly adapt to the appearance of jumboframe ~9k byte paths between
 ITEs and ETEs.

Coping with blocked or missing PTBs
-----------------------------------

I understand that for IPv4, between the ITE and ETE, SEAL does not
use DF=1 packets and so doesn't rely at all on PTBs.   Sometimes, I
think, the DF=0 arrangement would produce faster results than by
using PTBs as is done with IPv6. Other times it might be slower - it
depends on the location and nature of the MTU limits along the tunnel.

It does rely on the limiting router(s) producing a first fragment
which is the same length as the limiting PMTU of the ITE -> ETE path
- which seems reasonable.

For IPv6, I understand that SEAL relies on PTBs being sent by routers
in the ITE -> ETE tunnel path - and these being received by the ITE.

My impression is that there are problems today with some tunnels or
combinations of tunnels not generating PTBs.  Also, some networks
apparently stop the reception of PTBs from the DFZ, so this would
probably prevent SEAL's PMTUD from working.  As far as I know, the
SEAL ITE doesn't have a way with its handling of ordinary IPv6
traffic packets to request that the ETE acknowledge their successful
reception.

For both IPv4 and IPv6, if the SH ignores PTBs, or if there is some
filtering between the ITE and the SH which drops PTBs, then there's
no way the SH is going to adapt its packets to the lengths required
by SEAL to send them without fragmentation, segmentation or whatever.

As far as I know, SEAL does have "segmentation" capabilities - but
these are not intended to be used within the IRON CES architecture.
So SEAL's segmentation is no solution to this kind of failure of
PMTUD.  There's nothing to be done about this - the fault is in the
SH and/or the filtering, so these need to be fixed.

Comparison with Ivip's IPTM
---------------------------

Here is a rough comparison between my understanding of SEAL and my
IPTM approach to PMTUD tunneling when Ivip uses encapsulation:

    http://www.firstpr.com.au/ip/ivip/pmtud-frag/

  IPTM involves the ITRs caching enough of the IPv4 traffic packet
  to generate a valid PTB to the SH.  This is more expensive than
  SEAL.  However, this is only done when sending a B and A pair of
  packets, which is only when the length (after encapsulation) is
  in the Zone of Uncertainty.  Once this Zone is reduced to zero,
  there's no need to use the IPTM protocol, since traffic packets
  are either encapsulated or rejected with a PTB.

  Maybe IPTM doesn't require caching of part of the IPv6 traffic
  packet, since the PTBs should contain enough of it to generate
  a PTB.  However, IPTM can work even if no PTBs are received.
  It is probably best for the ITR to cache the first 540 bytes or
  so of those IPv6 traffic packets which are used in this
  probing of PMTU.

  IPTM uses the same protocol for both IPv6 and IPv4: a dual
  packet arrangement in which, if both are received, the
  traffic packet will be delivered and the ITR will learn of this
  and so be able to raise the lower limit of the range of possible
  PMTU values to this ETR, so reducing the Zone of Uncertainty.
  Specifically, IPTM does not use DF=0 packets in the tunnel, like
  SEAL does.

  IPTM makes use of PTBs from within the tunnel, but if none
  arrive at the ITR, then the ITR can still (usually) determine
  whether the long B packet arrived or not at the ITR.  When
  longer packets do not arrive and shorter ones do, in the
  absence of PTBs, the ITR can try sending shorter packets until
  a size is found which are reliably delivered.  (This is one
  of the many things I need to work on when developing IPTM.)

  If the ITR steps downwards by 20 bytes per attempt, it may
  overshoot the actual PMTU limit somewhat, but it will find
  a value which works, and is usually not too far below the
  real PMTU limit - all without any reliance on PTBs within
  the tunnel.  As far as I know SEAL needs PTBs from the tunnel
  routers to work with IPv6 - but perhaps I don't understand
  this part of SEAL correctly.

  So for both IPv4 and IPv6, IPTM should be able to estimate
  the PMTU even if no PTBs are received.

  IPTM will probably involve some limit on the maximum size of
  IPv4 DF=0 packet which will be allowed.  I suppose the ITR could
  fragment really large ones - if a host sent ~9k byte DF=0
  packets - but I think that sort of packet should not be
  tolerated or encouraged.

  IPTM will adapt to larger PMTUs by trying larger packets
  - assuming a SH sends them - after 10 minutes.

  IPTM differs very significantly from SEAL in that all SEAL's
  packets are capable of reporting a lower PMTU to the ITE.  This
  is not the case with Ivip and IPTM.  IPTM is only used when
  the ITR is unsure whether the traffic packet will run into an
  MTU limitation.  This would typically lead to the reduction and
  soon the elimination of the Zone of Uncertainty as successive
  attempts using IPTM either succeed or fail.

  Whenever an Ivip ITR tunnels a packet to an ETR and its
  length, once encapsulated (just 20 or 40 bytes extra, for IP in
  IP encapsulation) is no greater than the current minimum estimate
  for the PMTU to this ETR, then the ITR encapsulates the packet
  normally.  This means the outer header source address is that of
  the SH.  So the ITR would not get any resulting PTB.  The SH would
  get a PTB - would not recognise it.

  This means that the ITR may not notice a drop in PMTU until after
  10 minutes, when it would retry sending longer packets and so
  discover the new lower value.

  This is something I need to work on.  One approach is to have
  the ITR perform IPTM on these traffic packets few minutes.

  IPTM as currently described uses 32 bit nonces.  Perhaps I
  will change this to something like Fred's arrangement of
  a sequentially increasing value for each ETR, but starting
  from a randomly initialised value.

  IPTM's encapsulation overhead is 20 and 40 bytes for IPv4 and
  IPv6 respectively.  SEAL's is 28 and 48 - which is less than
  LISP's, since LISP has an 8 byte UDP header plus the 8 byte
  LISP header: 36 and 56 bytes for IPv4 and IPv6 respectively.

Comparison with LISP
---------------------

LISP has non-stateful and a stateful approach to handling PMTUD
problems, neither of which are mandatory.

The non-stateful approach might work if the constant was set well
below 1500, such as 1400 or so (depending on the lowest PMTU from any
ITR to any ETR) but it will lock the whole system into this size
without any possibility of using higher values closer to 1550 bytes,
or jumboframe ~9k byte paths in the DFZ as these become available.

I think the the stateful approach, from the OPENLISP project in Belgium:

  http://tools.ietf.org/html/draft-ietf-lisp-06#section-5.4.2

needs more work.

It doesn't mention how DF=0 packets would be handled, how the system
could work if PTBs were not received from the limiting tunnel router,
or how the ITR would explore larger values of PMTU after 10 minutes.

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 8 Feb 2010 12:25:41 -0800

Hi Robin,

> -----Original Message-----
> From: Robin Whittle [mailto:[email-masked]]
> Sent: Monday, February 08, 2010 1:37 AM
> To: RRG
> Cc: Templin, Fred L
> Subject: IRON: SEAL summary
>

No - RANGER is still RANGER and always will be. IRON tells
how the routing system used by RANGER works, with a first
emphasis of the Internet core domain of application. But,
the same IRON principles apply to any RANGER level of
recursion. So, it is "IRON-plus-RANGER", and not
"IRON-obsoletes-RANGER". IRON-RANGER might be a good
way to call it.

>   http://www.ietf.org/mail-archive/web/rrg/current/msg05976.html RW

Actually, the mid-level headers occur between the *inner*
IP header and the SEAL header.

>             8 bytes) or IPv6 Fragment Header (8 bytes).  So these

I am going to work on this. I think what I will do is
eliminate the need for the IPv6 header to include a
fragment header and instead handle IPv6 segmentation
and reassembly using SEAL the same as is currently
documented for IPv4.

>               IPv6: PTB from a router in the tunnel path.

Right. SEAL-FS (and not SEAL-SR) is used for the use
case of IRON in the core.

>             as it remains less than or equal to the limit value.)

I will clarify the cases in which S_MRU may be
set to zero. I think now that they apply only to
SEAL-FS and not to SEAL-SR.

> internal "segmentation" system.

Yes, that is correct for the case of IRON used in
the core. For IRON applied to other levels of the
RANGER recursion, SEAL-SR may be used instead of
SEAL-FS.

> the packet to the ITE.  I am not sure to what extent this would be

I am going to change this so that IPv6 does not use the
IPv6 fragment header but instead uses the SEAL header
(plus SEAL segmentation and reassembly when needed).

> the ETE to confirm that the packet arrived.

I'll fix this.

> Rejecting larger packets directly

This is entering a very sensitive area - when the ITE
gets a DF=0 packet, it needs to know when to use inner
fragmentation and what is the size that can be used for
inner fragmentation? If the ETE will not be doing
reassembly of any kind, the ITE has no choice but
to perform inner fragmentation using a safe initial
fragment size (e.g., 1280). But, it could always send
a "dummy packet" to probe the true inner packet size
so that future DF=0 packets might be able to avoid the
inner fragmentation. The draft doesn't currently say
this, but there is nothing to stop an implementation
from doing it.

> Adapting to changed PMTUs

That is not correct; SEAL can accommodate Jumbograms
using SEAL segmentation and reassembly if necessary.
I think this is made clear in the case of IPv4 as the
outer protocol, but I need to make sure to make it more
clear in the IPv6 case.

> Coping with blocked or missing PTBs

Actually, I will be re-working the IPv6 case so that IPv6
encapsulation also uses the SEAL header which has the
explicit acknowledgement feature.

> by SEAL to send them without fragmentation, segmentation or whatever.

It is true that SEAL does not attempt to fix MTU
problems that occur in the end site in front of
the ITE.

> Comparison with Ivip's IPTM

What happens if a routing change occurs after the zone
of uncertainty has been reduced to zero?

>   this part of SEAL correctly.

SEAL for IPv6 will be changed to use the SEAL header
instead of the IPv6 fragmentation header. Then, it
can use either data packets or dummy packets as
probes the same as for IPv4 - with explicit probe
responses from the ETE even if PTBs are being
dropped in the network. There might be something
in IPTM for SEAL to glean here.

>   LISP header: 36 and 56 bytes for IPv4 and IPv6 respectively.

IMHO, this is a significant limitation.

> or how the ITR would explore larger values of PMTU after 10 minutes.

This too.

Thanks for your comments,

Fred
[email-masked]

---

From: Robin Whittle <[email-masked]>
Date: Tue, 09 Feb 2010 13:40:34 +1100

Short version:   Fred's CES proposal is now called IRON-RANGER.

                 But in protecting DF=0 fragmentable IPv4 packets
                 and suggesting that we live with, and laboriously
                 adapt to, the growing practices of PTB filtering and
                 tunnels which don't support RFC 1191 and RFC 1981
                 Path MTU Discovery, is IRON-RANGER upholding
                 righteousness or .....

Hi Fred,

I plan to write a "IRON: SEAL summary V2" based on what I learnt from
your two recent on-list messages and one off-list message.  Here is
my response to your second on-list message.

> "IRON-obsoletes-RANGER". IRON-RANGER might be a good

OK . . .

I thought "LISP" and "NERD" were weedy names and so I was quite happy
with "Ivip" - energetically reminiscent of a 1950s abrasive sink
cleaner.

APT was brief and apt.

RANGER and (Navy, I assume) SEAL were always lurking in the distance.

Now it seems we have a full-tilt Acronym Arms Race with the formidable:

    >>>> IRON-RANGER <<<<

on the scene.  Other proposals would be well advised to mind their
manners . . .

The moral questions are not so clear-cut.  Is IRON-RANGER really
defending against the Forces of Evil?

Are DF=0 packets really the endangered species and unfortunate
refugees of past eras, which we should dutifully slice up, cart
around in multiple fragments, and painstakingly reassemble as if
nothing had happened?

Should stack and application programmers be required, by the value
judgements inherent in IRON-RANGER's PMTUD Modernization Program, to
update everything to use complex and messy RFC 4821 PMTUD?

Is IRON-RANGER's SEAL Highway Planning Department doing the right
thing by routing our communication pathways further and further
around a growing swamp caused by lousy tunnel implementations and
ill-advised filtering of PTBs?

Or is plucky little Ivip pursuing a more efficient and enlightened
future by recognising the incompatibility between this swamp and a
healthy future, by proposing it is best and easiest to Just Say No to
such swampificacious practices and by casting DF=0 packets to the
outer, along the lines of (msg05816)?

   DF=0 is an indulgence still granted to lazy bourgeois hosts
   in their dotage, enabling them to continue their regressive
   habits of unfairly burdening the proletariat of Worker
   routers with the responsibility for slicing up and
   individually carrying and reassembling their overweight,
   carelessly emitted, packets.  Come the Revolution . . . .

or:

   Citizens and businesses shouldn't have to pay extra for their
   Internet services just because a few applications and big
   corporations (Google) are burdening the Internet's routing
   system with over-long packets which require special handling.

> IP header and the SEAL header.

Something may need to be rewritten, since Figures 1 and 2 show
"mid-layer headers" after SEAL header.

> and reassembly using SEAL the same as is currently

OK - but as far as I know, in an IRON-RANGER scenario, the ITEs are
not intended to be continually sending a traffic packet by two or
more tunnel packets, whether by IPv4 fragmentation, IPv6
fragmentation or inbuilt SEAL segmentation - with the probable
exception of DF=0 IPv4 packets.

> case of IRON in the core.

I think that the IRON ID should attempt a summary of which parts of
SEAL are used for this CES system.  Likewise, it should contain a
reasonably self-contained description of what parts of RANGER and VET
are used.

> SEAL-FS and not to SEAL-SR.

OK.

> RANGER recursion, SEAL-SR may be used instead of

OK.

> (plus SEAL segmentation and reassembly when needed).

OK.

>> the ETE to confirm that the packet arrived.

OK - the SEAL header for IPv6 will have a flag to tell the ETE to
confirm reception.

> this, but there is nothing to stop an implementation

We may have to resort to this sort of dual packet tunneling for some
DF=0 packets, but I think the sooner DF=0 packets are deprecated, the
better.

> outer protocol, but I need to make sure to make it more

I think you could make SEAL to almost anything, but why would you
want any router to accept 64k and longer packets, up to 4 gigabytes
long, and fragment or segment then to be sent and later reassembled?

  http://tools.ietf.org/html/rfc2675

> problems that occur in the end site in front of

Nor any other CES.  Only RFC 4821 in the TCP stack or the application
packetization layers can cope with this.  I think it should be fixed,
rather than coped with.

> of uncertainty has been reduced to zero?

If the routing change lowers the PMTU, then there would be lost
packets.

As I wrote further down:

>>   This means that the ITR may not notice a drop in PMTU until
>>   after 10 minutes, when it would retry sending longer packets and
>>   so discover the new lower value.
>>
>>   the ITR perform IPTM on these traffic packets every few minutes.

Maybe the ITR needs to do this at more frequent intervals.  If such
changes were frequently encountered, it could also deliberately tell
the SH to send somewhat shorter packets than could be handled at
present, so that if a routing change dropped the PMTU by 30 bytes or
so, the resulting packets would still get through the new lower limit.

If the ITR function was in the sending host, it could easily detect
the PTBs which should arise from the new MTU limit and use them to
send a PTB to the SH's IP stack to lower the packet length accordingly.

> in IPTM for SEAL to glean here.

IPTM already uses a probing strategy which uses PTBs if they arrive
and otherwise relies on the ETR explicitly acknowledging the receipt
or non-receipt of the B packet, which is the same length as would
result from normal IP-in-IP encapsulation of the original traffic packet.

>> or how the ITR would explore larger values of PMTU after 10 minutes.

I agree.

> Thanks for your comments,

Thanks for discussing your proposal and PMTUD in detail.

  - Robin

---

From: Robin Whittle <[email-masked]>
Date: Tue, 09 Feb 2010 14:33:52 +1100

Based on discussions with Fred (my recent messages), here is my
revised attempt at describing SEAL, at least as far as it is used in
the IRON-RANGER Core Edge Separation scalable routing proposal.

Please see the initial attempt:

  http://www.ietf.org/mail-archive/web/rrg/current/msg05977.html

for references to the source documents and older messages and for the
comparisons with Ivip's IPTM and LISP, which Fred and I discussed in
(msg05979) and (msg05981).

 - Robin

ITEs and ETEs
-------------

An Ingress Tunnel Endpoint (ITE) is part of the VET interface in an
IRON router.  It uses SEAL to encapsulate and tunnel packets to a
distant IRON router, with SEAL's approach to PMTUD which enables the
IRON router which is the ITE to send Packet To Big (PTB) messages to
the Sending Host (SH).   The ETE (Egress Tunnel Endpoint) is
implemented in the VET interface section of the distant IRON router.

When SEAL and RANGER are updated as Fred plans, the ETE router will
be able to send a SEAL message to the ITE router telling it to
redirect packets to a given prefix, to another IRON router.  This
will involve a caching time and the SEAL_ID from the SEAL header of
the tunnel packet which gave rise to the redirection.  This is an
important part of IRON - which I will attempt to describe in a
separate message.

SEAL for IPv4 and IPv6
----------------------

I assume that for IRON, there will be no need for "mid-level headers"
such as UDP between the outer header and the SEAL header - but see
msg05927 and msg05976 and search for "ECMP".  (msg05980 and msg05981
indicate there is some confusion about UDP headers and where they
would go.)

    Please see previous messages for Fred's approach to doing this
    and my critique and timer-based suggestion.

    MHLEN   Constant mid-level header length.  In this attempt to
            describe SEAL, I will assume there is no need for these
            "mid-level" (?) headers - between the outer IP header and
            the SEAL header which precedes the encapsulated packet.
            So I will assume this = 0.

    HLEN    Constant outer header length: 20 for IPv4 and 40 for
            IPv6 plus the length of the SEAL header - 8 bytes I guess
            but Fred hasn't yet defined it.  It needs 4 bytes just
            for the SEAL_ID.

               IPv4 28           IPv6 48

SEAL for IPv4
-------------

   (MIN(S_MRU, S_MSS) - 28)

   (S_MRU - 28)

Encapsulation involves:

The IPv4 SEAL header has a 16 bit field "ID-Extension"  which is set
to the least significant 16 bits of SEAL-ID.  Section 4.2 discusses
this.  I haven't figured out exactly how the other bits would be set.

There is a separate SEAL PTB message (4.4.5.1.3) which is not
relevant to IRON - since it concerns SEAL's internal "segmentation"
system.

SEAL for IPv6
-------------

   (S_MRU - 48)

When encapsulating IPv6 packets, there will be a new SEAL header
which Fred is yet to define.

I guess it will be 8 bytes.  It will contain, at least, a 4 byte
SEAL_ID field and a flag to ask the ETE to tell the ITE the packet
was received correctly.

If the ITE sets the "Acknowledgement Requested" bit then the ETE will
tell the ITE it has been received.  The ITE can therefore try
different length packets to determine the PMTU to this ETE, even if
it receives no PTBs from whichever router is causing the limitation.

(See discussion in msg05979 and msg05981.)

Jumboframes
-----------

"Jumbograms" refer to IPv6 packets with special formats so they can
be longer than 2^16 bytes, and can be as long as 4 gigabytes.
Neither SEAL, Ivip's IPTM, nor any CES proposal attempts to deal with
these.

See discussion in msg05979 and msg05981 - Fred writes that SEAL can
accommodate jumbograms - but I can't see how.

For IPv4, between the ITE and ETE, SEAL does not use DF=1 packets and
so doesn't rely at all on PTBs.   Sometimes, I think, this DF=0
arrangement would produce faster results than by using PTBs as is
done with IPv6. Other times it might be slower - it depends on the
location and nature of the MTU limits along the tunnel.

For IPv6, SEAL relies on PTBs being sent by routers in the ITE -> ETE
tunnel path - and these being received by the ITE.

My impression is that there are problems today with some tunnels or
combinations of tunnels not generating PTBs.  Also, some networks
apparently stop the reception of PTBs from the DFZ, so this would
probably prevent SEAL's PMTUD from working.  When Fred revises SEAL,
the ITE will be able to tunnel IPv6 traffic packets and request that
the ETE acknowledge their successful reception.

No CES architecture tries to cope with these problems - they must be
fixed directly.

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 9 Feb 2010 14:20:39 -0800

Hi Robin,

> Sent: Monday, February 08, 2010 6:41 PM
> To: Templin, Fred L
> Cc: RRG
> Subject: Re: IRON: SEAL summary
>

I think the figures are OK. The top of the figure is
the outermost header, and descending downwards through
the figure shows successively more inner layers of
encapsulation. Is it somehow confusing?

> exception of DF=0 IPv4 packets.

In the core, we will not require core routers to
reassemble and those routers will use SEAL-FS.
Toward the edges, there may be places that would
benefit from using segmentation and reassembly,
e.g., to hide the encapsulation artifact for
tunnels. Those routers would use SEAL-SR.

OK.

>   http://tools.ietf.org/html/rfc2675

Hi speed data centers that have 9KB MTU links and
very low packet loss due to congestion might want to
present a large MTU to TCP (e.g., 64B) then carry the
segments as multiple 9KB packets using SEAL. This also
brings up the question of effectiveness of the link
level CRC in detecting errored data as a function of
the packet size. (Earlier works seemed to show that
CRC-32 performance deteriorates for packet sizes
larger than ~9KB.)

So, high speed data centers might want to use packet
sizes that are larger than the underlying links can
support natively.

> packetization layers can cope with this.  I think it should be fixed,

This can easily morph into a discussion of the
end-to-end principles and how they would view
MTU determination. I still think that the end
systems would be better served to take MTU
assurance into their own hands rather than
rely on an untrustworthy network.

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 9 Feb 2010 14:49:48 -0800

Robin,

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Robin Whittle
> Sent: Monday, February 08, 2010 7:34 PM
> To: RRG
> Subject: Re: [rrg] IRON: SEAL summary V2
>

This is discussed in SEAL, Section 4.3.5.

>   IPv6:    A PTB arrives at the ITE, and the ITE sends a PTB to the

The draft talks about jumbograms and cites RFC2675. But,
it could probably benefit from a sentence or two telling
how the Jumbo Payload Option is processed.

> smoothly adapt to the appearance of jumboframe ~9k byte paths between

Yes.

> Coping with blocked or missing PTBs

This discussion is not specific to SEAL nor even to
tunneling in general. If the SH is not accepting PTBs
from the gateway that connects the source's site to
the Internet, then there is a problem regardless of
the way in which the gateway connects.

Thanks - Fred

> _______________________________________________
> rrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/rrg

---

From: Robin Whittle <[email-masked]>
Date: Wed, 10 Feb 2010 13:12:26 +1100

Short version:    Low-key response to Fred's comments.

                  I think my SEAL summary V2 (msg05982) with Fred's
                  comments (msg05989) is OK.

Hi Fred,

> encapsulation. Is it somehow confusing?

OK - for some reason I was thinking of UDP as a "mid-layer" header.

The 3rd dot point below Figure 2 mentions UDP as an outer layer
header - but it is may be difficult for readers to make the link from
the diagram to that text.  Also, I suggest you explain the purpose of
using UDP or any other headers after the IPv4/6 header, and before
the SEAL header.

> tunnels. Those routers would use SEAL-SR.

OK:

   SEAL-SR is a functional superset of SEAL-FS, and requires that the
   tunnel endpoints support segmentation and reassembly of packets
   that are too large to traverse the tunnel without fragmentation.

OK.

> segments as multiple 9KB packets using SEAL. 

OK - I understand this is just within these data centers and not to
other networks via the DFZ.

TCP would be fast and efficient with ~64kB packets, but the SEAL-SR
system in their network (in the sending and receiving hosts, perhaps)
would send them as ~9k packets.

There is already something like this going on with Ethernet drivers
or the Ethernet chip itself:

 http://www.firstpr.com.au/ip/ivip/ipv4-bits/actual-packets.html

> This also

Maybe so, but I am focusing on scalable routing and getting packets
across the DFZ.

> rely on an untrustworthy network.

OK - we have different views on this.

 - Robin

---

From: Robin Whittle <[email-masked]>
Date: Wed, 10 Feb 2010 14:46:17 +1100

Hi Fred,

I think my SEAL summary V2 (msg05982) with your comments below is OK
for now.

 - Robin