Subject: [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper

From: Robin Whittle <[email-masked]>
Date: Fri, 12 Feb 2010 23:13:25 +1100

Short version:   I believe the GLI-Split designers were mistaken to
                 class their proposal as a Core-Edge Separation (CES)
                 architecture.  I am sure it is a CEE (Core-Edge
                 Elimination) architecture.

                 Comments on Lixia's and colleagues' 2008 paper
                 on Core-Edge Separation vs. Core-Edge Elimination.

                 I think they were mistaken to class GSE as a
                 Core-Edge Separation architecture.

                 I think this paper refers generally to the
                 core-edge "separation" being of the routing
                 systems of the edge networks from the core - but
                 the real distinguishing thing about CES is the
                 separation of one subset of global unicast address
                 space into a "edge" subset for scalably supporting
                 end-user networks with multihoming etc. from the
                 remaining "core" addresses.

                 The authors were in general the designers of APT,
                 which separated not just the address space into
                 two subsets, but had a long-term goal of actually
                 separating all the edge networks from the core, so
                 no edge host could physically send a packet to
                 any core address.  AFAIK, no other CES architecture
                 is intended to separate the routing systems like
                 this.

                 I think this double sense of the term "separate"
                 plus the mistaken classification of GSE as CES
                 contributed to the GLI-Split authors considering
                 their proposal to be a CES architecture too.  Yet
                 it is clearly CEE, since it implements the
                 Locator/Identification Separation naming model.

                 These terminological and classification problems
                 are compounded by LISP being inappropriately named -
                 since it is a CES architecture which does not
                 implement Locator/Identifier Separation.

                 Six/One Router is a translation-based CES
                 architecture.

                 GSE and GLI-Split are translation-based CEE
                 architectures.  They are unusual among CEE
                 architectures in that they only require the
                 new router functions and changes to the stack -
                 they do not require applications to be modified.
                 (However I just learnt that GLI-Split could
                 support new application functionality via a new
                 API.)

                 Comparison charts of:

                 CES: LISP-ALT/NERD, APT, Ivip, TRRP, TIDR,
                      IRON-RANGER and Six/One Router.

                 CEE: GSE, GLI-Split, ILNP, Name-Based Sockets.

Here is some discussion further to my understanding of the
distinctions between Core-Edge Elimination (CEE) architectures and
Core-Edge Separation (CES) architectures:

  CES & CEE are completely different (graphs)
  http://www.ietf.org/mail-archive/web/rrg/current/msg05865.html

This paper:

  Towards a Future Internet Architecture: Arguments for Separating
  Edges from Transit Core
  Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang
  http://conferences.sigcomm.org/hotnets/2008/papers/18.pdf

is really important to the RRG discussions, because it formalises the
architectural terminology for two important classes of scalable
routing solution.  To see my attempt to trace the origins of these
concepts and terms, please see:

  Re: [rrg] Terminology
  http://www.ietf.org/mail-archive/web/rrg/current/msg05966.html

I support this paper, but have some comments to make:

  1 - The definition of exactly what core and edge things are
      being separated or eliminated could be improved - to
      refer to separation into two subsets of addresses.

  2 - The suggestion that GSE is a CES architecture is mistaken.

  3 - The paper doesn't mention what I consider to be the biggest
      objection to CEE: that it implements the Locator/Identifier
      Separation naming model which will generally slow down the
      establishment of communication sessions and place extra
      burdens on all hosts - especially those on slow, unreliable
      wireless links.

The paper refers in general to:

    "separating edge networks from the transit core"

but really both CEE and CES maintain the already existent separation
between edge networks from the transit core.  (See discussion of
section 4.3 below for my understanding of why the authors may have
emphasised the separation of networks rather than the separation of a
new subset of scalable "edge" addresses from the remaining "core"
addresses.  The intended APT to ultimately separate the two systems
so much that an edge host would be physically unable to send a packet
to a core router.)

What is being separated in a CES architecture is the "edge" subset of
addresses from the remainder of the global unicast addresses, which
is then known as "core" addresses.  This is a different separate
concept from "separating edge networks from the core", since the
networks are already separated.  Its just that at present, these PI
prefix-using End User Networks (EUNs) use address space which all
core (DFZ) routers need to be concerned with.

>From the abstract, it can be seen that the CES definition is not the
opposite of the CEE definition:

CES:    The first direction, which we dub separation, calls for
        separating edge networks from the transit core, and
        engineering a control and management layer in between.

CEE:    The other direction, which we dub elimination, calls for edge
        networks to adopt multiple provider-assigned addresses to
        enable provider-based address aggregation.

(See below for discussion of the paper's more formal description of CEE.)

In fact, the CEE definition could reasonably include CES
architectures, since CES architectures generally use multiple ETRs,
each on a PA ("provider-assigned") address, for the purposes of
enabling the DFZ to only handle prefixes which are subject to
"provider-based address aggregation".

The most important architectural distinction between CES and CEE is:

  CES involves end-user networks' hosts using global unicast
  addresses from a specially handled "edge" subset of the global
  unicast address space, with the remainder being known as the
  "core".  IP addresses in both subsets will continue to be used, as
  are all today, both for Identifying hosts and for the Routing
  system to decide how to forward packets (i.e. "Locator).  But note
  that CES has a new, fancy, way of doing this for this subset of
  "edge" addresses - different from today's plain DFZ approach -
  which is highly scalable and allows these "edge" addresses to be
  used by EUNs for portability, multihoming inbound TE and perhaps
  for mobility.

  CEE involves end-user networks' hosts using something other
  than a global unicast IP address to Identify themselves.  Thus
  CEE implements the Locator/Identifier Separation naming model.
  Applications use this new Identifier, which is in a different
  namespace to IP addresses and any Locators used in the system,
  to identify each host.  Hosts can be reached via one or more
  potentially unstable Locators, and the applications are unaffected
  due to their reliance purely on Identifiers to specify which hosts
  are communicating.

CES preserves the existing IP naming model:

       Role             Level

       Text name  <---- FQDN
       Identifier <---] IP address
       Locator    <---]

while CEE creates one of several models, in which the Locator and
Identifier roles are played by different levels - different objects
in different namespaces.

CEE architectures always implement Locator/Identifier Separation.

No CES architecture implements Locator/Identifier Separation.  What
they separate is two subsets of the global unicast address space:
"edge" from the remaining "core".  Both types of address are used
both as Locators and as Identifiers.

Both subsets are used as Identifiers by all hosts.  Hosts make no
distinction between addresses in the two subsets.

Most routers make no distinction between how they handle packets with
destination addresses in these two subsets.  The exception is ITRs,
which make a big distinction.  ITRs implement the Locator
functionality of "edge" addresses in a totally different way: by
looking up mapping and tunneling to an ETR, rather than by forwarding
the packet to a neighbour.

More on naming models:

  Today's "IP addr. = ID = Loc" naming model should be retained
  http://www.ietf.org/mail-archive/web/rrg/current/msg05864.html

CES architectures never require changes to host stacks or applications.

Many CEE architectures require changes to both stacks and apps, but
at least two - GLI-Split and I think GSE - only require changes to
the stacks.

Here are some cases:

CES:  LISP, APT, Ivip, TRRP, TIDR and IRON-RANGER.

      There's no debate about this - they all create a special
      "edge" subset of the global unicast address space for EUNs
      to use for portability, multihoming, TE and perhaps mobility
      - and add things to the routing system to scalably get packets
      addressed to "edge" addresses to the destination networks
      without excessive burdens on the DFZ control plane.

      Except for Ivip's long-term "Modified Header Forwarding" modes,
      all these use encapsulation in order to tunnel packets from
      an ITR or ITR-like device to an ETR or ETR-like device.  This
      raises some thorny PMTUD problems.

      These CES architectures are all intended to work for both IPv4
      and IPv6.  The descend directly or indirectly from the original
      map-and-encaps architecture.  No host stack or application
      changes are required.  Things are added to the routing system -
      but in general routers stay the same.

CES:  Six/One Router - by Christian Vogt, though AFAIK it is no
      longer under development.

http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf

      This is clearly a CES architecture, because the end-user
      networks operate from a single prefix drawn from an "edge"
      subset of the global unicast address range.  (Sect. 2.1.)

         Figure 1 illustrates this setup and addressing for an
         edge network that is multi-homed with two providers.  This
         edge network has one border link to each provider, so two
         Six/One routers are in use. The edge addresses are from the
                                         ----
         prefix ABC::/64. The Six/One router on the border link to
         provider 1 translates between those and transit addresses
         from the prefix 1000::/64; the Six/One router on the border
         link to provider 2 translates between the same edge
         addresses and transit addresses from the prefix 2000::/64.

      A host HA in one Six/One Router network, addresses packets to
      a host HB in another Six/One Router network another using the
      same IP address in the destination field as HB uses as its
      own "edge" address.

         Edge addresses are for use within this and other upgraded
         edge networks.  They are globally unique, but do not appear
                                  ---------------
         in the global routing table because they cannot be
         efficiently aggregated.

      All hosts in Six/One routers use a form of DNS which returns
      these "edge" addresses for any host in any Six/One Router
      network.  So this is a genuine scalable "edge" subset of the
      global unicast address space.

      However, because Six/One Router has no equivalent to DITRs
      (Ivip) or PTRs (LISP), hosts in ordinary IPv6 networks can't
      send packets to the Six/One Router hosts using these scalable
      "edge" addresses.   So, from a non-Six/One Router network, a
      DNS lookup on the same FQDN of host HB will not return its
      "edge" address, but one or more another addresses, based on
      one or more addresses within the PA prefixes HB's network
      obtains from its one or more ISPs.  The address-rewriting
      router at the border of HB's network will rewrite the
      destination address of a  packet coming from some ordinary IPv6
      host HX, to the "edge" address of HB.

      This is explained in section 2.4.  No other CES architecture
      has this "split-horizon" (I think this is the right term) DNS
      arrangement.  I guess it was impossible to implement an
      equivalent of DITRs and PTRs.

      The hosts in Six/One Router behave as usual.  There are no
      stack or application changes.  The naming model used in the
      packets they send and receive is the same as with today's IPv4
      and IPv6, as illustrated above.

      Rather than tunnel packets across the DFZ with encapsulation or
      Modified Header Forwarding, Six/One Router uses address
      rewriting (AKA address translation) at EUN BRs or perhaps
      in a router in their ISP(s).

      Unfortunately, Six/One Router can't be used for IPv4 due
      (I guess) to the more chaotic EUN prefix lengths and (I am
      sure) due to the need for a 2 ISP multihomed EUN to consume 3
      times the amount of global unicast address space it really
      needs.  (One for its "edge" prefix and then the same amount
      of PA space from each of its ISPs.)  Also, when Christian and
      I last discussed it on the RRG, he wrote that he needed a
      flag bit from the IPv6 header, which would require a modified
      IPv6 header format and so require updates to all DFZ and many
      other routers.

CEE: GSE - Mike O'Dell, 1997

       http://ietfreport.isoc.org/idref/draft-ietf-ipngwg-gseaddr/

     This is an extension of his earlier 8+8 architecture - both are
     for IPv6 only.

     GSE is clearly a CEE architecture because it has separate
     objects for Identifying hosts and for Locating them.  There is
     no separate "edge" subset of the global unicast address space by
     which hosts in EUNs identify themselves.  The Identifier is the
     64 bit ESD.

     There needs to be new things in the network: an address
     rewriting function in the BRs of EUNs which adopt GSE.

     Host stacks needs to be somewhat modified, but I think this
     is mainly so header checksums are computed only on the ESD
     and not on other parts of the IPv6 address.

     As far as I know, applications do not need to be modified.

     Multihomed EUNs get a prefix from each of their two or more
     ISPs, and the identity of hosts is unrelated to this, or to
     any IP address.  These prefixes are PA, and the ISP may have
     a single large (short) prefix which it splits into many such
     prefixes for many such EUNs.  Host Identity - and so all
     communications establishment and session continuity - is
     determined by the value each host uses for the 64 bit ESD,
     which exists in a separate namespace from IPv6 addresses.

CEE: GLI-Split

http://www3.informatik.uni-wuerzburg.de/~menth/Publications/papers/Menth-GLI-Split.pdf

     I believe GLI-Split is a CEE architecture, because it introduces
     a separate "ID" object (Figure 2), with a separate namespace
     from IPv6 addresses or anything else, to Identify each host
     - which is different from the objects used to Locate them: the
     GL or LL Locators.

     The fact that both objects are combined with other things to
     form an IPv6 address does not affect the fact that these objects
     for Identifier and Locator are independent, and in separate
     namespaces, just as they are in ILNP, GSE and the other CEE
     architectures.

     GLI-Split implements Locator/Identifier Separation, as do all
     other CEE architectures.  From the Abstract:

         It implements the locator/identifier split and makes routing
         in the core of the Internet more scalable.

    More on GLI-Split at the end of this message.  The authors
    explicitly state that it is a CES architecture, but I am sure
    it is a CEE architecture.  I think they are following a mistake
    made which I believe was made in the the 2008 sep./elim. paper.

CEE: ILNP - Ran Atkinson.

     This is clearly a CEE proposal, because it has separate levels
     for Identifier and Locator.  It is a purely host-based
     descendent of GSE.

     There is no need for new routers, but the host stacks and
     applications all need to be upgraded, because the applications
     use Identifiers and Locators separately via the API to the
     stack (or do the applications use only Identifiers?).

     As with GSE and all other CEE architectures, a multihomed
     network uses two or more PA prefixes from its two or more
     ISPs, and the PA nature of those prefixes means each ISP can
     supply many such prefixes from a single prefix the ISP
     advertises in the DFZ.

CEE: Name Based Sockets - Christian Vogt.

     This is a CEE architecture because the Identity of each host
     is a FQDN and its one or more Locators are IPv6 addresses,
     again drawn from PA prefixes of the EUNs one or more ISPs.

     Name Based Sockets is unusual for a CEE architecture in that
     it appears to have a two-level naming model.  However, I am
     not sure how it could have this and still be able to support
     DNS lookups leading to multiple separate hosts.  I hope
     Christian will clarify this.

CEE? RANGI and Name Overlay Service

     I am yet to read these properly, but my impression is that they
     are both CEE architectures.

???: hIPv4 - Patrick Frejborg

     I think this was an attempt to create a new, larger, address
     space while enabling scalable routing and transport of the
     new format packets across the DFZ.  The ELOC part of the
     new address is not globally unique. (If it was, it would be
     an Identifier.)  The combination of the ALOC and ELOC provides
     the extended equivalent of an IP address today - meaning both
     a host Identifier and including a routing locator within the
     one object.

     I previously thought it was a CEE but I think it is neither
     CEE or CES.  Unfortunately, hIPv4 is not practical since DFZ
     routers are not ready to handle packets with the required
     option header.

???: Aggregation with Increasing Scopes.

     I don't yet understand this.

Here are some charts:

CES            IPv4   IPv6   Tunneling  Local            Mapping
architecture                            full-            speed
                                        database
                                        mapping
                                        servers?

LISP-NERD        v4     v6   Encaps.    ITRs are full    Slow
                                        database

LISP-ALT         v4     v6   Encaps.    No               Global
                                                         lookup

ALT              v4     v6   Encaps.    Yes              Slow

Ivip             v4     v6   Encaps.    Yes              Fast
                             or MHF

TRRP             v4     v6   Encaps.    No               Global DNS
                                                         lookup

TIDR             v4     v6   Encaps.    ITR-like         Slow - via
                                        DFZ routers      DFZ control
                                        have full DB.    plane.

IRON-RANGER      v4     v6   Encaps.    (Neither question applies
                                         directly to IRON-RANGER.)

Six/One Router          v6   Address    (Six/One Router's mapping

                             rewriting   system was never specified.)

CEE            IPv4   IPv6   App       Stack      Router
                             changes?  changes?   changes?

GSE                     v6   -         Stack      Router rewrites

GLI-Split               v6   -*        Stack      Router rewrites

ILNP                    v6   App       Stack      -

Name Based
Sockets                 v6   App       Stack      -

* GLI-Split does not require host changes, but Michael Menth just
  wrote to me offlist that new applications could be used, for
  instance for multipath protocols, if the stack had a suitable
  new API:

     "Applications usually communicate with identifier addresses,
      but also a more advanced API must be offered in addition to
      allow active multipath routing through applications or at
      least by transport layer protocols.

  He indicated that this was not in the current documentation but
  would be added in the future.

Back to the 2008 paper: on page 2, 2nd last paragraph:

     There are also other types of separation
     solutions besides Map & Encap. For example,
     Six-One Router [23] and GSE [19] use address
     rewriting, which rewrites the packet header
     to include information about the
     destination’s attachment point to the transit
     core.

For the reasons listed above, I believe GSE is a CEE architecture.

The fact that the CES architecture Six/One Router has routers
rewriting addresses does not mean that GSE is a CES architecture
simply because it too has routers rewriting addresses.

In Section 3, there is a more formal description of CEE:

     In order to achieve routing scalability, the
     elimination approach enforces provider-based
     address aggregation by eliminating all PI
     prefixes and de-aggregated PA prefixes.

I wouldn't state it so strongly.  I think it is more accurate to say
that when all hosts in all networks adopt CEE, the CEE mechanisms can
be relied upon entirely for portability, multihoming and inbound TE.
 Then, and only then, this means that no network needs PI space to
achieve these things.  This doesn't mean that PI space (the current,
unscalable, form of "edge" space) would automatically be eliminated -
just that it could be eliminated without any network missing out on
portability, multihoming or inbound TE for all its hosts and
communications.

     Each multihomed edge network will receive
     from each of its providers an address block out
     of a larger, aggregated block announced by the
     provider.

I would rephrase it slightly.  Each EUN which adopts CEE would
generally use space like this.  Each such EUN *could* use PI space.
Its just that PI space is no better than PA space for use with a CEE
architecture.  (Likewise a EUN using a prefix it gets as PA space
from one ISP via another ISP as a "more specific PA prefix" - which
also has the effect of adding another prefix to the DFZ control plane.)

The degree to which they can multihome etc. depends on how many of
the hosts they are communicating with are also using the CEE
architecture, as noted in 4.1.

     The multihomed site does not inject PI prefixes
     or more specific PA prefixes into the routing
     system. Instead, each host in a multihomed site
     is given multiple PA addresses. For example, as
     shown in Figure 2, the host obtains two addresses,
     one from each of its network’s ISPs.

Typically, this is what would happen.

The "elimination" in this description refers to the elimination of a
class of address space, the current unscalable "edge" subset which is
primarily PI prefixes and also, to some extent (I don't know how
much) these "more specific PA prefixes".

The paper's initial use of "separation" was not in terms of address
space, but in terms of separating edge *networks* from the core.
Edge networks are always separate from the core.

I think that a more correct use of "separation" would be to refer to
the deliberate creation of a subset of "edge" space which can provide
what EUNs need, but in a scalable manner.  Then, there will be more
EUNs getting the portability, multihoming and inbound TE they need,
with less and less of them using PI prefixes or "more specific PA
prefixes" to get it - which are also "edge" uses of address space,
but the kind we are trying to discourage because they are unscalable.

In section 4.3, I think I see why the authors refer more to
separating edge networks from the core.  I think what they are
referring to is the end-point of APT adoption:

  1 - All EUNs use APT, and so use the scalable "edge" subset of
      the global unicast address space.

  2 - Therefore, no EUN needs PI space or "more specific PA
      prefixes".  So these are all converted either to the new
      scalable "edge" subset of space or to the remainder -
      the "core" subset.

  3 - Therefore, ISPs advertise prefixes into the DFZ which are
      all in the "core" subset.

  4 - Because all ISPs and EUNs use APT, there is no need for
      APT's equivalent of LISP's Proxy Tunnel Routers or Ivip's
      DITRs.  (APT's arrangement was never well documented, but
      it too could provide full support for packets sent from
      non-upgraded networks.)  Therefore, there is no need to
      advertise any "edge" space in the DFZ.

  5 - Now, if it was desired - as the APT designers desired -
      it would be possible, in theory, to prevent any host
      in an EUN from being able to send a packet to any host
      or router on a "core" address.  Hosts in EUNs only need
      to communicate with other hosts in EUNs.  (Traceroute
      and ping wouldn't concern DFZ routers, since all packets
      between EUN hosts would be tunneled across the DFZ.)

Most of the APT designers wrote this 2008 paper.

AFAIK, no other CES architecture had this complete, physical,
separation of EUN hosts from being able to send or receive packets to
or from "core" addresses as a goal.  I specify it as a non-goal for Ivip.

>From section 4.3:

   Separating edges from the transit core provides
   additional features that are sorely missing in
   today’s Internet.  With separation, an end host
   can send packets through the transit core, but
   can no longer address a packet to any specific
   device inside the transit core. Although the
   separation does not eliminate any specific
   security threat, it raises the bar against
   malicious attacks targeted at the global routing
   infrastructure. In addition, the mapping layer
   between edge and core networks can serve as a
   mounting point for badly-needed control and
   protection mechanisms, and can also act as a
   cushion layer between the edge and core,
   allowing each side to deploy innovations without
   any involvement of the other side. We now
   elaborate on each of these benefits.

They go on to discuss "Rolling out new protocols." and "DDoS
mitigation." which they argue would be possible or easier only with
complete separation of the "edge" networks from the "core" networks.

I don't agree with these benefits, and I doubt it would be feasible
to achieve such complete separation.  I won't mention why, because
that is not the point of this discussion.  I am trying to explain why
the authors of this important paper seemed to equated, in my terms:

   Separation of a subset of the global unicast space into a
   scalable "edge" subset.

with:

   Separation of the "edge" networks so their hosts couldn't send
   packets to, or receive them from, devices in the "core".

They also mention a third argument for this "separation": "Ingress
traffic engineering.".  I have two comments about this.

Firstly, inbound TE works fine due to any amount of adoption of the
CES architecture (that is separation of the scalable "edge" addresses
into a subset) and does not depend on the complete separation of
"edge" from "core" networks, which is what this section 4.3 concerns.
 As long as the CES architecture supports all packets sent from
non-upgraded networks, inbound TE will be for all the adopting
networks incoming packets, no matter how many other networks have
so-far adopted the CES architecture.

Secondly, what they write in the second paragraph about Site1 having
preferences for which of Site2's two or more ETRs to use doesn't
strike me as part of most CES architectures.  I think the aim of most
CES architectures is to to have the recipient site control which of
its ISPs the packets arrive via.  This is inbound TE.  Maybe APT had
some concept of the sending EUN controlling this, but I guess most
sending EUN wouldn't care which ISP the packets went to.

I think the discussion in section 5 about multipath being used with
CES is not really about multipath as some other people conceive of
it.  My understanding of true multipath, such as MPTCP, is the
sending host (or perhaps a router)and perhaps the receiving host (or
its router?) being able to choose which of multiple ISPs in the
sending host's network packets may go out from, at the same time as
choosing which of multiple ISPs in the recipient host's network the
packets will travel through.   With CES, I don't see how a sending
host could affect either.

The routers in the sending host's network can choose which ISP link
to send out the packets on (assuming the ISPs accept packets with
"edge" addresses, which they will when CES is widely used).  If those
routers are ITRs, then in some or many CES architectures (but not
Ivip) then perhaps the ITR might be locally programmed to prefer one
of multiple destination ETRs over others - so in principle the ITRs
in the sending network could affect the incoming load balance of the
destination network.  However, as far as I know, the intention with
all CES architectures is to allow the sending network to choose the
outgoing ISP and the recipient network to choose (inbound TE) its
incoming ISP(s).  In the case of Ivip, the recipient network only
provides a single ETR address, so there is no option for the sending
network to have its ITRs tunnel traffic to some other ETR of the
destination network.

I think this paper doesn't mention what I regard as the greatest
objection to CEE architectures - their adoption of the
Locator/Identifier Separation naming model, as I discuss in (msg05864).

I the GLI-Split authors were mistaken to state that GLI-Split is a
CES architecture.

The summary:

http://tools.ietf.org/html/draft-irtf-rrg-recommendation-04#section-10.1

includes text indicating that perhaps the authors consider GLI-Slit
to be a CES architecture:

    GLI-Split implements a separation between global routing
    (in the global Internet outside edge networks) and local
    routing (inside edge networks) and using global and local
    locators (GLs, LLs).

Yes, but all EUNs have separate routing from the DFZ - so this is not
 a function of GLI-Split.

     o  Hierarchical aggregation of routing information in the
        global Internet through separation of edge and core
        routing.

These phrases seem to imply that GLI-Split is a Core-Edge Separation
architecture.

Back to the GLI-Split paper, there is explicit recognition of how it
involves the separation of Locator and Identifier functions.  This is
a feature of all CEE architectures, and is not (despite LISP's name)
something which CES architectures do:

    Separating current IP addresses into two independent
    pieces of reachability and identification information
    helps to reduce this growth and is called locator/
    identifier split (Loc/ID split) [4]. The stable
    identifier (ID) gives a global name to a node. A
    changeable locator (Loc) describes how the node can
    currently be reached through the global Internet.
    Furthermore, a mapping system (MS) is needed to map
    locators to identifiers. This principle makes routing
    in the stable Internet core more scalable because core
    routing is not affected by changed attachment points
    and multihoming of edge networks. The deployment of
    Loc/ID split in the Internet requires modifications to
    the current routing and addressing architecture.

In section 6, there is a further mention of "separation" of
"networks" rather than of the scalable "edge" address subset.  This
is a reference [32] to the 2008 sep./elim. paper and so simply
repeats its terminology:

    The authors of [32] identified two different
    strategies: separation of core and edge networks
    and elimination of de-aggregated provider-independent
    and provider-aggregatable addresses from BGP routing
    tables.

The next paragraph continues:

    Proposals implementing separation can be subdivided
    into address rewriting, map-and-encaps, and source
    routing approaches.

I am not sure what the "source routing" "separation" proposals are,
but so far this repeats the 2008 paper's taxonomy of considering a
proposal which does address rewriting as a "separation" (CES)
architecture: both Six/One Router and GSE.

But I am sure it is a mistake to characterize GSE as a CES
architecture, which is stated two sentences later.

The GLI-Split designers state that their architecture was "evolved
from the early ideas of GSE".  (Quotation below.)  They then assert
that ILNP and Six/One router are likewise "evolved from the early
ideas of GSE".  I agree this is the case with ILNP - and Ran states
this clearly: http://ilnp.cs.st-andrews.ac.uk/

However, I disagree that Six/One Router "evolved" from GSE.  There is
no mention of GSE in Christian's paper.  While both Six/One Router
and GSE involve address rewriting at the borders of EUNs, there are
fundamental differences between Six/One Router and any CEE
architecture, as listed above.

GSE is clearly a CEE architecture and is implementing
"Locator/Identifier Separation".

Six/One Router is not.  The word "separation" only appears in the
references of the Six/One Router paper.  Likewise "locator".
"Identifier" doesn't appear at all.

    With address rewriting, border routers add global
    locator information to packets destined for a
    different domain by coding this information into
    source and destination addresses for transit purposes.

    GLI-Split falls into that class.

GLI-Split does address rewriting, as does GSE (CEE, but mistakenly
classified as a CES architecture) and Six/One Router, which is
correctly classified as a CES architecture.

But this does not mean that GLI-Split is a CES architecture.  It is
clearly a CEE architecture.

    Also Six/One Router [9, 33] uses address rewriting.

This is true.

    Identifiers are only locally routable addresses.

This statement appears to be about Six/One Router, but it cannot be
true to say this of Six/One Router.  The term "Identifier" does not
appear in the Six/One Router paper I am quoting from, which I recall
is the latest version.  The last time we discussed Six/One Router on
the RRG was in August 2008, referring to this no-longer existent file:

http://users.piuha.net/chvogt/pub/2008/vogt-2008-six-one-router-design.pdf

I don't seem to have an electronic copy of this, but Christian
announced it on 2008-07-12:

  http://psg.com/lists/rrg/2008/msg01801.html

and the MobiArch08 paper I am quoting from:

http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf

was last updated on 2008-07-24, so I figure this is the final version.

I think it is wrong to state: "Identifiers are only locally routable
addresses.".

The "edge" addresses used by hosts in Six/One Router networks (which
I guess is what the GLI-Split authors were referring to) are indeed
only "locally routable" in a direct sense, but they are globally
unique, are returned by any DNS query in the world made from within a
Six/One Router network.  In a sense these addresses are "globally
routable" in that packets with these source and destination addresses
can be transported to any Six/One Router network in the world, via
the writing and rewriting of the upper 64 bits.  This in no way makes
them "Identifiers" - they are IP addresses and hosts and at least
some routers treat them just like IP addresses today: both as
Identifiers and Locators.

   When communicating with nodes in different domains,
   the addresses are rewritten 1-to-1 through stateless
   NAT in border routers to globally routable transit
   addresses. A major focus of Six/One Router is
   improved multi-homing support.

This is all fine.

Then there is some discussion of ILNP, which looks fine to me.  Then:

   GLI-Split, ILNP, and Six/One Router have evolved from the
   early ideas of GSE (global, site, and end-system address
   elements) [36, 37].

This is true of GLI-Split and ILNP, but not of Six/One Router.

   It essentially codes a global locator, a local
   locator, and an identifier into an IPv6 address.
   Addresses are dynamically combined from these parts.
   It uses only the identifier for TCP checksum
   calculation and requires host upgrades for deployment.

This is a brief description of GLI-Slit, but it is a mistake to think
 GSE, ILNP or GLI-Split are Core-Edge Separation architectures.

 - Robin

---

From: Robin Whittle <[email-masked]>
Date: Thu, 18 Feb 2010 13:54:14 +1100

Short version:    "Separation" is a verb and "core" and "edge" are
                  adjectives.  The important terms:

                    CES - Core-Edge Separation
                    CEE - Core-Edge Elimination

                  do not specify a subject.

                  The above terms are meaningful and highly useful if
                  the subject is "address", with further description of
                  exactly what is being separated or eliminated.

                  It is not meaningful to consider the subject to
                  be "networks", except in one particular sense,
                  which is unrelated to scalable routing.  Obviously
                  it makes no sense to talk of "Eliminating core
                  and edge networks" or to eliminating the distinctions
                  between them.  The DFZ and end-user networks are
                  already separate and will remain forever distinct
                  and separate.

                  The use of "network" in "Separation of the core
                  network (DFZ) from (all) edge networks" has a specific
                  narrow meaning which only applies to APT and is not
                  relevant to scalable routing.

                  The 2008 paper mentioned separation in two ways:

                  1 - Separation of *address space*: global unicast
                      addresses into a new, scalable, "edge" subset
                      with the remainder being known as "core",
                      which includes the addresses of DFZ and other
                      ISP routers, ITRs, ETRs and Default Mappers.

                      But in reality, in terms of achieving all
                      the goals of scalable routing, the "core"
                      addresses can still include all those end-user
                      networks which are on PA space, which don't
                      need portability, including hundreds of millions
                      of residential, SOHO and other end-user networks.

                      This sense of "separation" is applies to all the
                      CES architectures - APT, LISP, Ivip, TRRP, TIDR
                      and IRON-RANGER - and is vital to their ability
                      to solve the routing scaling problem.

                      So this "address" sense of "separation" is
                      the important one in understanding the meaning
                      and use of the term "Core-Edge Separation".

                  2 - The separation of all edge *networks* from the
                      "core".  They they are already separate, but this
                      second "network" sense of separation refers to the
                      the prevention of packets sent from any "edge"
                      network from arriving at the addresses of any core
                      device: (DFZ router, other routers, ITRs, ETRs,
                      DMs etc.  This has also been referred to as:

                         Universal division line, between "the core"
                         and all edges

                      which is part of the title of a subsequent
                      message.  This second sense is a long-term goal of
                      APT, but not of the other architectures.  It is
                      not at all required for APT or any other CES
                      architecture to be able to solve the routing
                      scaling problem.

Further to my message:

  http://www.ietf.org/mail-archive/web/rrg/current/msg06009.html

Here are few more thoughts on the 2008 paper:

 - Robin

Initial CES and CEE definitions
-------------------------------

In the introduction, page 1 leading into page 2, here is the initial
definitions of CEE and CES.

CEE:  Solutions in the elimination category require that
      edge networks take address assignments from their
      providers; as a result a multihomed edge network
      will use multiple PA addresses internally and must
      modify end hosts to support multihoming.

This is not at all about elimination of the distinction between of edge
and core networks.  It is about there being no more need for the current
kind of unscalable "edge" space: PI prefixes advertised in the DFZ.
Instead, all end-user networks run on one or more PA prefixes, and the
new CEE arrangements - typically upgraded stack and applications, and
for IPv6 only - enable the end-user networks and their hosts to achieve
portability (of Identifiers), multihoming and inbound TE.

"Elimination" is of a previously existing (PI) "edge" class of *address*
space, and furthermore the solution of the routing scaling problem
without the generation of a new subset of "edge" space, in contrast to
CES architectures which do create such a new subset of scalable "edge"
space.

So with CEE, the "elimination" term is used in respect of *addresses*.

With CES, there is a sense in which this paper refers to separation in
terms of *networks* (preventing packets sent from any "edge" network
from reaching any device with a "core" address), but this only applies
to APT and is not important to its ability to solve the routing scaling
problem.  LISP and Ivip at least are CES architectures which have no
such goal.  None of the CES architectures need to achieve this
"networks" sense of separation in order to solve the routing scaling
problem.

CES:  Solutions in the separation category insert a
      control and management layer between edge networks
      and today’s DFZ, which we refer to as the Internet’s
      transit core; edge networks would no longer
      participate in transit core routing nor announce
      their prefixes into it.

Neither the first or second parts of this sentence directly concern the
prevention of packets flowing between devices with "core" addresses and
those with "edge" addresses, which is what is meant by:

   Universal division line, between "the core" and all edges

I write more about this in a message to follow.

The first part refers to the ITRs, ETRs or whatever.  The second refers
to those edge networks which today advertise their PI space as prefixes
in the DFZ, no longer doing so once they adopt the CES managed scalable
"edge" space.

The first part of the sentence doesn't refer specifically to
"separation", but does involve placing something "between" "edge"
networks and the (transit) "core": the DFZ.

The "edge" end-user networks are already separate from the core, in that
they are different, independent, routing systems and within themselves
are not involved in the DFZ.  Today, the only way of multihoming and
"edge" end-user network is for it to have PI space (or perhaps to
advertise some PA space from one ISP via another, which amounts to the
same thing in terms of impact on the DFZ) and to advertise this via two
or more ISPs to the DFZ, or perhaps via its own DFZ routers.

So these PI-using end-user "edge" networks are placing burdens on the
DFZ routing system, but they are not really part of it.  They don't
advertise prefixes of any other networks - if they did, then we would
consider them transit providers, or ISPs.

So "edge" networks are already "separate" from the DFZ.  Some of them
(PI-using end-user networks) advertise their own space.  The rest use
space from their ISPs, which they receive as "PA" space.  Only the
PI-using end-user networks directly drive the routing scaling problem
today.  But the hidden part of the routing scaling problem is the subset
of the non-PI-using end-user networks which want or need portability,
mulithoming and inbound TE, but can't get it.

The first part of the sentence again:

      Solutions in the separation category insert a
      control and management layer between edge networks
      and today’s DFZ, which we refer to as the Internet’s
      transit core;

This doesn't imply that the edge networks are not already separate from
the "core".  It just states that a new system is added in between them.

The second part:

      edge networks would no longer participate in transit
      core routing nor announce their prefixes into it.

refers to the separation of *address space* (not networks), with the
edge network's space being part of an implicitly "edge" subset of the
global unicast address range which is not advertised (at least directly)
into the core=DFZ.

In fact, to handle packets sent by hosts in non-upgraded networks, a CES
architecture does need to advertise the "edge" space in the core.
However, it doesn't need to advertise every such end-user "edge" prefix
individually in the core.  All that is needed is a "coarse" prefix (as
Dino recently referred to these otherwise unnamed things in LISP) or
what is known in Ivip as a Mapped Address Block (MAB).  One MAB can
cover thousands of individual prefixes, of thousands of different
end-user networks - and so solve the routing scaling problem by
burdening the DFZ with just one prefix, rather than the thousands which
would be required if these were all conventional PI prefixes.

There is no mention in this original definition of a "separation"
between the core and all edge networks which would result in the
prohibition of packets being sent from "edge" addresses to "core"
addresses.  For a discussion of what that might involve, please see a
forthcoming message: "APT: Universal division line, between 'the core'
and all edges".

In the previous message in the current thread, I discussed to some
extent of the goals set forth in the 2008 paper which could supposedly
be achieved with this "prevention of packets from edge to core", AKA
"Universal division line" approach to separating edge from core
*networks*.  Search for "4.3" in that message.

The are titled:

  Rolling out new protocols
  DDoS mitigation
  Ingress traffic engineering

The last of these - "Ingress traffic engineering" - does not require any
particular level of adoption of a CES architecture, as long as the
architecture supports all packets sent by non-upgraded hosts, which
LISP, Ivip and in principle APT can do.  This goal certainly doesn't
depend on the "network" separation of being able to prevent "edge" hosts
from sending packets to routers or other devices in the "core".

The "Rolling out new protocols" goal could well be supported by
widespread or perhaps universal adoption of CES by the end-user networks
- but I don't see how it could rely upon prevention of edge hosts from
sending packets to core routers.

Likewise the paper's discussion of the "DDoS mitigation" goal (to
whatever extent this loose proposal is actually feasible) could be
achieved by using the mapping system and ITRs, ETRs etc. of any CES
architecture, irrespective of whether "edge" hosts were prevented from
sending packets to "core" routers.

So I think this section 4.3, to the extent it is true (and I think third
goal is definitely realistic) applies to all CES architectures in the
"separation of *address space* into edge and core subsets" sense, and is
not at all dependent on the "Universal division line separation of
*networks* to prevent edge hosts sending packets to core routers".

Likewise, in the other sections regarding "separation":

  4.1 Aligning Cost with Benefits
  4.2 Accommodating Heterogeneity
  5. SEPARATION IS COMPATIBLE WITH MULTIPATH TRANSPORT

seem to depend only on CES in terms of separating addresses into edge
and core subsets, and not at all on the "Universal division line
separation of *networks* to prevent edge hosts sending packets to core
routers".

I conclude ....  I put my conclusion at the start as "Short version".

---

From: Robin Whittle <[email-masked]>
Date: Mon, 22 Feb 2010 02:01:54 +1100

This is a revised version of my (msg06060), with new terms:

   SPI   Scalable Provider Independent:  "Edge" space handled by a
         CES architecture's ITRs, ETRs etc. and so which is Provider
         Independent (portable) and suitable for multihoming and
         inbound TE.

   MAB   Mapped Address Block:  A prefix advertised in the DFZ which
         covers a typically large amount of SPI space which is
         typically used for many (tens to hundreds of thousands)
         of individual SPI micronets (Ivip) or EID prefixes (LISP).
         Dino recently used the term "coarse prefix" to refer to
         the same thing in LISP.

   EUN     End User Network:  Any user's single host or network, from
           an IP-connected cellphone, to a single IPv4 DSL service to
           a university, corporate or government network.

   EUN-PI:  End User Network using (unscalable) PI space.

   EUN-PA:  End User Network using PA space - including home & SOHO
            DSL etc. end-users.

   EUN-SPI: End User Network using SPI space = "edge" space in a CES
            architecture.

   CES   Core-Edge (address) Separation: a subset of the global
         unicast address space is separated from the remaining
         "core" global unicast addresses and is supported by the
         CES system (ITRs, mapping system and ETRs) to be used as
         "SPI" space.  Also a class of scalable routing architectures
         in which this is done - including LISP, APT, Ivip, TIDR and
         IRON-RANGER.

   CEE   Core-Edge (address) Elimination:  A class of scalable
         routing architectures in which the Locator / Identity
         Separation naming model is introduced, replacing the
         current IPv4/v6 model in which the IP address plays both
         the Identifier and Locator roles.  Includes ILNP, GSE,
         GLI-Split, Name-Based Sockets.  (Does not include LISP -
         LISP does not use Locator / Identity Separation.)

         Hosts have portability of Identifiers and are able to do
         multihoming and inbound TE on PA IP addresses, which are
         also known as "core" addresses.  Therefore there is no need
         for any "edge" addresses - the unscalable PI addresses
         which are currently the only way of achieving portability,
         multihoming and inbound TE, or any other type of IP address
         with special characteristics for EUNs.

   EUN-SPI Full Adoption:  (Applicable only to a CES architecture.)
         All EUNs which want or need portability, multihoming and/or
         inbound TE have adopted SPI space.  This includes any
         EUN-PI networks - so there are no more PI prefixes.  The
         CES architecture has achieved 100% of the routing
         scalability improvements it could possibly achieve.  Many
         hosts are still on ISP's prefixes - ISP's hosts and all
         EUN-PA networks - but these in no way constitute a scalable
         routing problem, since they do not burden the DFZ or result
         in any loss of portability, multihoming and/or inbound TE to
         any EUN which wants these benefits.  (Those which do are
         already EUN-SPI networks.)

   Universal SPI Adoption:  Every host - including those of ISPs and
         of those in what were formerly EUN-PI or EUN-PA networks -
         uses SPI "edge" space.  This is well beyond "EUN-SPI Full
         Adoption", and does not involve any further improvement to
         routing scalability.

   CENI  Core-Edge Network Isolation:  Once Universal SPI Adoption
         has been achieved, the routing system is altered so no host
         can send packets to any "core" address: any address of a DFZ
         or some other (ISP) routers, or of any ITR or ETR.  (This
         may not make much sense.)  Only APT had this as a goal, but
         it seems APT's designers, who made up many of the authors
         of the 2008 paper, thought (and perhaps still think) that
         other CES architectures such as LISP and Ivip had this goal
         too - but this is not the case.

Short version:   Core-Edge  Separation / Elimination
                 \________/ \______________________/
                 Adjectives  Verbs

                 Are the adjectives referring to:

                      addresses?
                      networks?

                 In "Core-Edge Elimination", the adjectives refer to
                 addresses: there being no more need for "edge"
                 addresses, meaning today's unscalable PI addresses,
                 or any other kind of  address for EUNs.

                 If the adjectives referred to "networks", it would
                 make no sense, since neither the core network nor
                 edge networks are being eliminated - nor are the
                 distinctions between them being eliminated

                 If the adjectives in "Core-Edge Separation" are
                 assumed to refer to "addresses" then this is a
                 meaningful and useful term:  Separation of global
                 unicast addresses into "edge" and "core" subsets
                 makes sense and is a useful distinction for
                 categorising the CES class of scalable routing
                 proposals, the other class (CEE) eliminates the
                 need for "edge addresses".

                 If the adjectives referred to "networks", then
                 "Core-Edge Separation" makes no sense, since these
                 networks are already separate from each other and
                 will always remain so.

                 However, it seems that the term "Core-Edge
                 Separation" is sometimes used in a way which
                 conflates:

                    Separation of a global unicast addresses into a
                    subset of scalable "edge" addresses, leaving the
                    remainder as "core" addresses.

                 with:

                    Isolation of all "edge" networks from the "core"
                    network (the DFZ, ISP routers, ITRs and ETRs) so
                    that hosts in "edge" networks cannot send packets
                    to "core" addresses.

                 This leads to lots of confusion.

                 I think this paper refers generally to the
                 core-edge "separation" being the separation of
                 the global unicast *address* space into "core"
                 and "edge" subsets.  This is the defining
                 characteristic of CES architectures.

                 However, I believe they were also thinking of, and
                 sometimes referring to with the same "core-edge
                 separation"  term, something very different:
                 Core-Edge *network* "separation", but really meaning
                 "Core-Edge Network *Isolation*".  This was a goal of
                 APT, but is not a goal of other CES architectures -
                 but the authors may have thought that other CES
                 architectures did have this goal.

                 Many of the authors of this 2008 paper were the
                 designers of APT, which separated not just the
                 address space into two subsets, but had a long-term
                 goal of actually "separating" (really *isolating*)
                 all the edge  networks as a set, from the core, so
                 no edge host could physically send a packet to any
                 core address.

                 I think this double sense of the term "separate":

                    Separation of global unicast address space into
                    "edge" and "core" subsets.

                    ("Separation" = Isolation) of all edge networks
                    from the core network.

                 plus the paper's mistaken classification of GSE as
                 CES contributed to the GLI-Split authors considering
                 their proposal to be a CES architecture too.  Yet
                 GLI-Split is clearly CEE, since it implements the
                 Locator/Identification Separation naming model.

                 Six/One Router has some similarities to a CES
                 architecture, but uses address rewriting (AKA
                 "translation") rather than tunneling.  I think it
                 is not a proper CES architecture, in part because
                 hosts in upgraded networks can't be reached by
                 hosts in non-upgraded networks on their real
                 addresses, requiring DNS tricks to return different
                 IP addresses to hosts in non-upgraded networks,
                 which is the only way the latter type of host can
                 initiate communications with a host in a Six/One
                 Router network.

                 Comparison charts of:

                 CES: LISP-ALT/NERD, APT, Ivip, TRRP, TIDR, and
                      IRON-RANGER.

                 I believe the GLI-Split designers were mistaken to
                 class their proposal as a Core-Edge Separation (CES)
                 architecture.  I am sure it is a CEE (Core-Edge
                 Elimination) architecture.

Please also see these messages for discussion of the current IP
naming model, why I believe the IP address plays both the Identifier
and Locator roles - and how this is still 100% true when a CES
architecture such as LISP or Ivip etc. is used and the destination
address is an EID (LISP) or SPI address (Ivip):

  LEIDs, SPI & ordinary IP addresses as both IDs & Locs
  http://www.ietf.org/mail-archive/web/rrg/current/msg06070.html
  http://www.ietf.org/mail-archive/web/rrg/current/msg06082.html
  and other messages in this thread.

This paper:

  Re: [rrg] Terminology

  1 - The definition of exactly what core and edge things are
      being separated or eliminated could be improved - to
      refer to separation into two subsets of addresses and to
      use a different term for what I refer to as "CENI - Core-
      Edge Network Isolation".

but really both CEE and CES maintain the already existent separation
between edge networks from the transit core.  (See discussion of
section 4.3 below for my understanding of why the authors may have
emphasised the "separation of networks" (really CENI) rather than the
separation of a new subset of scalable "edge" addresses from the
remaining "core" addresses.  They intended APT to ultimately
"separate" (really *isolate*) the two systems so much that an edge
host would be physically unable to send a packet to a core router.
They also seem to have assumed that LISP and other CES architectures
were intended to do this as well, which is not the case

What is being separated in a CES architecture is a new, scalable
"edge" subset of *addresses* (SPI addresses in Ivip - EID addresses
in LISP) from the remainder of the global unicast addresses, which
are then known as "core" addresses.  This is a different concept from
"separating edge *networks* from the core", since the edge networks
are already separate from each other and separate from the DFZ, and
this use of "separation" really means isolation between things which
are already separate.

At present, any EUN which is actually getting portability,
multihoming and inbound TE is getting it with PI prefixes - which are
advertised in the core routing system (DFZ) just like ISP's prefixes.
 While these PI prefixes are for "edge" networks (since these are
EUNs, rather than ISP or transit provider networks), these PI
prefixes are part of the "core" address space, since all DFZ routers
need to develop best paths for each such PI prefix.

  CES involves end-user networks' hosts using global unicast
  addresses from a specially handled "edge" (AKA SPI, AKA EID) subset
  of the global unicast address space, with the remainder being known
  as "core" addresses.

  Global unicast IP addresses in both the "core" and "edge" subsets
  will continue to be used, as are all today, both for Identifying
  hosts and for the Routing system to decide how to forward packets
  (i.e. as a "Locator").  See msg06070 and msg06082.

  But note that CES has a new, fancy, way of implementing the
  Locator semantics of destination addresses whose IP address is
  in the new scalable "edge" (SPI, EID) subset.  msg06082 has a
  detailed example of this 2nd algorithm, which is only performed
  by ITRs.  The original 1st algorithm, as used by all routers except
  ITRs, still works as normal on these "edge" destination addresses.

       Role             Level

       Text name  <---- FQDN

Most routers make no distinction between how they handle packets with
destination addresses in these two subsets.  The exception is ITRs,
which make a big distinction.  ITRs implement the Locator
functionality of destination "edge" addresses in a totally different
way: by looking up mapping and tunneling the packet to an ETR, rather
than by forwarding the packet to a neighbour.

More on naming models:

Here are some cases:

      These will work in  principle and in practice for both IPv4 and
      IPv6.

CES-like - Six/One Router. See footnote.

     This is an extension of his earlier 8+8 architecture - both are
     for IPv6 only due to the way multiple items are combined to
     create the final IP address.

     Multihomed EUNs get a prefix from each of their two or more
     ISPs, and the identity of hosts is unrelated to this, or to
     any IP address.  These prefixes are provided by the ISP as
     PA space.  The ISP typically has a single large (short) prefix
     which it splits into many such PA prefixes for many such EUNs.
     Host Identity - and so all communications establishment and
     session continuity - is determined by the value each host uses
     for the 64 bit ESD, which exists in a separate namespace from
     IPv6 addresses.

CEE: GLI-Split

     I believe GLI-Split is a CEE architecture, because it introduces
     a separate "ID" object (Figure 2), with a separate namespace
     from IPv6 addresses or anything else, to Identify each host.
     This is different from the objects used to Locate them: the
     GL or LL Locators.

    More on GLI-Split at the end of this message.  The authors
    explicitly state that it is a CES architecture, but I am sure
    it is a CEE architecture.  I think they are following a mistake
    made which I believe was made in the the 2008 sep./elim. paper.
    See also my draft critique of GLI-Split and what the designers
    will hopefully soon write in response: (msg06056).

CEE: RANGI

     See my recent questions and Xiaohu Xu's reply.

???: Name Overlay Service (NOL)

     See (msg06062).  Neither CEE nor CES.  It is NAT-like and I
     do not think it is at all practical.

     Now in version 05, which is different from what I critiqued.
     See: "hIPv4-05: new header" thread.

     CEE or CES.

     I don't yet understand this, but the authors state that the
     end-result somewhat resembles their previous APT CES
     architecture.

Here are some charts:

ILNP                    v6   App       Stack      -

Name Based
Sockets                 v6   App       Stack      -

* GLI-Split does not require host changes, but Michael Menth wrote
  to me off-list that new applications could be used, for instance
  for multipath protocols, if the stack had a suitable new API:

The fact that the somewhat CES-like architecture Six/One Router has
routers rewriting addresses does not mean that GSE is a CES
architecture simply because it too has routers rewriting addresses.

Note that the verb "eliminate" has an object: a class of *address*.
This is congruent with my claim that the important CES and CEE
distinctions concern separation or elimination of classes of address
- not "separation" (meaning isolation between) two classes of *network*.

I wouldn't state it so strongly - a CEE architecture doesn't
"enforce" anything or forcibly "eliminate" PI prefixes.

I think it is more accurate to say that when all hosts in all
networks adopt CEE, the CEE mechanisms can be relied upon entirely
for portability, multihoming and inbound TE.  Then, and only then,
this means that no network needs PI space to achieve these things.
This doesn't mean that PI space (the current, unscalable, form of
"edge" space) would automatically be eliminated - just that it could
be eliminated without any network missing out on portability,
multihoming or inbound TE for all its hosts and communications.

I would rephrase it slightly.  Each EUN which adopts CEE would
generally use space like this.  Each such EUN *could* use PI space.
Its just that PI space is no better than PA space for use with a CEE
architecture.  (I am including in my definition of "PI" usage a EUN
using a prefix it gets as PA space from one ISP via another ISP as a
"more specific PA prefix" - which also has the effect of adding
another prefix to the DFZ control plane.)

>From the abstract:

   The first direction, which we dub separation, calls for separating
   edge networks from the transit core, and engineering a control and
   management layer in between.

Edge networks are already separate from the core.  This initial use
of "separation" probably was intended to mean "isolating all edge
networks from being able to send packets to any core address" - which
is clearly something the APT designers intended for APT.  It seems
the authors also thought that all other CES architectures had the
same intention - but this is not the case.

So I think the authors at times conflate two completely different things:

   CES   Core-Edge (address) Separation
   CENI  Core-Edge Network Isolation

referring to both with the same term "Core-Edge Separation" and
furthermore imply that CES architectures other than APT - LISP, Ivip
and others - also intend to do both of the above.

Nonetheless, most of their usage of "Core-Edge Separation" seems to
refer to the separation of a new, scalable, "edge" subset of the
global unicast *address* space from what remains as the "core" - and
does not refer to the "Separation" (isolation between) core and edge
*networks*.

  1 - All EUNs use APT, and so use the scalable "edge" subset of
      the global unicast address space.  Furthermore, all hosts in
      ISPs which need to communicate with hosts in EUNs are also
      using SPI space.

      This is not just what I now call: "EUN-SPI Full Adoption".

      It goes well beyond this - well beyond what is required to
      maximise scalable routing benefits - to what I call "Universal
      SPI Adoption".

  2 - Therefore, no EUN needs PI space or "more specific PA
      prefixes".  So any space which was previously a PI prefix
      has been converted either to the new scalable "edge" subset
      of space or to the remainder - the "core" subset.

  3 - Therefore, ISPs advertise prefixes into the DFZ which are
      all in the "core" subset.  ISPs use these addresses for
      ITR and ETRs, and for internal and DFZ routers.

      This is what I now call "CENI - Core-Edge Network Isolation".

AFAIK, no other CES architecture had this complete, physical,
(separation) isolation of EUN hosts or routers in EUNs - all on
scalable (SPI = EID) "edge" addresses - from being able to send or
receive packets to or from "core" addresses as a goal.  I specify it
as a non-goal for Ivip.

They go on to discuss "Rolling out new protocols", "DDoS
mitigation" and "Ingress Traffic Engineering" - all of which they
argue would be possible or easier only with complete "separation" of
the "edge" networks from the "core" networks.

I think "Ingress Traffic Engineering" is possible with any level of
adoption of APT (assuming that APT, like LISP with PTRs and Ivip with
DITRs supports packets sent from non-upgraded networks, which it can)
- so this depends on Core-Edge (address) Separation, but not on
"EUN-SPI Full Adoption".  Support for "Ingress Traffic Engineering"
certainly doesn't depend on "Universal SPI Adoption" or "CENI
Core-Edge Network Isolation".

   Side-note on Ingress Traffic Engineering:

      Inbound TE works fine due to any amount of adoption of the
      CES architecture.  As long as the CES architecture supports
      all packets sent from non-upgraded networks, inbound TE will
      be for packets arriving in all the adopting networks, no
      matter how many other networks have so-far adopted the CES
      architecture.

      What the authors write in the second paragraph about Site1
      having preferences for which of Site2's two or more ETRs to
      use doesn't strike me as part of most CES architectures.  I
      think the aim of most CES architectures is to to have the
      recipient site control which of its ISPs the packets arrive
      via.  This is inbound TE.  Maybe APT had some concept of the
      sending EUN controlling this, but I guess most sending EUN
      wouldn't care which ISP used by the destination EUN the
      packets went through.

In their discussion of the other two goals: "Rolling out new
protocols" and "DDoS mitigation", to the extent that these goals are
facilitated by any kind of core-edge separation, they too are
dependent on "Core-Edge (address) Separation" - and do not depend at
all on "EUN-SPI Full Adoption", "Universal SPI Adoption" or "CENI
Core-Edge Network Isolation".

So I think the phrase at the start of this paragraph "Separating
edges from the transit core" is misleading if it means "isolating"
edge networks from being able to send packets to core addresses.  It
would be better to use the phrase: "Deploying an architecture in
which a scalable subset of 'edge' space is separated from the
remaining 'core' global unicast addresses" - AKA "Deploying a CES
architecture".

The routers in the sending host's network can choose which ISP link
to send out the packets on (assuming the ISPs accept packets with
"edge" addresses, which they will have to when CES is widely used).
If those routers are ITRs, then in some or many CES architectures
(but not Ivip) then perhaps the ITR might be locally programmed to
prefer one of multiple destination ETRs over others - so in principle
the ITRs in the sending network could affect the incoming load
balance of the destination network.  However, as far as I know, the
intention with all CES architectures is to allow the sending network
to choose the outgoing ISP and the recipient network to choose
(inbound TE) its incoming ISP(s).  In the case of Ivip, the recipient
network only provides a single ETR address, so there is no option for
the sending network to have its ITRs tunnel traffic to some other ETR
of the destination network.

Many of the authors of the 2008 paper also wrote the 2009 RRG
proposal: AIS / Evolution AKA "Aggregation with Increasing Scopes: An
Evolutionary Path Towards Global Routing Scalability".  In this they
seem also to be conflating CES (address) with what I now call CENI -
Core-Edge Network Isolation.  I will write more about that in my
forthcoming discussion / critique of the AIS proposal.

I think the GLI-Split authors were mistaken to state that GLI-Split
is a CES architecture.

The summary:

Except for the mention of "source routing" "separation" proposals
(which I don't understand at all), this repeats the 2008 paper's
taxonomy of considering a proposal which does address rewriting as a
"separation" (CES) architecture: both Six/One Router and GSE.

The GLI-Split designers state that their architecture was "evolved
from the early ideas of GSE".  (Quotation below.)  They then assert
that ILNP and Six/One router are likewise "evolved from the early
ideas of GSE".  I agree this is the case with ILNP - and Ran Atkinson
states this clearly: http://ilnp.cs.st-andrews.ac.uk/

However, I disagree that Six/One Router "evolved" from GSE.  There is
no mention of GSE in Christian Vogt's paper.  While both Six/One
Router and GSE involve address rewriting at the borders of EUNs,
there are fundamental differences between Six/One Router and any CEE
architecture, as mentioned in the footnote below.

Six/One Router is not.  The word "separation" only appears in the
references of the Six/One Router paper.  The words "Locator" and
"Identifier" don't appear at all.

GLI-Split does address rewriting, as does GSE (CEE, but mistakenly
classified as a CES architecture) and Six/One Router.

This is true.

The "edge" addresses used by hosts in Six/One Router networks (which
I guess is what the GLI-Split authors were referring to) are indeed
only "locally routable" in a direct sense, but they are globally
unique, are returned by any DNS query from within any Six/One Router
network.  In a sense these addresses are "globally routable" in that
packets with these source and destination addresses can be
transported to any Six/One Router network in the world, via the
writing and rewriting of the upper 64 bits.  This in no way makes
them "Identifiers" - they are IP addresses and hosts and at least
some routers treat them just like IP addresses today: both as
Identifiers and Locators.

This is all fine.

 - Robin

Footnote on Six/One Router:

      I previously stated that this was "clearly a CES architecture"
      but now I think it is not one, despite some resemblances.

      Between hosts in upgraded networks, it appears these end-user
      networks operate from a single prefix drawn from an "edge"
      subset of the global unicast address range.  (Sect. 2.1.)

      So there isn't a single address by which a host in a Six/One
      Router network can be contacted or communicated with by both
      other such hosts, or hosts in ordinary IPv6 networks.  To those
      hosts in ordinary networks, the "edge" subset of addresses
      doesn't work.  No other CES architecture is like this, and I
      now think it is not correct to think of Six/One Router as being
      as much a CES architecture as LISP, APT, Ivip, TRRP, TIDR or
      IRON-RANGER.

      Unfortunately, Six/One Router can't be used for IPv4 due
      (I guess) to the more chaotic EUN prefix lengths and (I am
      sure) due to the need for a 2 ISP multihomed EUN to consume 3
      times the amount of global unicast address space it really
      needs.  (One for its "edge" prefix and then the same amount
      of PA space from each of its ISPs.)  Also, when Christian and
      I last discussed it on the RRG, I recall that he wrote that he
      needed a flag bit from the IPv6 header, which would require a
      modified IPv6 header format and so (I guess) require updates to
      all DFZ and many other routers.

---

From: Robin Whittle <[email-masked]>
Date: Wed, 24 Feb 2010 17:41:43 +1100

This is a revised version of my msg06060 and msg06089 (v2) message,
to correct my mistaken statement that ILNP requires upgraded
applications.  Please see (msg06103) and (msg06105) for Ran and I
discussing this.  I have also added a note that ILNP optionally
allows routers to rewrite Locator fields in ILNP packets.

I have added RANGI to the CEE architecture table and added some
notes about a forthcoming, fully decentralised, real-time, mapping
system for Ivip and potentially LISP.

Changes are marked '!'.

  - Robin

                      addresses?
                      networks?

                 with:

                 GSE and GLI-Split are translation-based CEE
                 architectures.  Like ILNP (which I understand will     !
                 optionally allow routers to rewrite Locator fields     !
                 in ILNP packets) GSE and GLI-Split they do not         !
                 require IPv6 applications to be modified.              !
                 (However I just learnt that GLI-Split could
                 support new application functionality via a new
                 API.)

                 Comparison charts of:

                 CEE: GSE, GLI-Split, ILNP, Name-Based Sockets, RANGI.  !

This paper:

  Re: [rrg] Terminology

CES

CEE

  Hosts (either upgraded stack and ordinary IPv6 application, or        !
  upgraded stack and upgraded applications) use this new Identifier,    !
  which is in a different namespace to IP addresses and any Locators    !
  used in the system, to identify each host.  Hosts can be reached      !
  via one or more potentially unstable Locators, and the applications   !
  are unaffected due to either:                                         !
                                                                        !
     1 - The non-upgraded application is unaware of the changed         !
         Locators, due to the actions of the upgraded stack and         !
         potentially the actions of upgraded routers which rewrite      !
         Locator fields in packets, or:                                 !
                                                                        !
     2 - The application sees or generates the different Locator        !
         values, but continues the session without disruption           !
         because the session is bound to the Identifier of the          !
         remote host.                                                   !

       Role             Level

       Text name  <---- FQDN

More on naming models:

Here are some cases:

CES-like? - Six/One Router. See footnote.

CEE: GLI-Split

     This is clearly a CEE proposal, because it has separate levels
     for Identifier and Locator.  As currently specified, it is a     !
     purely host-based descendent of GSE.  However, according to      !
     (msg06103, see also msg06105), ILNP has, or will have, optional  !
     router rewriting of Locator fields.

     There is no need for new routers (but see previous sentence),    !
     but the host stacks need to be upgraded.  These ILNP upgraded    !
     hosts work with ordinary IPv6 applications - but see (msg06105)  !
     where I ask if with a new API, upgraded applications could work  !
     directly with ILNP's Locator / Identifier Separation naming      !
     model.                                                           !

CEE: RANGI

     See my notes in (msg06093).  The authors state that the                 !
     end-result somewhat resembles their previous APT CES                    !
     architecture.                                                           !

Here are some charts:

CES architectures
-----------------

               IPv4   IPv6   Tunneling  Local or         Mapping             !
                                        "nearby"         speed               !
                                        full database                        !
                                        mapping servers                      !

Ivip             v4     v6   Encaps.    Local or         Fast - with real-   !
                             or MHF     "nearby" *       time updates to     !
                                                         ITRs which need it. !

* I am about to write a message "DRTM - Distributed Real Time Mapping        !
  for Ivip and LISP" which extends the new arrangements of (msg05975)        !
  to add an intermediate arrangement which will probably be an               !
  enduring one, though it will still be possible for ISPs and EUNs to        !
  run their own local (within their network) full-database QSD mapping       !
  query server.  The new arrangement will rely on typically "nearby"         !
  query servers provided directly or indirectly by the companies which       !
  run the MABs.  Typically, these query servers will be within a few         !
  thousand km of the ISP or EUN's Map Resolvers, through which ITRs          !
  query these new query servers.  So this is no longer "local", but          !
  it will typically "close enough to be fast and reliable" - so avoiding     !
  the objections to global query server systems such as LISP-ALT.  At        !
  the same time, the new arrangement will not involve any single server      !
  being required to hold the full mapping database - so overcoming, I        !
  hope, some objections to Ivip concerning "globally synchronised            !
  mapping database" and the like.                                            !

CEE architectures
-----------------

Please see (msg06105) where I respond to Ran Atkinson's assertion            !
that his ILNP proposal is not a CEE architecture.  So far, I think           !
he has not supported his assertion with any arguments - so for now           !
I consider ILNP to be a CEE architecture.                                    !

             IPv4   IPv6  Do IPv6    Stack      Router upgrades              !
                          applicat   upgrades   required - for               !
                          ions need  required?  rewriting Locator            !
                          to be                 or other fields?             !
                          upgraded?                                          !
                                                                             !
GSE                  v6   No         Yes        Yes                          !
                                                                             !
GLI-Split            v6   No*        Yes        Yes                          !
                                                                             !
ILNP                 v6   No^        Yes        Optional router rewriting    !!!
                                                of Locator fields.           !!!
Name Based                                                                   !
Sockets              v6   Yes        Yes        No                           !
                                                                             !
RANGI                v6   Yes        Yes        Only for proxies to          !
                                                IPv6 hosts.                  !

* GLI-Split does not require application changes, but Michael Menth          !
  wrote to me off-list that new applications could be used, for              !
  instance for multipath protocols, if the stack had a suitable new          !
  API:                                                                       !

^ Please see (msg06105) where I ask Ran whether ILNP could include           !
  a new API so upgraded applications could directly work with ILNP's         !
  "Locator / Identifier Separation" naming model.                            !

The summary:

states this clearly: http://ilnp.cs.st-andrews.ac.uk/ .

This is true.

This is all fine.

This is a brief description of GLI-Split, but it is a mistake to think
GSE, ILNP or GLI-Split are Core-Edge Separation architectures.

 - Robin