Subject: [rrg] CES & CEE are completely different (graphs)

From: Robin Whittle <[email-masked]>
Date: Tue, 02 Feb 2010 00:33:29 +1100

Short version:    CES architectures are completely different from
                  CEE architectures - and CES is very much superior
                  to CEE.  There are no proposals which combine
                  elements of both.

                  I argue in a companion message:

                    Today's "IP addr. = ID = Loc" naming model
                    should be retained

                  that the current 2-level naming structure - in
                  which the IP address performs the roles of both
                  Identifier and Locator - is superior to the CEE
                  Loc/ID separation approach in which these roles
                  are played by two different objects, in different
                  namespaces.

                  While CEE makes life easy for the routing system,
                  it burdens all hosts with extra work, extra
                  traffic and extra delay just to establish almost
                  any communication session.

                  CES is superior to CEE because it maintains the
                  current naming system.

                  CES is also much easier to adopt than CEE.

                  CES provides immediate full benefits to all
                  adoptors - and provides routing scaling benefits
                  in direct proportion to the level of adoption.

                  CEE only provides substantial benefits to adopters
                  - or significant scaling benefits - after all, or
                  almost all networks have fully adopted it.  This
                  means upgrading essentially every host stack and
                  rewriting essentially every application to use the
                  new CEE Identifier-based method of communication.

Hi Joel,

In "Re: [rrg] Multiple critiques, choice of single proposal, ..."
(msg05846) you wrote:

> If someone wants to add text on Identifier / Locator separation as a
> useful architectural principle, I guess I can't object.  

Do you mean Core-Edge Elimination (CEE) or Core-Edge Separation
(CES)?  Please see this recent message for my concerns about how this
term means different things to different people:

  Terminology: Loc/Id sep., LISP, CEE & CES, namespace
  http://www.ietf.org/mail-archive/web/rrg/current/msg05858.html

> And I consider the CEE / CES debate to be missing the point.

I completely disagree.  These are both capable of solving the scaling
problem, but they are completely different approaches with completely
different costs and methods of achieving their goals.

A CES architecture (LISP, APT, Ivip, TRRP, TIDR or RANGER):

  1 - Maintains the current 2 level naming arrangement - where the
      FQDN is performs the "Text name" role and the Identifier and
      Locator roles are both performed by the IP address.

          Role             Level           Conventional IP & with CES

          Text name  <---- FQDN
          Identifier <---] IP address
          Locator    <---]

  2 - Scalable routing is achieved by creating an "edge" subset of
      the global unicast address range - the remainder of which is
      known as the core.  The "edge" addresses are all covered by
      DFZ-advertised prefixes, but these are few in number, since
      each such prefix (a MAB - Mapped Address Block in Ivip) covers
      a large number of "edge" addresses.  This means the "edge"
      addresses are not much of a burden on the DFZ control plane -
      and do not add much to the numbers of routes to handled by
      the RIBs and FIBs of DFZ routers.

      A new set of network elements - ITRs, ETRs and a mapping
      system - handle all packets addressed to an "edge" address -
      by tunneling them across the DFZ to an ETR, which is always
      on a "core" address.  This is done in a way which supports
      the portability, multihoming, inbound TE and ideally mobility
      needs of end-user networks of all sizes.

      Furthermore, this system enables the edge space to be divided
      much more finely than is possible with the DFZ, since with
      the IPv4 DFZ at least, there is a widely supported convention
      not to propagate prefixes longer than /24.  Ivip and LISP can
      divide "edge" space down to single IPv4 addresses.  Ivip for
      IPv6 manages "edge" space in units of /64.

  3 - Requires no change to host stacks, APIs or applications.
      (Nonetheless, in Ivip the ITR function can optionally be
      implemented in the sending host.)

  4 - Does not affect most internal and DFZ routers, but requires
      the new ITR and ETR functions to be performed in new devices
      and/or existing routers.

  5 - Provides benefits for adoptors immediately - full portability,
      multihoming, inbound TE and ideally mobility benefits to each
      adoptor, irrespective of how many other networks have adopted
      it:

            ^
         B  |*********      Core-Edge Separation (CES)
         e  |*********
         n  |*********
         e  |*********
         f  |*********
         i  |*********
         t  |*********
            |*********
            |*********
            0--------->
              Effort ~= level of adoption

  6 - Provides routing scaling benefits in direct proportion to
      how widely it is adopted, by two mechanisms.  Firstly, more
      end-user networks get the portability, multihoming etc. they
      want/need.  Secondly, they get these benefits without
      significant load on the DFZ.  This will lead to some reduction
      in the number of PI prefixes in the DFZ as current PI-using
      networks adopt the CES architecture.  PI-using networks could
      either convert their PI prefixes to "edge" space and being able
      to use this space more flexibly, in finer slices - or they
      could relinquish their PI space and rent some "edge" space.
      Since "edge" space can be more finely divided, they probably
      don't need as much of it as with PI space, for which the
      finest chunk is /24.

            ^
         B  |        *      Core-Edge Separation (CES)
         e  |       **
         n  |      ***
         e  |     ****
         f  |    *****
         i  |   ******
         t  |  *******
            | ********
            |*********
            0--------->
              Effort ~= level of adoption

A CEE architecture (ILNP, GLI-split, Name Based Sockets, HIP etc.):

  1 - Changes the current naming arrangement so there are separate
      objects, with separate namespaces, for the Identifier and
      Locator roles.  I think most CEE approaches use 3 levels -
      FQDNs for the "Text names", and separate levels to perform
      the Identifier and Locator roles.  In some CEE architectures
      such as ILNP, the lower part of the IPv6 address performs
      the Identifier role and the upper part of performs the
      Locator role.

          Role             Level            Some CEE architectures

                                            such as HIP
          Text name  <---- FQDN
          Identifier <---- HIT
          Locator    <---- IPv6 address

          Role             Level            Some CEE architectures
                                            such as ILNP
          Text name  <---- FQDN
          Identifier <----           IIII IIII } Combined into
          Locator    <---- LLLL LLLL           } one IPv6 address

      I don't yet fully understand Name Based Sockets, but it has a
      naming structure something like this.  In practice, I think
      the Identifier role may be performed by a combination of
      Identifier and Locator, so a single text FQDN can be used
      with "round-robin DNS" to select one of several separate
      hosts, each of which must have a separate Identifier.

          Role             Level            Name Based Sockets?

          Text name  <---] FQDN
          Identifier <---]
          Locator    <---- IPv6 address

  2 - Scalable routing is achieved by applications no longer being
      dependent on stable IP addresses for session continuity.  This
      is because they use an Identifier to determine the host to which
      each communication session is established and maintained.  The
      CEE stack figures out which of potentially multiple Locators
      to use, and Locators can usually be added to and deleted from
      the set of allowable Locators for any one Identifier, as the
      session continues.

      Identifiers are portable.  Multihoming is achieved by the stack
      managing multiple Locators to continue the session if the
      end-user network becomes unreachable by the currently used
      Locator.  Inbound TE is achieved by which Locator is used in
      outgoing packets - since in general the other host will send
      packets back to this host using the Locators this host most
      recently used in outgoing packets.  Mobility is achieved by
      doing all this, and maintaining sessions, despite frequent,
      ad-hoc, changes of Locators.

      When (almost) all a network's hosts are having (almost) all
      their communications occurring with the new CEE arrangement,
      then portability, multihoming and ideally mobility can be
      provided just as well on PA prefixes as on PI.  At this point
      the routing scaling benefits finally occur - all networks can
      do portability multihoming etc.  Those which used to do these
      things via conventional PI space no longer need this PI space.

  3 - Before there can be substantial scaling benefits, all host
      stacks and all applications must be upgraded to use the CEE
      "Loc/ID separation" naming structure.

  4 - Requires no changes to internal of DFZ routers.  There is
      typically a need for a new mapping system by which hosts
      can look up an Identifier and receive the one or more Locators
      that host is using.  (Name Based Sockets uses the existing
      DNS.)

  5 - Provides portability, multihoming, inbound TE and ideally
      mobility benefits to each adoptor, in proportion to how
      many other networks have adopted it.  This is because the
      CEE host and its CEE applications can only provide these
      benefits when communicating with other CEE applications
      in other CEE hosts.  (A good CES system provides these
      benefits to packets sent by all hosts, including those
      in networks without the CES's ITRs.)

      For an adoptor (assuming their hosts communicate, in
      general, with a mix of hosts in all other networks), the
      proportion of communications with the benefits (portability,
      multihoming, TE and ideally mobility) increases linearly
      with the level of adoption.

      However, for most networks, I think it is fair to say that
      they only get real usable benefits when all, or nearly all,
      of their communications are using the new system.  Only then
      can their network be fully portable and multihomed without
      the need for PI space.

            ^
         B  |        *      Core-Edge Elimination (CEE)
         e  |       .*
         n  |      ..*      .  Proportion of communications
         e  |     ...*         with the benefits.
         f  |    ....*
         i  |   .....*      *  Actual total benefits generally
         t  |  ......*         only arise when all, or almost
            | .......*         all, communications have the
            |........*         benefits.
            0--------->
              Effort ~= level of adoption

  6 - Provides routing scaling benefits in two forms:

        A - When most or all networks which have PI space can
            abandon it and use PA space instead.  This is when
            (almost) all of their communications use the new
            system - which is when (almost) all other networks
            have fully upgraded all their host stacks and all
            their applications.

        B - When those without PI space have all, or almost all
            of their communications using the new system, for
            the same reasons as above.

      In both cases, the curve is:

            ^
         B  |        *      Core-Edge Elimination (CEE)
         e  |        *
         n  |        *
         e  |        *
         f  |        *
         i  |        *
         t  |        *
            |        *
            |        *
            0--------->
              Effort ~= level of adoption

CES and CEE are *completely* different ways of solving the routing
scaling problem.

I think everyone agrees that a good CES architecture is easier to
introduce on a voluntary basis than any CEE architecture, since the
adopting networks get full benefits immediately, don't need to alter
their host stacks, or require upgraded versions of their applications.

I think many people believe that the final state of the Internet with
 good CEE architecture is a better than could be achieved with the
best CES architecture.  I think this is because of two interdependent
beliefs:

  1 - The routing system should be kept simple, if there is any
      way hosts can be changed to achieve this.

  2 - The current 2 level IP naming structure - making the IP
      address perform both the Identifier and Locator roles -
      is architecturally flawed and should be fixed to make
      these independent levels, each with a different namespace.
      The practical benefits of this are largely that the
      routing system can be made to scale well, while remaining
      as simple as it is today.  The hosts themselves, with the
      aid of an Identifier -> Locator mapping system, will do
      all the portability, multihoming, TE and mobility work
      themselves.

I disagree with this viewpoint.

I think the routing system should serve the hosts - and that it will
be easier to upgrade the routing system to make it scalable with a
CES architecture, than to upgrade all hosts and all their
applications to implement CEE.

I think the current 2 level IP naming structure is better than the
Loc/ID Separation CEE approach, as explained in the companion message.

Here is my understanding of the arguments for why CEE is the best
long-term arrangement for the Internet:

  1 - Adding new network elements (as are required by CES) is
      inefficient, expensive and contrary to the principle of
      keeping the network simple and elegant, while implementing
      as much of the total functionality of the Net in the hosts.

      Host CPU power is inexpensive.  Host software can be upgraded
      without the need to buy new hardware or establish the
      commercial mechanisms for building and running the ITRs, ETRs
      and mapping system which is required for CES.

  2 - The current IP naming model is fundamentally broken, because
      it makes the one object - the one level of a 2 level system -
      the IP address, play the roles of both Identifier and Locator.

      This is architecturally wrong and it makes it difficult or
      impossible to create a routing system which support large
      numbers of end-user networks with portability, multihoming,
      inbound TE and mobility.

      The right way (as Tony would say) to do it is to create new
      protocols so all hosts communicate with the use of independent
      Identifiers and Locators.

Here are my arguments for why CES is superior to CEE:

  1 - While CES involves adding significant new things to the
      routing system, this is a lot easier than upgrading all
      host stacks rewriting all applications.

      CES early adopters get full benefits, no matter how many
      other networks adopt it.  So a CES is far easier to have
      adopted at all, and then to have fully adopted, than any
      CEE architecture.

      While a CES architecture needs a mapping system, so do most
      CEE architectures.  The CES mapping system is arguably more
      efficient and so can be less costly than that of a CEE
      mapping system because:

         a - The CES mapping system is only used by ITRs - and
             each ITR caches the mapping and will frequently
             use the same mapping to support the communications
             of multiple hosts.

         b - A CES mapping lookup is only required at the start
             of communications sessions where the destination
             address is in the "edge" subset.  Even if CES is
             adopted by all end-user networks which want or need
             portability, multihoming etc. there will still be
             a large number of hosts using PA space.  This includes
             almost all home, SOHO and small commercial customers
             who are doing fine, and will continue to be happy with,
             their one IPv4 (or later an IPv6 /64) of PI space.

             A CEE lookup is needed by both the initiating and
             responding hosts for almost all new communication
             sessions.  The only exceptions are where the
             responding host - such as a DNS server - doesn't care
             if the packet it sends goes to a host which has a
             different Identifier to that used in the request
             packet.

      CEE can't be useful for IPv4, since multihoming a /18
      network would require a /18 of PI space from one ISP and
      another /18 from another ISP.  So it would at least double
      address consumption.

      CEE can't work with hosts behind NAT.  Ideally NAT won't
      be used with IPv6, but this cannot be assured.

      There is another difficulty which I foresee, which I don't
      think has been discussed much - how a CEE copes with a
      host which has one or more Locators on global unicast
      space and one or more locators on one or more private
      address spaces.

      Global unicast Locators can be used to communicate with
      other global unicast Locators - but not private Locators.

      Two hosts with Locators in the same private network space
      can use them to communicate.  This would probably be
      preferred over using the global unicast Locators.  The
      CEE stack and the various protocols between hosts will
      need to cope with the various complications which arise
      from this.

      CEE hosts may need to inform a very large number of other
      hosts they are communicating with if they adopt a new
      Locator and stop using another.  This information also
      needs to be securely uploaded into the DNS and into
      the Identifier -> Locator mapping system.  This is a
      lot of work, particularly for hosts on slow links which
      are subject to congestion and packet loss due to radio
      difficulties.

      IPv6 addresses can have their lower 64 bits frequently
      changed in order to make it difficult for anyone outside
      the network to trace activity, such as to web servers, to
      individual computers - and so by implications to individual
      people.   However, to be able to do this with CEE would also
      involve each such host changing its Identifier as well.
      This could be done, but it involves the creation and deletion
      of state in the public global DNS and Identifier -> Locator
      mapping systems.  This would be onerous and may compromise
      the privacy benefits of the technique.  For CEE architectures
      in which the lower 64 bits of an IP address is the Identifier
      and where Identifiers must be globally unique, then this
      means the randomization of Identifiers can only be within
      a range restricted by whatever range of Identifier space the
      network has been allocated.

  2 - While CEE does involve a simpler network, this benefit is far
      to slight to make up for greater effort hosts must make when
      beginning new sessions.  Please see the companion message:

         Today's "IP addr. = ID = Loc" naming model should be
         retained

      for more details.  This is particularly the case for the
      future, in which most hosts will be running continually
      from small batteries and will generally rely on 3G or similar
      wireless links.

So while both CES and CEE architectures can both, in principle, solve
the routing scaling problem, they do so in completely different ways,
involve completely different adoption hurdles and result in
completely different host protocols in the final system.

> Firstly, many of the solutions and not purely one or the other.  As Noel
> has observed, one can migrate LISP (ostensibly a CES solution) into the
> host.

Whether or not this is done, LISP remains entirely a CES
architecture.  Just because a LISP MN can be its own ETR and ITR
doesn't mean that LISP in any way resembles a CEE system.  The ITR
and ETR functions in such a host would be simple additions to th the
stack.  The IP stack, its API and all applications would remain
unchanged.  The two-layer IP naming structure remains unchanged.

> Architecturally, and in terms of the end state, I think that CEE is a
> much better target.

I hope you will respond to my arguments to the contrary.

> In terms of deployability and related issues, CES techniques tend to
> have a big advantage.

Yes.

> Hence, I have a personal preference for solutions which straddle this
> boundary, 

What are these?  I don't know of any such solutions.

>         rather than seeing utility in arguing about which one we want.

There are huge differences between CEE and CES architectures.

> And I don't the the RG has a clear agreement on this at all.

Not yet.

  - Robin

---

From: Robin Whittle <[email-masked]>
Date: Tue, 02 Feb 2010 01:34:02 +1100

Short version:   The correct use of "Locator/Identifier Separation".
                 NAT is evil - and incompatible with CEE.
                 SIP will work fine with CES.

Hi patte,

In "Multiple critiques, choice of single proposal, consensus on
constraints due to voluntary adoption?" you wrote:

> Joel,
> 
> I throw some thoughts into the CES/CEE debate - just some observations
> that I think might be interesting to share with you
> 
> A CES approach is preserving the hosts and applications, no changes
> needed there, right?

This is correct.

> But this is at the same time the biggest evidence that a CES approach
> is not really a true Identifier/Locator separation architecture - 

Yes - Core-Edge Separation architectures do not involve "Locator /
Identifier separation".  Please see:

> a CES solution engineers a certain address space (PI) from one
> repository to another, that is, from the DFZ routers' RIB/FIB to a
> mapping database. 

Yes - a subset of the global unicast space is "separated" into
a subset known as "edge" space, just for end-user networks, which can
be used by them for portability, multihoming etc. in a highly
scalable fashion.  This involves the individual divisions of this
space not being handled by the DFZ routers' RIB/FIB - but by a new
system of ITRs and ETRs, where the ITR tunneling behaviour is
controlled by the destination "edge" address and the results this
brings from a mapping database: basically one or more ETR addresses.

> So why can't you then consider CES as a true
> Identifier/Locator separation architecture?

CES involves separation of a subset of the global unicast space into
the new "edge" subset, and with the remainder being known as "core"
space.

This is not at all separation of Identifiers from Locators.  This is
a common error of thought - a mistake I have made in the past too.
The most likely cause of it is the mistaken choice of name for LISP.
 Maybe the LISP designers thought that by designing what is now known
as a Core-Edge Separation scheme (like LISP, APT, Ivip etc.) they
were really going to achieve the same things in something like the
same way as true Locator / Identifier Separation architectures such
as HIP, which predate LISP's late 2006 origins.

Just because a Core-Edge Elimination scheme such as HIP uses Locator
/ Identifier Separation to achieve scalable routing goals doesn't
mean that LISP, which uses completely different techniques (Core-Edge
Separation) to also achieve scalable routing, is also a Locator /
Identifier Separation architecture.

Other people have mentioned in the past that LISP is a misnomer.  We
need to recognise this very clearly, otherwise it will be impossible
to use the perfectly valid term "Locator / Identifier Separation"
without causing a lot of confusion.  It refers to Core-Edge
Elimination - not to Core-Edge Separation.

These CEE and CES terms are more recent - the last two years or so, I
think.   I think the most important document which uses these terms is:

    Towards a Future Internet Architecture: Arguments for Separating
    Edges from Transit Core
    Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang
    http://conferences.sigcomm.org/hotnets/2008/papers/18.pdf

This argues the superiority of CES over CEE.  Please also see a
message I just wrote:

   CES & CEE are completely different (graphs)
   http://www.ietf.org/mail-archive/web/rrg/current/msg05865.html

> We know that there are overloaded applications, which are carrying
> Identifier&Locator information in there headers&payload. You can
> detect them by forcing them to traverse a NAT device - two most common
> applications that I can think of is IPsec and SIP. These protocols
> will work nicely in a CES approach but in a CEE where a true
> Identifier/Locator separation is applied you get into trouble. 

You correctly state that CEE is Locator/Identifier separation.

As far as I know, CEE won't work with NAT.  That is not necessarily a
bad thing with IPv6, since it has long been hoped that NAT would not
infest IPv6.

> It is
> also a well known fact that architecturally IPsec and SIP are broken,
> both are relying on IP addresses 

I think the current 2 layer IP naming model is *better* than the more
"architectural correct" arrangement of having independent levels of
separate objects, in separate namespaces, for Identifier and Locator.

My recent message

  Today's "IP addr. = ID = Loc" naming model should be retained
  http://www.ietf.org/mail-archive/web/rrg/current/msg05864.html

will be heresy to many RRG folks.

As far as I know, IPsec and SIP will work fine with CES.

If the CES scheme is providing multihoming, and being extended with
the TTR Mobility architecture to provide global mobility (each MN
keeping its one or more IPv4 addresses or IPv6 /64s, no matter where
it connects, including behind NAT) then SIP should work just as well
as it does without CES.  It may not accord with the "architecturally
pure" vision of all hosts having independent Identifiers and
Locators, but that doesn't mean it is "broken".  It works today, and
it will keep working with a CES solution to the routing scaling problem.

One potential advantage of TTR Mobility is that a MN can have one or
more addresses behind NAT, but can still use its one or more IPv4
addresses or IPv6 /64s of "edge" address space: global unicast space,
known as SPI (Scalable PI) in Ivip, or EID space in LISP.  This
constitutes an automatic, globally portable, form of NAT traversal
which will operates fine through any kind of NAT, or multiple levels
of NAT, because it involves the MN making a two-way tunnel out to one
or more TTRs.  Those tunnels always traverse all kinds of NAT to any
depth.

> - so actually it is not the CEE's
> fault that you get into trouble with these kind of applications. And a
> new architecture comes with a cost, not everything can be preserved as
> such - if everything is preserved, then it can't be a new
> architecture.

Yes, but the only benefit of CEE I know of, apart from architectural
purity, is that it enables multihoming and perhaps mobility on a
scalable basis without the need to upgrade the routing system at all.

This is impressive and interesting, but it comes at a cost, as I
argue in my recent message about how CES is very different from CEE:
more delays in establishing most communication sessions, more work
for hosts, more packets going back and forth between hosts, more
lookups by hosts into slow, less then 100% reliable global query
server systems (such as DNS) - and all this is made worse still for
any host which is on a wireless link with congestion, delay, dropped
packets etc.

Then there is the fact that CEE only delivers substantial benefits to
adopters and to scalable routing after everyone has adopted it in
their hosts and after essentially all applications have been
rewritten to use the new CEE API in the stack.

This is never going to happen.  The Internet is not so broken that we
have to rewrite the apps.

The current IP address = Identifier = Locator arrangement works and
has major advantages in terms of being faster and lighter-weight for
all hosts.  Scalable routing and global mobility can be done better
and more easily with CES than CEE.  CES preserves the current IP
address model and does not require us to alter host stacks or rewrite
any applications.

> So is NAT really that evil as its reputation? 

I think NAT is very evil.  The any-to-any connectivity model of the
Internet is an excellent thing.  I think its a tragedy (though
understandable) that IPv4's address length is 32 bits rather than 40
or 48.  48 would be heaps.  So would 64.  Given that it was 32 bits,
its a major blunder not to have a way out of this via an API which
returned to all apps the current length of the addresses used in the
Internet.  If apps could have been ready for 64 bits, and have been
made to work with 32 for a while, then a transition from IPv4 to
something with more bits would be much easier.

> NAT has forced the
> application people to decouple the location and identifier - from
> application's point of view, you can't really rely on the IP address
> since it might be changed in a middlebox. And this is good,
> applications that works well through a NAT middlebox shouldn't have
> trouble with a CEE architecture.

If we can ever move to IPv6 or something else, NAT would be the
number one thing I hope never comes with us.

NAT itself is not standardised and the NAT traversal protocols are
consequently very messy and may not work, depending on the flavour of
NAT.

>From a very broad, excessively reductionist "architectural"
perspective, you might look at a bunch of developers who have
succeeded in making their code work when running on hosts behind a
variety of NAT boxes, despite the fundamental problems of these hosts
not being reachable from the global Internet - and say "Oh good -
these apps are so well written they work no matter where the hosts
are."  But these efforts usually rely on the use of servers, such as
STUN servers and other contrivances which would never be needed if
every host was on a global unicast address.

But this capacity to cope with NAT usually (always?) depends on these
ugly servers and ugly NAT traversal techniques.  This is not at all
the same as what a CEE architecture does.  It is the result of an
heroic effort to overcome a pestilence which should never have been
introduced - but it had to be introduced because the limited number
of IPv4 addresses did not enable ISPs to dish out 16 or whatever IPv4
spaces to every DSL, HFC cable or whatever home, SOHO and small
company customer.

 - Robin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 01 Feb 2010 09:39:30 -0500

First note that I am not at all sure that the terms CES and CEE are even 
useful.  I will use them to respond to you, but since they largely 
represent specific points in engineering tradeoffs, they actually make 
the discussion harder.

 From a deployment perspective, it is very useful if one can leave the 
hosts, and the site routing, alone.  Therefore, many of the proposed 
solutions have either an end state or a transition state that leaves 
those items unchanged.

One of the reasons why I have not tried to write text about the value of 
ID / Locator separation is that even within this limited community, 
there have been significant differences is when the term "identifier" 
should be used.  In my book, an IEEE MAC address is an identifier.  it 
remains an identifier even if one is using a bridged network that is 
forwarding packets based on that identifier.  Other people have 
different books, i.e. they use the terms slightly differently.

Thus, for me, the question of whether a given bit string is an 
identifier has to do with whether its assignment and usage is location 
insensitive, rather than whether some forwarding system could use it. 
(To take an extreme example, if compact routing for Internet traffic 
worked, we could just use identifiers with no locators, and still have a 
scalable system.  The "locators" in that system are the landmarks used 
for tunneling the traffic.)

As a minor point, SIP is probably a bad example to use in analyzing 
these issues.  SIP can be used in multiple ways.  I believe you are 
actually referring to the SDP message, which is frequently carried by 
SIP, and which frequently carries an IP address.  Note that if we had 
resolvable identifiers which worked through NATs, and which had the same 
syntax as IP addresses, those could be carried in SDP, possibly 
improving the situation significantly.

the quesiton of what layer the identifiers are at is one that again 
causes a significant difference of opinion.  And also causes 
complications.  In a true MP-TCP environment, one might well want to 
view the identifier as being above the TCP flows.  On the other hand, in 
a regular TCP environment, placing the identifier below TCP so that TCP 
picks up a significant degree of locator invariance without needing 
MP-TCP is also valuable.  Hence, in terms of the abstract question of 
whether some stable identifier has value, I do not want to get tied up 
in the question of which layer should hold the identifier.

Yours,
Joel M. Halpern

Patrick Frejborg wrote:
> is not really a true Identifier/Locator separation architecture - a
> CES solution engineers a certain address space (PI) from one
> mapping database. So why can't you then consider CES as a true
> Identifier/Locator separation is applied you get into trouble. It is
> both are relying on IP addresses - so actually it is not the CEE's
> So is NAT really that evil as its reputation? NAT has forced the
> About MPTCP, I think an identifier shouldn't be placed at all in the
> routing space, i.e. in the IP header, if you really obey the
> terminology that has been defined at:
> http://trac.tools.ietf.org/group/irtf/trac/wiki/RRGTerminology
> Thus the identifier should be placed in the transport or application
> layer so the session can continue regardless of what is happening in
> the routing (locator) arena. This will open up the path towards host
> mobility that are not that heavily depending on a network
> architecture.
> 
> This is also not about CES against CEE - we need a set of tools
> because what will get adapted by the end users is unknown, but it has
> to be cheap and it has to offer some new services or a path towards
> new services
> 
> -- patte
> 
> On Sat, Jan 30, 2010 at 8:59 PM, Joel M. Halpern <[email-masked]> wrote:
>> If someone wants to add text on Identifier / Locator separation as a useful
>> architectural principle, I guess I can't object.  Although we have not
>> actually done a very good job of articulating why it matters in conjunction
>> with things like Multipath TCP.  (I think it does still matter.  My point is
>> that we do not have a good description of why.)
>>
>> The question of IPv4 support is an example of something the RG has not
>> converged on.
>>
>> Firstly, many of the solutions and not purely one or the other.  As Noel has
>> observed, one can migrate LISP (ostensibly a CES solution) into the host.
>> Architecturally, and in terms of the end state, I think that CEE is a much
>> better target.
>> In terms of deployability and related issues, CES techniques tend to have a
>> big advantage.
>> boundary, rather than seeing utility in arguing about which one we want.
>> Yours,
>> Joel
>> Noel Chiappa wrote:
>>>    > From: "Joel M. Halpern" <[email-masked]>
>>>
>>>    > I can not foresee any process which would come to an actual agreement
>>>    > on a recommendation from the RRG, and therefore conclude that the
>>>    > survey is probably the most effective outcome we can achieve as a
>>>    > community.
>>>
>>> While I agree there's probably not agreement of a specific technical
>>> proposal, what about architectural direction(s)? E.g. I thought we did
>>> have
>>> rough consensus on the need for the separation of location and identity?
>>>
>>> True, the workshop a couple of years back (Amsterdam?) I think already
>>> recommended this, so we're not breaking a lot of new ground there; still,
>>> it
>>> would be useful to re-affirm that conclusion, in a larger group.
>>>
>>> And how about IPv4 support? Is a solution which doesn't support IPv4
>>> judged to
>>> be plausible? Do most people agree with that one?
>>>
>>> Also, can we say anything about CEE/CES? I think a lot of people think
>>> that
>>> CES is sort of necessary for the short term, because any solution will
>>> never
>>> really get deployed otherwise. At the same time, there appear to be
>>> reasons
>>> that a solution has to be able to migrate in the CEE direction in the long
>>> term. I don't know how many agree with these, but it might be worth
>>> exploring
>>> that.
>>>
>>> Any others?
>>>
>>>        Noel
>>>
>> _______________________________________________
>> rrg mailing list
>> [email-masked]
>> http://www.irtf.org/mailman/listinfo/rrg
>>
>

---

From: Scott Brim <[email-masked]>
Date: Mon, 01 Feb 2010 10:38:26 -0500

Noel Chiappa allegedly wrote on 01/30/2010 13:40 EST:
> proposal, what about architectural direction(s)? E.g. I thought we did have

Yes and I think we've distilled "locator/identifier separation" down to:
To the extent possible, identification functions should stop depending
on location-dependent inputs.  However, (1) we are still unresolved
about what network-layer support is needed for identification -- for
example even if we need whole node identifiers, stack names, etc. -- and
(2) we don't have specifics about what to do about "legacy" identification.

> Also, can we say anything about CEE/CES? I think a lot of people think that
> CES is sort of necessary for the short term, because any solution will never
> really get deployed otherwise. At the same time, there appear to be reasons
> term. I don't know how many agree with these, but it might be worth exploring
> that.

I don't think it's a good idea to declare either approach the long term
winner, since we can't know what the future holds and even if one wins
there will be new approaches by then.  A CES approach may be able to
migrate to CEE in the long term, but also vice versa.  Ideally we should
document how they interwork and make deployment of both easy -- whether
both get deployed or not, that's architecturally a good idea.

Joel M. Halpern allegedly wrote on 01/30/2010 13:59 EST:
> useful architectural principle, I guess I can't object.  Although we
> have not actually done a very good job of articulating why it matters in
> conjunction with things like Multipath TCP.  (I think it does still
> matter.  My point is that we do not have a good description of why.)

I am glad to _help_ but I hesitate to commit to writing it up all by
myself by the deadline (no time!).  If you, for example, want to take
responsibility for the text, we can have some good phone calls.

Scott

---

From: Scott Brim <[email-masked]>
Date: Mon, 01 Feb 2010 10:57:39 -0500

Patrick Frejborg allegedly wrote on 02/01/2010 02:47 EST:
> is not really a true Identifier/Locator separation architecture

But that's okay.  The goals of

- robust scalable routing and addressing, and

- weaning endpoint identification functions from location-dependent
  data

are complementary but you do not have to satisfy them with a single
piece of technology.  In fact, if you decouple the solutions the whole
system becomes more flexible.  It would be fine if a CES approach takes
care of scaling the networks and something else takes care of loc/id
separation in the endpoints.

> to be cheap and it has to offer some new services or a path towards

Sounds good.

> for tunneling the traffic.)

Note "assignment and usage".  Exactly.  A bit field itself may be
intended to be used in a certain way, but what matters is how it is
actually used.  Any bit field can be used by a location function or an
identification function or both.  The problem is when usage doesn't
correlate with assignment, and an identification function uses fields
that are location-dependent.

> in the question of which layer should hold the identifier.

Yes.  That's a separate question from building a robust scalable routing
and addressing system.  In RRG, let's _assume_ that the endpoints work
out their own LIS problems.  Given that assumption (but also
extrapolating what the endpoints will want to do, now that they have
better architected stacks), how would you build routing?

Scott

---

From: Patrick Frejborg <[email-masked]>
Date: Mon, 1 Feb 2010 21:55:55 +0200

Hi Robin,

comments in-line

>    http://conferences.sigcomm.org/hotnets/2008/papers/18.pdf

Great paper, now I understand better the CEE&CES terms

You will find it hard to categorize hIPv4, it is partly CES because it
is splitting up the address space in core and edge addresses - but
without any new mapping databases and the change should be applied at
the hosts - not a map&encap solution.  Also partly CEE, in order to
achieve dynamic multi-homing a multi-path transport protocol should be
used. So hIPv4 is an odd bird...an unorthodox approach but in research
work that should be allowed...

> bad thing with IPv6, since it has long been hoped that NAT would not

NAT is evil, no doubt about that. But if we wouldn't have had NAT most
likely the application people would have written more applications
that are relying directly on IP-addresses and a migration of
application to e.g. IPv6 would be much harder. So it is bad but there
is something good also.

> will be heresy to many RRG folks.

Here I get confused again about the usage of the identifier term. If
you split up the addresses into two parts as in CES you will have an
IP address block that is visible in the DFZ (in LISP that is RLOC) so
why not give it the name e.g. Global Locator. Then the edge IP address
space is only visible inside the local network (in LISP that is EID)
so why not give it the name e.g. Local Locator - both are used for
routing purposes anyway and resides in the IP header. But let us not
start an argument about this, it is like it is - but it is very
confusing for beginners (like me) on the list.

> have to rewrite the apps.

True, but NAT has done us a favor here. The application which works
well over NAT shouldn't care what is happening on the network layer
and most of the applications working over Internet shouldn't be a big
issue. But then when you have a look on what is used inside an
enterprises' Intranet it is a different story - there is quite a lot
of applications that has been tailored for businesses and if you
change that it will be expensive, very expensive

> address model and does not require us to alter host stacks or rewrite

The current address model is soon running out of addresses, what then?
When you are there you have to go after the host stack and some
applications needs to be rewritten anyway. Sooner or later you need to
deal with this problem, now there is a chance to do something more
intelligent with IPv6, before it is getting more deployed.

> Internet is an excellent thing.

It is evil, no doubt - but it is today part of the package.

-- patte

---

From: Robin Whittle <[email-masked]>
Date: Tue, 02 Feb 2010 11:41:49 +1100

Short version:     The only applications I can think of which work OK
                   behind NAT and which would be easy to use with
                   IPv6 would be client-server applications, in which
                   the client may be behind NAT and the server cannot
                   be.  These are not the most challenging types
                   of applications to make work from behind NAT.

Hi patte,

> application to e.g. IPv6 would be much harder. So it is bad but there

I can't think why an IPv4 application which has been made to work
from behind NAT, I guess with specific messy protocols and external
servers to trick the NAT box into allowing various things to happen,
would be easier to modify for IPv6 operation.  As far as I know,
making an application able to operate from behind NAT isn't some
special generalised independence from how each host is connected or
where, which would somehow ease rewriting the code for IPv6.

This is pretty generalised discussion.  Can you think of some
examples?  Below I consider client-server applications where the
client may be behind NAT and does all its work with a server which is
not behind NAT, using HTTP - which could be over IPv4 or IPv6.
However, other applications need communication between two or more
hosts which may be behind NAT, ideally without a server.

> issue. 

I am not sure this is true - except to the extent that if the
application simply relies on an underlying protocol, such as TCP or
HTTP, to connect to a server which is not behind NAT.  It is easy to
do client-server applications where the client is behind NAT and the
server isn't.  If the application uses HTTP as its basis, and always
tells the HTTP layer to open a session to a given URL, then I agree,
the application has nothing to do with IPv4 or IPv6, and will work
fine no matter how the HTTP system makes the session to the specified
server - assuming the URL somehow gives the HTTP layer access to its
choice of IPv4 and IPv6 server.

But this is only one subset of applications.  The real trick with NAT
is to make it possible for the behind-NAT host to receive a packet
from some other host (which itself may be behind NAT) when this host
has not previously opened up a session to the other host.  A further
trick is to make this work if the unfortunate host is behind two or
more layers of NAT.

I don't see how such necessarily messy arrangements for outsmarting a
variety of types of NAT (and the host can't easily tell which kind of
NAT it is behind, or how many levels) necessarily means that the
application can be made to work on IPv6 without much fuss.

> But then when you have a look on what is used inside an

There's no way all these applications are going to be rewritten for
the purpose of supporting scalable routing.  They might be rewritten
if there is ever sufficient impetus to make them work with IPv6.

CES provides scalable routing much better and with more immediate
benefits to adoptors and routing scalability than CEE.  Only CEE
requires rewriting applications.

Yes.

  - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Tue, 2 Feb 2010 10:33:25 +0200

On Tue, Feb 2, 2010 at 2:41 AM, Robin Whittle <[email-masked]> wrote:
> examples?

One example is HTTP and the usage of cookies. If you have a lot of
clients (thousands) behind a forwarding proxy then how to do server
load-balancing in front your servers? You can't use the source address
to identify a unique client, every client behind the proxy is having
the same source address in their connections to the server. So the ADC
in front of your server usually identifies the clients by the help of
cookies, and direct the client always to the correct "real server". So
when the connection lands at the real server the source address has
been changed by the forwarding proxy and the destination address has
been changed by the ADC (if the ADC operates in L3 mode) - NAT
everywhere and far from easy to troubleshoot
http://en.wikipedia.org/wiki/HTTP_cookie
And you can try it yourself, take a session to  a web site, shut down
your interface and use another interface to continue on the same site.
Most likely the site will continue to serve you from where you left
off.
A cookie is an identifier that is decoupled from the address space and
network layer, the more these kind of applications there is deployed
the easier it would be to change the network and address architecture.
I think that the SCTP verification tag and MP-TCP token has similar
characteristics, and they are more of general nature and not so much
tied to a certain application. Both the verification tag and token are
32-bit, if these transport protocols becomes more used the middleboxes
could start to track sessions with the help of the 32-bit "session
identifier". MP-TCP is for TCP only, UDP should be abandoned and SCTP
shuold be used instead - but SCTP is plagued by the evil NAT and thus
there is a catch-22 situation.

This is an architectural discussion/speculation and there is a lot of
hurdles to get us there, maybe impossible to achieve...

-- patte

---

From: Robin Whittle <[email-masked]>
Date: Tue, 02 Feb 2010 23:41:05 +1100

Hi patte,

> http://en.wikipedia.org/wiki/HTTP_cookie

OK - this is just a client-server application, and since the client
can communicate perfectly well with the server, as long as it opens
the session from behind NAT, then NAT makes no difference.  Since the
whole thing happens at the HTTP level, IPv4 or IPv6 is not an issue
either.

But applications which attempt to do things which would be easy and
natural without NAT, such as sending video, audio or whatever from
one host to another, are difficult or impossible with hosts behind
one or more layers of NAT.

> the easier it would be to change the network and address architecture.

Yes, but this is just client-server applications.

> hurdles to get us there, maybe impossible to achieve...

I think SCTP is an interesting and promising protocol, but I don't
know much about it.  I will probably steer well clear of the
multipath TCP debates.  I understand it requires changes to some or
all DFZ routers.  If we can change all DFZ routers, then there's a
lot more I would be doing than MP-TCP!

  - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Tue, 2 Feb 2010 18:08:25 +0200

> Yes, but this is just client-server applications.
Very true, it is just one application.
But it is causing about 52% of the Internet traffic and increasing
Have a look on slide 22 at
http://www.nanog.org/meetings/nanog47/presentations/Monday/Labovitz_ObserveReport_N47_Mon.pdf
But once you get into enterprises the traffic profile is different

-- patte

---

From: Lixia Zhang <[email-masked]>
Date: Tue, 2 Feb 2010 09:10:53 -0800

On Jan 30, 2010, at 10:40 AM, Noel Chiappa wrote:

> recommended this, so we're not breaking a lot of new ground there;  
> still, it

(I'm sweeping through all RRG exchanges trying to collect the arch  
related issues)  I saw Tony already confirmed this point earlier: yes  
we have a rough consensus on documenting the need for separating  
location and identity.

However I also see gap/ambiguity still exists on concept and  
terminology. devils always in details. I believe when Noel mentioned  
"identity" he probably had in mind some entity identity (there has  
been debate on whether host identity or stack identity), not interface  
identity (that'd move us back to address)

> be plausible? Do most people agree with that one?

(personal opinion) pragmatically we need solutions to keep today's IP4  
world going. IP6 rollout is likely to be a very loooong process.

> term. I don't know how many agree with these, but it might be worth  

(again personal opinion) to me it's a markable step to classify  
various proposals into CEE/CES. It may all look obvious now, but two  
years back we did not have such a clear view.
As Joel pointed out, CEE/CES is not architecture per se by itself.  
However in our report writing, I see CEE/CES useful as it can help  
sort out different types of solutions, and we can explain the  
different design tradeoffs made by each type.

Lixia

---

From: Scott Brim <[email-masked]>
Date: Tue, 02 Feb 2010 13:13:51 -0500

Lixia Zhang allegedly wrote on 02/02/2010 12:10 EST:
> (again personal opinion) to me it's a markable step to classify various
> proposals into CEE/CES. It may all look obvious now, but two years back
> we did not have such a clear view.
> However in our report writing, I see CEE/CES useful as it can help sort
> out different types of solutions, and we can explain the different
> design tradeoffs made by each type.
> 
> Lixia

There are a number of criteria by which we can categorize architectural
approaches.  Many are not binary yes|no but rather dimensions -- lines
between two extreme cases, with a particular approach lying somewhere
along that line.  If you're going to categorize different approaches you
have to start somewhere, and CES/CEE is as good a criterion as any as
long as you don't treat it as alpha and omega.  It's just one criterion,
you have to consider others as well, and it's not perfectly binary.