Subject: [rrg] A Revised critique for LISP

From: Lixia Zhang <[email-masked]>
Date: Sat, 13 Feb 2010 14:39:57 -0800

After a number of exchanges with Robin and Noel, here is a revision of  
LISP critique (sorry Noel I'm one day late -- it took much longer than  
I thought to put things together and I may have missed some worthwhile  
point due to word limit)

This is largely a revision from Noel's critiques, combined with  
comments from Robin and myself.  The focus is on LISP as a routing  
scalability solution; I did not include any comments on the mobility  
support part

Comments welcome!

Lixia
-----------

LISP Critique

LISP continues to go through changes based on lessons learned in  
actual deployment. This critique is based on the (understanding of)  
description at the time of this writing. This critique includes issues  
from fundamental architectural limitations, potential problems that  
require co-ordinated change, and new issues as the results of the  
design; in addition it also includes a clarification of some basic  
definitions.

First, two basic terms in LISP needs clarification: as state in LISP  
drafts:
    "LISP specifies an architecture and mechanism for replacing the
    addresses currently used by IP with two separate name spaces:
    Endpoint IDS (EIDs), used within sites, and Routing Locators  
(RLOCs),
    used on the transit networks that make up the Internet
    infrastructure."
Thus "EID" in LISP is not a host identifier, but IP addresses used  
within a site for packet delivery. Furthermore, an RLOC is not simply  
any IP address reachable in the global default free zone; it is an  
special address that binds to an attachment point of a site.

Regarding the architecture, LISP's most serious challenges are due to  
the fact that it effectively divides today's routed IP address space  
into two, edges and the core, which comes with all the challenges that  
such a grand division brings; the list below attempts to capture the  
major ones that have been identified (not in priority order).

The first question is whether, or how, one can draw a clear boundary  
to sort existing networks into the core and edges. For example where  
should one put those transit networks that do not provide global  
connectivity (e.g. Internet2)? Do such networks belong to the core or  
edges?  Does the core represent a connected cloud (bar transient  
failures)?

The second class of challenges arises from the fact that the  
reachability to each edge destination is now a combined result of 3  
major components: the mapping database that captures connectivity  
between an edge site and its TRs, realtime status between an edge site  
and its TRs, and the connectivity between ITR and ETR to encapsulate  
packets over. In designing these components, three goals are often in  
conflict: minimizing overhead, minimizing complexity, and maximizing  
performance.

Because the mapping database will be very large in size, LISP lets  
ITRs query mapping on demand, which brings up the question of how to  
handle packets while ITR waiting for the mapping information. The  
current decision (dropping packets) favors simplicity at the cost of  
data performance.  Would be feasible to buffer the packets? How deep  
such a buffer could be? Such questions need future research.

Another issue arises from caching the mapping information: caching  
improves the performance, but introduces the problem of detecting, and  
replacing, outdated mappings. This is a very lengthy topic with many  
subtly different failure modes, which cannot be covered here in any  
detail.

Because of this caching effect, and the fact that the ETR to a  
multihomed destination site is chosen at ITR, LISP design also faces  
challenges of response to component failures. LISP cannot easily test  
reachability of ultimate destinations (e.g. behind an ETR).

Regarding the mapping system, the ALT design has potential performance  
and scaling issues (e.g. concentration of request load at the top- 
level nodes); an interface has been built to allow replacement of the  
mapping system.  Another issue is potential identification (EIDs)  
namespace provider lock-in, unless some mechanism can be worked out to  
allow multiple competing providers to provide resolutions from EIDs to  
ETRs (perhaps as part of a new mapping system).  This last point  
touches on an even more important issue: an ISP's performance  
critically depends on the performance of the mapping system, a single  
mapping system to serve all seems problematic.

Another class of issues relates to network management and operations.
- Although LISP does provide significant tools for multi-homing,
   load-sharing, optimal-entry-selection, etc, these currently depend  
on correct
   configuration; response to component failures is also limited.
- LISP is currently working through NAT boxes, but only in limited
   configurations. In particular, due to the use of fixed UDP ports,  
it is not
   currently possible to support more than one ETR behind a NAT box.
- Encapsulations of data packets increase the packet size and may lead  
to PMTU problems.

Last but not least: One would not be able to see global routing table  
size reduction unless/until LISP has been adopted by significant  
number of networks. On the other hand, LISP is potentially a useful  
tool in data centers where its one-level of indirection may help  
significantly simplify the support for virtual servers.

---

From: Eliot Lear <[email-masked]>
Date: Sun, 14 Feb 2010 00:08:11 +0100

Lixia,

I have a problem with the below text:

> reachability of ultimate destinations (e.g. behind an ETR).

As a critique it suffers from lack of specificity.  To say that LISP (or
any of the other proposals, for that matter) "cannot easily" do
something, absent in this case an explanation of what the 'caching
effect' is, for lack of better words, pot shots.  What is one supposed
to do with this "information"?  No citations, no nothing.  I do not mean
to defend LISP here, but simply point out that I remain deeply concerned
about the format and content of this report.  Maybe I haven't been
paying attention for a while, but weren't reachability bits supposed to
handle a significant number of these cases?

>   configuration; response to component failures is also limited.

Two problems with the above statement.  The term "limited" suffers the
exact same problem as my previous complaint.  Second, network management
always lags underlying functionality.  Unless you are saying that there
is some fundamental problem that *prevents* network management, this
would seem to be an unfair criticism.

>   configurations. In particular, due to the use of fixed UDP ports, it
> is not

I've not been following LISP-NAT well enough to know if the above
statement holds true, but I will say that this is the sort of specific
statement that can at least be tested, challenged, agreed to, etc.

Eliot

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 13 Feb 2010 15:35:21 -0800

>>  LISP is currently working through NAT boxes, but only in limited
> statement holds true, but I will say that this is the sort of  
> specific statement that can at least be tested, challenged, agreed  
> to, etc.

FYI, LISP-NAT and LISP NAT-traversal are two different features. I  
want to clear that up. LISP-NAT is a way a doing non-LISP site to LISP- 
site interworking by translated a non-routable address (an EID) so  
packets can return to a LISP site from a non-LISP site.

LISP NAT-traversal is used when a LISP router is behind a traditional  
NAT box or is co-located with a traditional NAT box. The use-case here  
is when the LISP-site uses private addresses.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 13 Feb 2010 15:50:59 -0800

> Comments welcome!

Comments inline.

> description at the time of this writing. This critique includes

LISP is going through engineering changes as a reflection of  
deployment. There really aren't any architectural changes happening.

> issues from fundamental architectural limitations, potential  
> problems that require co-ordinated change, and new issues as the  
> results of the design; in addition it also includes a clarification  
> of some basic definitions.
>
> within a site for packet delivery. Furthermore, an RLOC is not  
> simply any IP address reachable in the global default free zone; it  
> is an special address that binds to an attachment point of a site.

A LISP EID is a host identifier. It is also a locally scoped locator  
that does not require global routing knowledge from core routers.

> Regarding the architecture, LISP's most serious challenges are due  
> to the fact that it effectively divides today's routed IP address  
> space into two, edges and the core, which comes with all the  
> challenges that such a grand division brings; the list below  
> attempts to capture the major ones that have been identified (not in  
> priority order).
>
> connectivity (e.g. Internet2)? Do such networks belong to the core  
> or edges?  Does the core represent a connected cloud (bar transient  
> failures)?
>
> between an edge site and its TRs, realtime status between an edge  
> site and its TRs, and the connectivity between ITR and ETR to  
> encapsulate packets over. In designing these components, three goals  
> are often in conflict: minimizing overhead, minimizing complexity,  
> and maximizing performance.
>

An implementation *could* queue packets. This has the same  
architectural tradeoffs that ARP and IPv6 Neighbor Discovery have. And  
the architecture says data packets could be used as Map-Requests.

> improves the performance, but introduces the problem of detecting,  
> and replacing, outdated mappings. This is a very lengthy topic with  
> many subtly different failure modes, which cannot be covered here in  
> any detail.

But you make is sound that LISP doesn't address this. There are  
numerous solutions to this and several are implemented.

> challenges of response to component failures. LISP cannot easily  
> test reachability of ultimate destinations (e.g. behind an ETR).

Well it could. It could ping EIDs. Just like a BGP router could ping  
an endpoint host. But it doesn't because it would not scale. It  
doesn't mean that because a BGP route is in a routing table that the  
destination is reachable.

So please compare apples and apples. That means compare what the new  
LISP idea challenges are with the same challenges existing routing  
architectures have.

> Regarding the mapping system, the ALT design has potential  
> performance and scaling issues (e.g. concentration of request load  
> at the top-level nodes); an interface has been built to allow

If you say ALT doesn't scale, then you must say the BGP based Internet  
doesn't scale. Are you willing to say that?

> replacement of the mapping system.  Another issue is potential  
> identification (EIDs) namespace provider lock-in, unless some  
> mechanism can be worked out to allow multiple competing providers to  
> provide resolutions from EIDs to ETRs (perhaps as part of a new  
> mapping system).  This last point touches on an even more important  
> issue: an ISP's performance critically depends on the performance of  
> the mapping system, a single mapping system to serve all seems  
> problematic.

There are no address providers. It is a one time sale. You buy an  
address block from a registry and the relationship is over.

If you want to bind it with locators and buy service from a Mapping  
Service Provider (MSP), then you have choices because they all attach  
at the ALT at the same aggregation layer.

We are modeling this in our 3rd generation of the LISP test network.

The only relationship a LISP site has with a service provider is the  
link that attaches to it and the  single address the SP assigns to the  
CE side of the link. Once you disable the contract with the SP, the  
link goes away and the address does. The EID-prefixs and other  
locators do not change. The mapping entry for the site gets updated  
dynamically in the Map-Server as well as existing caches and  
communication continues. In fact, TCP connections stay up during this  
change.

>  configuration; response to component failures is also limited.

Can you sight examples where this is true. Just making a claim doesn't  
mean it is true.

>  currently possible to support more than one ETR behind a NAT box.

This is easily fixed with a protocol change and not an architectural  
change. And this is only the case when the NAT and LISP are not co- 
located in the same box(es).

> - Encapsulations of data packets increase the packet size and may  
> lead to PMTU problems.

Just like existing tunneling solutions that seem to work and generate  
revenue in practice.

> Last but not least: One would not be able to see global routing  
> table size reduction unless/until LISP has been adopted by  
> significant number of networks. On the other hand, LISP is  
> potentially a useful tool in data centers where its one-level of  
> indirection may help significantly simplify the support for virtual  
> servers.

This is simply not true. The more specifics from site advertisements  
are withdrawn at the expense of coarse prefixes being injected by  
PITRs. The difference in the number of 1 or 2 orders of magnitude.

And why is there two points in the same paragraph? The data center  
comment came out of no where.

Dino

>
> _______________________________________________
> rrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/rrg

---

From: Robin Whittle <[email-masked]>
Date: Sun, 14 Feb 2010 15:20:52 +1100

Hi Lixia,

My position regarding critiques is that it is unrealistic of you and
Tony to combine or rewrite parts of the texts of multiple critiques and
expect the authors of those critiques to be happy with the result.  If
you didn't have a 500 word limit, then two or more critiques could be
included verbatim - which would be the best approach, I think.

Your rewritten critique:

  http://www.ietf.org/mail-archive/web/rrg/current/msg06032.html

has 745 words.  If you would allow another 255 words, then you could
have both my critique and Noel's, as cohesive, independent pieces of
text - and keep me and I think Noel happy.

Your new critique does not cover all the things I mention in mine:

  http://www.ietf.org/mail-archive/web/rrg/current/msg05728.html

If you only want a single critique, then I would still prefer mine to be
in the RRG Report than either Noel's or your rewrite of Noel's - but
what I really want is Noel's and mine to be in the report, verbatim, on
equal terms.

I don't think it would be a good use of time to debate most of the text
you have created - though below I concur with Dino's concerns about the
last paragraph.

In an ID:

  http://tools.ietf.org/html/draft-whittle-rrg-critiques

(currently just a place-holder) I will include full length versions of
all the critiques, including Noel's which do not appear in full in the
RRG Report.

   Abstract

   This ID will contain the full-length versions of all critiques
   which were not included in full in the RRG Report.  It may also
   point to discussions I consider significant, and include any
   "rebuttals" or "reflections" which were not included in full in
   the Report.  This ID is to support and document the IRTF Routing
   Research Group discussions concerning scalable routing, in early
   2010.  It is not a formal product of the RRG.  To what extent
   its contents are supported by RRG consensus can only be
   determined by the co-chairs.

Dino wrote:

> This is simply not true. The more specifics from site advertisements are
> withdrawn at the expense of coarse prefixes being injected by PITRs. The
> difference in the number of 1 or 2 orders of magnitude.

Yes, but I think factors of ~10 or ~100 are an underestimate.

These "coarse prefixes being injected by PITRs" have no specific name in
LISP, but I call the same things "Mapped Address Blocks" (MABs) in Ivip.
 I expect IPv4 MABs to be /24s to /16s or perhaps /12s.  Since many
end-user networks only need one or a handful of IPv4 addresses, I think
each MAB could serve the needs of thousands of end-user networks.  I
think the same applies to LISP, so I would say for LISP and Ivip that
there will generally be factors of 10^3 to 10^4 ratios between the
number of end-user prefixes supplied and the number of covering
("coarse" / MAB) prefixes advertised in the DFZ.

It is a feature of a CES architecture such as LISP or Ivip - both of
which support portability, multihoming and inbound TE benefits for all
packets irrespective of the level of adoption - that the scaling
benefits are in direct proportion to the level of adoption.  So I think
your first sentence is incorrect and any such statement does not apply
to LISP or Ivip.

Please see:

  CES & CEE are completely different (graphs)
  http://www.ietf.org/mail-archive/web/rrg/current/msg05865.html

The routing scaling benefits resulting from LISP and Ivip will be in
direct proportion to adoption.  There is no threshold in the level of
adoption - as implied by your use of "significant number of networks" -
at which routing scaling benefits begin

            ^
         B  |        *      Core-Edge Separation (CES)
         e  |       **
         n  |      ***
         e  |     ****
         f  |    *****
         i  |   ******
         t  |  *******
            | ********
            |*********
            0--------->
              Effort ~= level of adoption

> comment came out of no where.

Indeed.  What is a "data center"?  I have no idea what this second
sentence means.  It doesn't seem to be a critique of LISP.

  - Robin