Subject: Re: [rrg] Map and Encaps

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 1 Dec 2008 10:45:43 -0800

From: Tony Li [mailto:[email-masked]] 
<snip>

>My concern is that if we turn towards map-&-encap solutions, where we
can no longer actually run hosts in the native Internet architecture
(i.e., without encapsulation), we have, in some sense, 
>fundamentally given up on the ability of the host to act as a first
class element in that architecture.
>Instead, it must now be shielded, protected from reality and encased in
its private bubble of non-reality.

>Put another way: I see the strong appeal of a recursive architecture
and I embrace that portion of the goal.  However, all recursion starts
with a base case, and it troubles me that the host cannot be an element
of that base case.  
>It implies that the base is flawed and has limitations in its
functionality.  Noel likes to say that the measure of an architecture is
its ability to adapt to new requirements that are not yet forseen.  
>What new requirements will arise in the base architecture where the
fundamental lack of host participation will cripple us?

Tony,

Thank you for this reply. Unfortunately, I fear that you didn't
precisely state your insights on this posting such as you did on your
previous posting. (Or, if you did, I somehow didn't clue in.) I
therefore suspect that my reply here will not address your actual
concern. If so, could you please re-state your insights using different
wording? 

Map-and-encaps shields the host from knowing the actual network topology
such that an arbitrarily complex network topology can be created that is
transparent to the end user host as well as to other networks. This
allows the Internet to actually consist of a series of arbitrarily dense
hierarchical systems each on a local basis. The only requirement is that
each local system interface with other systems at the same level of
recursion in which it entered (i.e., the ingress and egress interface(s)
be at the same recursion level). This gives the network provider a great
deal of flexibility to address tomorrow's unforeseen requirements. 

I believe that this is A Good Thing because we have repeatedly
confronted huge network deployments that could not easily be supported
by two tiered (for IGP) network design hierarchies. To use your wording,
we in Boeing have known since circa 1993 that IP is "flawed" in that
manner because we repeatedly have broken IP products and routing by our
vast size. Consequently, we eventually invented multi-tiered solutions
for such large deployments (e.g., when I last dealt with Boeing's
Corporate Enterprise network it was a 3-tiered hierarchical IGP
infrastructure). I can well imagine that similar scaling problems also
exist for EGP, though I have not worked in that domain myself.

I do not resonate with your desire that the host should have insight
into network topology. Perhaps this is because my first protocols were
BSC and SNA. I am very familiar with the weaknesses of those systems
including the operational kludges we had to endure to get them to scale
into large deployments (e.g., source routing -- ouch!!). The reason I
became excited about IP almost two decades ago was because IP freed the
host from having to know anything about the network. In my way of
thinking, this is a fundamental source of the power and attractiveness
of IP. Specifically, I do not want the host to know about the network
design because, whenever such knowledge is required, it ultimately
creates a configuration nightmare for the large end user.

I believe that map-and-encaps should solely be a network infrastructure
design alternative. To be effective, the hosts should not be aware that
map-and-encaps (or any other network topology tool) exists. 

In my biased opinion this means that claims such as "LISP provides an
identifier-locator split" is fundamentally unrelated to the fact that
HIP provides an identifier-locator split. Quite obviously, HIP does so
in a manner that is real to the host (including applications) while LISP
does so in a manner that is known to routers only. [My own bias is that
LISP's claims in this regard are marketing while HIP's claims are close
to what the ROAD Group originally intended.]

--Eric
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 1 Dec 2008 12:54:21 -0800

>In some ways (and I admit that this is an imperfect 
>analogy), map-&-encap asks our hosts to trust their 
>transport to a technology that they can no longer 
>interact with.  How do we manage the underlying network?  
>What happens when a host needs some exceptional 
>service from the network?  This seems problematic 
>and complicated, when the root architecture of the 
>network must be truly simple and elegant.

How can hosts make requests of the network today? The only direct
mechanism that I am aware of is QoS. The only other mechanisms that come
to my mind are out-of-band systems such as application-layer admission
control systems -- the latter often rely on ECN reporting. Cumulatively
these systems rely on the TOS-byte of the IPv4 header and the IPv6
equivalent. What am I missing? How does anything change with
map-and-encaps? To my mind, there is no change. What are you seeing that
I am apparently missing?

For example, if your reply to my first query (above) mentions MPLS then
we will disagree -- I know of no host based systems that use MPLS. I am
only aware of network services using it (e.g., L3VPN). It wouldn't
surprise me if people try to create host services that use MPLS. The
problem with doing that is that it doesn't scale. More to the point,
such systems create configuration headaches for the large end user,
which is A Bad Thing. 

I've also seen deployments try to use the IP header's source route
option (e.g., to differentiate between gateways). That's another
technique that doesn't scale. 

Yet even so, both approaches (MPLS, Source Route) don't change with
map-and-encaps because they can only know about their own recursive
instance (i.e., our hosts are not be aware that our ISPs (note the
plural) internally convey our traffic across their infrastructures using
MPLS).
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Christopher Morrow" <[email-masked]>
Date: Tue, 2 Dec 2008 04:07:15 -0500

argh!!! wrong src, sorry.

On Sat, Nov 29, 2008 at 2:48 PM, Christopher Morrow
<[email-masked]> wrote:
> On Sat, Nov 29, 2008 at 2:08 PM, Templin, Fred L
> <[email-masked]> wrote:
>>>|> That implies that the
>>>|> ETR does a mapping lookup on the receipt of a packet, buffers
>>>|> the packet until the lookup succeeds, and the does the
>>>|> compare.
>>>|
>>>|Oh you mean like the IPv6 neighbor discovery process!?
>>>
>>>
>>>Two wrongs don't make a right.
>>
>> Why buffer the packet until the lookup succeeds? Why not
>> just accept the first few packets while a lookup is done
>
> a synflood is a bunch of 1 packet flows :( you lose, I win! yippee! :(
> Seriously though, if you send through 'some' of the bad packets all
> the attacker has to know is how many 'some' is... in the worst case
> the answer is 'one'.
>
> Buffering is bad, really, really bad.
>
> -chris
>

---

From: "Fleischman, Eric" <[email-masked]>
Date: Tue, 2 Dec 2008 10:47:09 -0800

From: Christopher Morrow [mailto:[email-masked]] 
On Mon, Dec 1, 2008 at 3:54 PM, Fleischman, Eric
<[email-masked]> wrote:
>>>In some ways (and I admit that this is an imperfect analogy), 
>>>map-&-encap asks our hosts to trust their transport to a technology 
>>>that they can no longer interact with.  How do we manage the 
>>>underlying network?
>>>What happens when a host needs some exceptional service from the 
>>>network?  This seems problematic and complicated, when the root 
>>>architecture of the network must be truly simple and elegant.
>>
>RSVP - windows media player (or another media player I'm forgetting
right now) does RSVP >reservation requests... fun fun fun!

Wow, Chris, this is really weird: I helped create parts of that product
when I worked for Microsoft a bit over a decade ago. In my way of
thinking, this is an example of the part of my posting (that was deleted
above) where I mentioned out-of-band application admission control.
Session scheduling does impact network load and therefore is relevant to
the network. However, the only mechanism in practice that is available
to hosts (that I am aware of) that can impact routing is QoS (e.g., the
IP host routing option doesn't scale). 

The nature of my push-back to Tony is that I believe that the
capabilities that Tony was ascribing to hosts aren't capabilities that
actually are available to hosts at all. Rather, they are capabilities
that are available to networks because of map-and-encaps (e.g., MPLS).
Therefore, to argue against map-and-encaps because it encroaches on
capabilities that exist for hosts (but which actually don't exist for
hosts) is a spurious objection to map-and-encaps. Rather, I believe that
the actual situation is a further testament to the prevalence of
map-and-encaps today.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Tony Li" <[email-masked]>
Date: Wed, 3 Dec 2008 18:55:34 -0800

Hi Eric,

|>> How can hosts make requests of the network today? The only direct 
|>> mechanism that I am aware of is QoS. 

IGMP and ARP immediately come to mind.  Various other IP options,
traceroute, ping, etc. ...

Take a look at the OAM discussion that's going on in CCAMP right now.  How
do we debug the network if we can't send requests from the host?

|The nature of my push-back to Tony is that I believe that the
|capabilities that Tony was ascribing to hosts aren't capabilities that
|actually are available to hosts at all. Rather, they are capabilities
|that are available to networks because of map-and-encaps (e.g., MPLS).

Let's be careful.  MPLS is an encapsulation, to be sure, but there's not a
lot of 'mapping' going on.  Sure, with an L3VPN or TE solution, you have a
FEC that forwards your packets down a particular LSP, but all of this is
based on pushed or configured routing information.  Not the same thing at
all.  As we've said before many times, the mapping is a very tricky part of
the equation.

Also, we should note that the reason that we get benefits from MPLS is that
it allows us to have a hybrid network architecture, where we can support
both connectionless (i.e. packet switching) and connection oriented (i.e.
circuit switching) styles.  If we go down the path of map-&-encap, we're
effectively deciding to run on top of a connection oriented infrastructure.
What then happens to the hybrid strength of the architecture?

Tony

---

From: Lan Wang <[email-masked]>
Date: Wed, 3 Dec 2008 22:36:47 -0600

> effectively deciding to run on top of a connection oriented  

Could you clarify your last sentence?  It's unclear to me how map-&- 
encap and connection-oriented infrastructure are related here ...

Lan

---

From: "Tony Li" <[email-masked]>
Date: Wed, 3 Dec 2008 23:17:21 -0800

Hi Lan,

|> If we go down the path of map-&-encap,  
|> we're
|> effectively deciding to run on top of a connection oriented  
|> infrastructure.
|
|Could you clarify your last sentence?  It's unclear to me how map-&- 
|encap and connection-oriented infrastructure are related here ...

In a map-&-encap system, you basically end up with a series of tunnels
(connections) between xTR's that form the connection-oriented
infrastructure.  You're then running your packet switching on top of it and
you can no longer do native layer packet switching.

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 3 Dec 2008 23:29:19 -0800

> What then happens to the hybrid strength of the architecture?

There is no difference between IP running over a connection oriented  
architecture and map&encap running over a connection oriented  
architecture.

Dino

---

From: "Tony Li" <[email-masked]>
Date: Wed, 3 Dec 2008 23:33:09 -0800

|There is no difference between IP running over a connection oriented  
|architecture and map&encap running over a connection oriented  
|architecture.

... except the mapping function.  ;-)

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 3 Dec 2008 23:35:00 -0800

You were talking about the data-plane. Why did you change course.

Dino

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 01:21:01 -0800

|You were talking about the data-plane. Why did you change course.

I was talking about the whole architecture.

Tony

---

From: "William Herrin" <[email-masked]>
Date: Thu, 4 Dec 2008 10:48:25 -0500

> What then happens to the hybrid strength of the architecture?

Huh? Your point escapes me.

In production we use MPLS to support both core routing (where the RIB
load on intermediate routers is reduced) . We also use it to provide
virtual circuits, such as virtual ethernets between one customer
location and another.

The latter use has nothing whatsoever to do with IP internetworking.
It's just another function that MPLS happens to be good for. My
routers can also implement frame relay switching and provide a PVC
between two customer locations. So what?

Why would map-encap for Internet core routing have any impact at all
on the use of MPLS for virtual circuits?

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: [email-masked] (Noel Chiappa)
Date: Thu, 4 Dec 2008 11:05:56 -0500 (EST)

> From: "Tony Li" <[email-masked]>

    > If we go down the path of map-&-encap, we're effectively deciding to run
    > on top of a connection oriented infrastructure.

I don't think I agree with this. It is certainly true that we're almost
building a new packet-switching system on top of an existing one - and this
certainly creates some problems (which I'm currently struggling with offline,
along with a number of other people).

I saw "a new packet-switching system" because at the ITR you have to keep
track of all the ETRs which are available to get to the ultimate destination;
decide which one to use; make sure it's up, and switch to a different one if
it's not; etc, etc. All sounds very familiar, doesn't it? :-)

But I don't at all see how treating the entire Internet as a NBMA network
means we're treating it like a "connection oriented infrastructure", any more
than when we did almost exactly the same thing, when we started running IP
over the old ARPANet. All we're doing is treating a composite NBMA network
(an internetwork, to be technical) as if it were a classic single physical
NBMA, which the 'routers' (xTRs in this case) use to communicate directly
with each other, without much knowledge of what's going on inside that 'black
box' NBMA they are using for data carriage.

What's troubling me much more (wearing both engineering and architectural
hats) is that we are, in effect, building a second packet-switching system on
top of the first, but separate from it - and with only the dimmest of plans
(if any, in most cases) to make sure that in the long run we have a path _out_
of that kludgy, exponentially-complex mess, back to something a little more
integrated, clean, flexible, powerful and robust.

    > MPLS ... allows us to have a hybrid network architecture, where
    > we can support both connectionless (i.e. packet switching) and
    > connection oriented (i.e. circuit switching) styles.

Ever since this was first explicitly pointed this out to me (by Yakov), I've
always felt it was one of the strong points of MPLS (although I think it was
still called the older name when I had that chat), and of couse of this class
of architecture in general.

(Indeed, a system I proposed some time before provided both connectionless -
which I termed datagram - and flow-oriented service, on top of a
flow-oriented substrate. I use the term 'flow-oriented' because to many
people 'connetion-oriented' also means in a lot of other stuff, like
reliability, which I don't mean to include.)

	Noel

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 08:57:34 -0800

Hi Bill,

|Huh? Your point escapes me.
|
|In production we use MPLS to support both core routing (where the RIB
|load on intermediate routers is reduced) . We also use it to provide
|virtual circuits, such as virtual ethernets between one customer
|location and another.
|
|The latter use has nothing whatsoever to do with IP internetworking.
|It's just another function that MPLS happens to be good for. My
|routers can also implement frame relay switching and provide a PVC
|between two customer locations. So what?
|
|Why would map-encap for Internet core routing have any impact at all
|on the use of MPLS for virtual circuits?

You're right, you're missing my point.  ;-)

The point is that once you decide to run the network on top of virtual
circuits, that means that you can no longer do packet switching on the 'bare
metal'.  

In some sense, this is an admission that IP is insufficient.  It needs an
underlying layer of support before it can be run.  I would much rather see
IP be self-sufficient.

Tony

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 09:20:10 -0800

Hi Noel,

|    > If we go down the path of map-&-encap, we're effectively 
|deciding to run
|    > on top of a connection oriented infrastructure.
|
|I don't think I agree with this. It is certainly true that we're almost
|building a new packet-switching system on top of an existing 
|one - and this
|certainly creates some problems (which I'm currently 
|struggling with offline,
|along with a number of other people).

Ok, but the underlying system is effectively a set of connection-oriented
tunnels.  Yes, you're using packets to emulate that connection, but you're
definitely treating the tunnel as a point-to-point link.

|But I don't at all see how treating the entire Internet as a 
|NBMA network
|means we're treating it like a "connection oriented 
|infrastructure", any more
|than when we did almost exactly the same thing, when we 
|started running IP
|over the old ARPANet. All we're doing is treating a composite 
|NBMA network
|(an internetwork, to be technical) as if it were a classic 
|single physical
|NBMA, which the 'routers' (xTRs in this case) use to 
|communicate directly
|with each other, without much knowledge of what's going on 
|inside that 'black
|box' NBMA they are using for data carriage.

I guess I'm not seeing too much of a conflict between these views.  Yes, you
end up creating an NBMA architecture.  I guess another way of making my
point is to look at how well we've done on dealing with NBMA systems before.
IMHO, it's not been all that well.  For the old ARPANet, we ended up burying
the IMP number inside of the IP address.  Then, because routing didn't work
so well, we ended up taking multiple hops across the NBMA network.

We've also tried to solve the NBMA problem with NHRP.  That didn't go so
well because we couldn't really find a way to keep the levels (virtual &
physical) separated.

Other efforts to solve it have required manual configuration of IP addresses
within IGPs.  Not even possible.

For the sake of the current discussion, I'll stipulate that we can solve the
routing issues with the mythical perfect mapping funtion.  However, I'm
still gravely concerned that we can no longer run natively and end up with
what you describe as a "kludgy, exponentially-complex mess".

|What's troubling me much more (wearing both engineering and 
|architectural
|hats) is that we are, in effect, building a second 
|packet-switching system on
|top of the first, but separate from it - and with only the 
|dimmest of plans
|(if any, in most cases) to make sure that in the long run we 
|have a path _out_
|of that kludgy, exponentially-complex mess, back to something 
|a little more
|integrated, clean, flexible, powerful and robust.

Agreed.  Moreover, if there is a way out, why do we need this kludge as a
stepping-stone...

Tony

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 4 Dec 2008 09:41:20 -0800

>|> If we go down the path of map-&-encap, we're effectively deciding to

>|> run on top of a connection oriented infrastructure.
>|
infrastructure.  
>You're then running your packet switching on top of it and you can 
>no longer do native layer packet switching.

I'm not following this part of your insight, Tony. I suspect that you
are using "connection oriented" incorrectly. Map-and-encaps introduce an
intermediate source and destination that comprise a tunnel. Entities
that use that tunnel view the potentially multi-hop tunnel as being a
single hop. Connectionless packet switching indeed occurs across these
tunnels but it also can occur within them as well (e.g., ESP in tunnel
mode, IP-IP, GRE, military COMSEC).

By contrast, people of our generation who went through numerous circuit
switching-versus-packet switching wars find the term
"connection-oriented architecture" very emotive. Are you trying to play
off of that emotion in this instance? Regardless, true
connection-oriented architectures always historically did resource
reservation and path establishment before a session is created. In this
case, however, sessions don't exist at all (unless they were created by
other means (e.g., HIP)). True, it does have (partial) path
establishment but definitely not resource reservation. Lacking sessions
it can't be a connection-oriented system in my book. Rather, it is
something else entirely: a "worm hole" within a connectionless world.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 09:43:49 -0800

Hi Eric,

|I believe that this is A Good Thing because we have repeatedly
|confronted huge network deployments that could not easily be supported
|by two tiered (for IGP) network design hierarchies. To use 
|your wording,
|we in Boeing have known since circa 1993 that IP is "flawed" in that
|manner because we repeatedly have broken IP products and routing by our
|vast size. Consequently, we eventually invented multi-tiered solutions
|for such large deployments (e.g., when I last dealt with Boeing's
|Corporate Enterprise network it was a 3-tiered hierarchical IGP
|infrastructure). I can well imagine that similar scaling problems also
|exist for EGP, though I have not worked in that domain myself.

I think that this says more about our IGP's than anything about IP.  PNNI,
for example, allows more layers of hierarchy.  Similarly, hacking IS-IS to
provide more hierarchy is also relatively trivial.  None of this requires a
level of virtualization.

|I do not resonate with your desire that the host should have insight
|into network topology. Perhaps this is because my first protocols were
|BSC and SNA. I am very familiar with the weaknesses of those systems
|including the operational kludges we had to endure to get them to scale
|into large deployments (e.g., source routing -- ouch!!). The reason I
|became excited about IP almost two decades ago was because IP freed the
|host from having to know anything about the network. In my way of
|thinking, this is a fundamental source of the power and attractiveness
|of IP. Specifically, I do not want the host to know about the network
|design because, whenever such knowledge is required, it ultimately
|creates a configuration nightmare for the large end user.

Agreed, but in removing the capability from the host, you also lose
flexibility and generality.  The strength of IP is that knowledge in the
host is not _required_, but is allowed.  We're doing away with that
capability.

Tony

---

From: "William Herrin" <[email-masked]>
Date: Thu, 4 Dec 2008 12:44:47 -0500

Tony,

We're generally not switching IP on the bare metal even now. As often
as not we're switching ethernet frames, MPLS packets or sonet frames.
Map-encap changes the adjacency of IP nodes in the network but it
doesn't fundamentally change the switching model. Right?

Regards.
Bill Herrin

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 4 Dec 2008 10:08:51 -0800

Hi Tony,

>-----Original Message-----
>Sent: Thursday, December 04, 2008 9:20 AM
>To: 'Noel Chiappa'; [email-masked]
>Subject: Re: [rrg] Fundamental objections to a
host-basedscalableroutingsolution
>definitely treating the tunnel as a point-to-point link.

The new model would indeed look like Switched Virtual Circuits
between xTRs connected to the global Internet DFZ, but that does
not mean that the old model goes away. IMHO, this should be about
arresting routing table growth at or near current levels w/o
having to change the service models for existing deployments. In
the meantime, however, address scaling and new services are
accommodated by map-and-encaps and by rolling out IPv6. 

>so well, we ended up taking multiple hops across the NBMA network.

ISATAP addresses bury the L2 (IPv4) address within the L3 (IPv6)
address, but thankfully that address form is really only needed
for next-hop resolution and operation of the IPv6 ND protocol.
That, and for hosts that configure their own xTRs - but, I'm
not sure we'll see many of those connected directly to links
within the DFZ.

>within IGPs.  Not even possible.

The ISATAP NBMA next-hop resolution model asks for a well
maintained mapping system. Within a small site it may be
sufficient to simply have mappings to a few hosts and default
routers, but at Internet-scale we will certainly need to be
able to map more-specific routes. IMHO, we seem to be seeing
some positive developments along the lines of the mapping
system even in recent discussions on this list.  

>what you describe as a "kludgy, exponentially-complex mess".

But, you still can run natively in exactly the same way we
do it today; IPv4 routing in the DFZ, large IPv4 enterprises
and IPv4 NATs do not go away. Only new networks and services
will be the ones that use map/encaps for IPv6, but AFAICT that
would not make them "second class citizens".

As was once told to me long ago, the approach is one of
building a "second story" on top of the existing Internet.
I think (but am not certain) that this is essentially the
same thing Noel has described as "jack-up".

Anyway, I don't see kludge and complex mess in any of this;
I see only benefits that could not be realized with a non
map-and-encaps approach. And, as also was once told to me,
"complexity is in the eye of the beholder".

Fred
[email-masked]

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 10:15:05 -0800

Hi Eric, 

|By contrast, people of our generation who went through numerous circuit
|switching-versus-packet switching wars find the term
|"connection-oriented architecture" very emotive. Are you trying to play
|off of that emotion in this instance?

Of course not, I'm much more dispassionate than that.  Connection-oriented
architectures have some significant advantages.  That's why we ended up
using them for RSVP/TE, for example.  You can't reasonably set up an
explicit path for a traffic aggregate in a connectionless architecture (with
sane overhead).

|Regardless, true
|connection-oriented architectures always historically did resource
|reservation and path establishment before a session is created. 

True, however, that's not a precondition.

|In this
|case, however, sessions don't exist at all (unless they were created by
|other means (e.g., HIP)). True, it does have (partial) path
|establishment but definitely not resource reservation. Lacking sessions
|it can't be a connection-oriented system in my book. Rather, it is
|something else entirely: a "worm hole" within a connectionless world.

Ok, well, we live with different books.  In my book, a connection-oriented
system is one where point-to-point connections are the basic building block
in the architecture.  

Tony

---

From: "Fleischman, Eric" <[email-masked]>
Date: Thu, 4 Dec 2008 10:18:32 -0800

>Agreed, but in removing the capability from the host, you also lose
flexibility 
>and generality.  The strength of IP is that knowledge in the host is
not _required_, 
>but is allowed.  We're doing away with that capability.

I wondered how you could agree with my point that hosts should not have
visibility into the network by design and then state that capabilities
to do so were removed from the host. It then hit me that my postings
were about normal application data plane functions but you are
apparently thinking about network management -- very different topics. 

I agree that managers cannot know anything about structures that occur
at a different recursion layer than themselves. However, this is only
one of a large repertoire of problems that make network management very
challenging. The current state of network management for the Large End
User is very troubling -- ditto for highly mobile networks. I've always
viewed this as an inherent failing of network management itself (i.e.,
the foundation is inadequate) but I concur that map and encaps makes it
worse.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 18:35:45 -0800

Hi Eric,

|>The strength of IP is that knowledge in the host is
|not _required_, 
|>but is allowed.  We're doing away with that capability.
|
|I wondered how you could agree with my point that hosts should not have
|visibility into the network by design and then state that capabilities
|to do so were removed from the host. It then hit me that my postings
|were about normal application data plane functions but you are
|apparently thinking about network management -- very different topics. 

Just to be very careful here: I agree that not requiring that the host know
about the network is a very strong idea.  However, I also feel pretty
strongly that the host should be able to know about the network.  Management
of the network (e.g., traceroute) is made possible only when the end-host
has visibility.

|I agree that managers cannot know anything about structures that occur
|at a different recursion layer than themselves. However, this is only
|one of a large repertoire of problems that make network management very
|challenging. The current state of network management for the Large End
|User is very troubling -- ditto for highly mobile networks. I've always
|viewed this as an inherent failing of network management itself (i.e.,
|the foundation is inadequate) but I concur that map and encaps makes it
|worse.

Is it really the failing of management?  Or is it concurrent with the
addition of a layer?  If you add a layer of abstraction, then how is
management supposed to work given that it is prohibited about knowing about
the lower layer?  In my book, this is exactly the architectural failing of
having an additional network layer.  Yes, we live with it today for the sake
of VPNs and the like, but is that really what we want for everyday life?

Regards,
Tony

---

From: "Tony Li" <[email-masked]>
Date: Thu, 4 Dec 2008 18:40:46 -0800

|We're generally not switching IP on the bare metal even now. As often
|as not we're switching ethernet frames, MPLS packets or sonet frames.
|Map-encap changes the adjacency of IP nodes in the network but it
|doesn't fundamentally change the switching model. Right?

Hi Bill,

Right now, when we're doing L3 switching on an IP header, I consider that to
be the atomic primitive, regardless of the L2 encapsulation.

What has changed has been the introduction of these 'L2.5' technologies like
MPLS.  There we switch based on the label, not on the real payload
addressing, and it does introduce complexity.  

Are we very sure that that model is what we're willing to live with moving
forward?  Consider that some MPLS implementations, as part of the MPLS
switching operation, grovel all the way up the label stack and find the IP
header to hash its addresses so that they can correctly do ECMP.  Is that
the kind of architecture that we consider clean and neat?

Regards,
Tony

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 00:43:59 -0500

Hi Tony,

I say I MPLS is a plain old layer-3 technology that happens to run
another layer-3 technology on top of it. I say GRE is the same.

And I say that ethernet frames in the current incarnation of ethernet
are also a layer 3 technology, with packets stored and forwarded in a
switched network that can and in many cases does span hundreds of
forwarding nodes. Only the logic that recognizes the start and end of
the frame over what these days is a point-to-point link between host
and switch still qualifies as layer-2.

Calling ethernet frames layer-2 merely because in the ethernet of two
decades ago they really were treated as layer 2 just serves to confuse
the architectural discussion with a distinction that has long since
ceased to exist.

They're all layer-3 technologies today. We stack up as many layer-3
protocols as we need to to get to layer 4 and we use a mapping
function or protocol at each level of the layer-3 stack to find the
right address at the next level down. "Map-encap" just adds another
layer-3 level to the two or three that are *already there*.

While useful as shorthand, we all know the OSI Network Model is fubar.
It would be a shame to let it box in our thinking.

> the kind of architecture that we consider clean and neat?

I'll cede the point because:

A) This is not an inherent limitation of map-encap systems. TRRP
functions quite happily with the ITR directly on the originating host
and the end-user application software in full control of its behavior.

B) The notion of stripping the host of any architectural possibility
of interacting directly with the network core makes me pretty nervous
too.

Regards,
Bill Herrin

---

From: [email-masked]
Date: Sat, 06 Dec 2008 11:43:47 -0500

> the kind of architecture that we consider clean and neat?

If only MPLS had left out the TTL (and EXP) fields, and just put in a  
word-offset to the IP header (or pseudowire header if you prefer).

Regards,

// Steve

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 8 Dec 2008 09:42:53 -0800

>Is it really the failing of management?  
>Or is it concurrent with the addition of a layer?  
>If you add a layer of abstraction, then how is management supposed to
work given that it is prohibited about knowing about the lower layer?  
>In my book, this is exactly the architectural failing of having an
additional network layer.  >Yes, we live with it today for the sake of
VPNs and the like, but is that really what we want for everyday life?

My opinions about IP network management and SNMP were frequently not
popular in the IETF SNMP community, but my firm opinion is that network
management for IP systems has **never** worked well for the large end
user. It suffers in several ways, notably including (sometimes subtle)
schema mis-matches between different products, poor scaling,
demonstrably inadequate security (hopefully the ISMS WG is fixing this),
and a desperate need for better information fusion. It is conceivable to
me that deployments with only a few different types of nodes may not
have trouble with network management, but my observation with vast
deployments is that the IP network management technology verges on
broken. This was the case well before maps-and-encaps became popular --
though map-and-encaps has definitely not improved an already problematic
system as you observe. Regardless, map-and-encaps can't be blamed for a
system that is inherently flakey.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 8 Dec 2008 10:25:32 -0800

I should have also mentioned is that in many (most?) maps-and-encaps
deployments (e.g., L3VPN, Military COMSEC) there are policy issues that
influence what network management can and cannot see. Thus, the actual
situation is considerably more complex than your argument implies
because policy always trumps both routing and management.

-----Original Message-----
From: Fleischman, Eric 
Sent: Monday, December 08, 2008 9:43 AM
To: [email-masked]'; 'Brian E Carpenter'
Cc: [email-masked]; 'Noel Chiappa'
Subject: RE: [rrg] Map and Encaps

>|one of a large repertoire of problems that make network management 
>|very challenging. The current state of network management for the 
>|Large End User is very troubling -- ditto for highly mobile networks. 
>|I've always viewed this as an inherent failing of network management 
>|itself (i.e., the foundation is inadequate) but I concur that map and 
>|encaps makes it worse.
>
>

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 16:17:08 -0500

Excerpts from Tony Li on Mon, Nov 24, 2008 06:27:00PM -0800:
> There is a portion of the community (significant, IMHO) that feels that
> there are significant drawbacks to the map-&-encap strategy.  While I can't
> claim to represent all of them or even a significant breadth of their
> opinions, there are some issues that have been raised.

I think you've raised some good ones but could have filtered them by
how significant they are.

> - Map-&-encap doesn't solve the problem where it truly occurs: at
> the host.  The convolution of locator and identifier happens
> precisely because the transport layer has reused the address as part
> of the connection identifier.  Until we change this, we're simply
> band-aiding around the problem.  And band-aids really aren't the way
> to create a lasting architecture.  Please note that this also
> implies that routing-only solutions aren't the correct direction.

That problem is the need to have an identifier, usable by session
control, that is independent of topological location.  That is a
different problem from routing and addressing scaling in the core.
Some approaches to endpoint-based loc/id separation also indirectly
reduce the routing scaling problem, but that doesn't mean that
something that attacks routing scaling directly is not attacking "the
problem".  You can run a network-based solution for routing scaling
and a host-based solution for separate session-usable identifiers
separately.  Architecturally that could be cleaner.  Yes we should ask
if host-based approaches to loc/id separation, or maybe NAT-based
ones, make map-and-encap irrelevant.  No, we shouldn't say that
attacking routing scaling directly is attacking the "wrong" problem.

> - Map-&-encap comes with significant overhead.  It adds another
> network layer header, and some additional per packet overhead.  In
> changing the MTU, it requires additional mechanisms that some (tho I
> appear to be alone in this ;-) find concerning.

Yes.  

> - Map-&-encap creates some attack vectors.  If an attacker sends you
> a packet from a forged (and possibly random) EID, and that causes
> you to respond and perform a mapping lookup on the forged EID, then
> the attacker can cause you to fill your mapping cache and thus
> create a significant performance issue.  While it's already
> difficult in the Internet today to trace forged source addresses, it
> would seem like tracking forged EIDs will be even harder, as they
> are likely to come with addresses from forged ITR addresses.  There
> are probably ways to address this, but they would seem to come with
> Even More Mechanisms.

I owe you a response on that.  Some ideas have been floating around.
As you can see I'm somewhat behind. :-p

> - Is virtualization really the right approach?  It's been noted that
> map-&-encap is basically equivalent to running the entire Internet
> inside of a VPN, where the mapping function conveys the edge of the
> 'primary' infrastructure that terminates the VPN tunnels.  Some find
> it architecturally disquieting to run our most essential function
> virtually when we find that we cannot run it natively.

Map-and-encap is not a VPN by any means, nor is there anything virtual
about it.  Routing, forwarding, and addressing are all common.  There
is only one kind of packet crossing the core.  This isn't about
"overlay", it's about "separation".

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 16:59:11 -0500

Excerpts from Noel Chiappa on Thu, Dec 04, 2008 11:05:56AM -0500:
> What's troubling me much more (wearing both engineering and
> architectural hats) is that we are, in effect, building a second
> packet-switching system on top of the first, but separate from it 

As argued elsewhere ... in order for it to be a different "packet
switching system", it would have to have different addressing,
routing, and switching.

In the scope of the globally routed Internet, addressing is the same
-- a BGP neighbor in the core BGPs with an ETR just as with any other
node.  Routing is the same -- good old BGP.  The switching is the same
-- every packet is switched by the same core routers in the same way.

All that's different is that packets are prepared for crossing the
globally routed Internet at a new point, which is to say that when
they implement map-and-encap, edge sites are separated away from the
core, but the core is the same.  There is no new switching system,
only a new adaptation, at its edges, to the existing switching system,
once a prefix is removed from it.  

Excerpts from Tony Li on Thu, Dec 04, 2008 09:20:10AM -0800:
> Ok, but the underlying system is effectively a set of
> connection-oriented tunnels.  Yes, you're using packets to emulate
> that connection, but you're definitely treating the tunnel as a
> point-to-point link.

At the last three IETF meetings I've told you that encapsulation does
not imply "tunnel", and it certainly doesn't imply a network-layer
connection.  There is no setup phase, and there is no shared state
necessary for a decapsulator to receive a packet.  If I were a
decapsulator, you could encapsulate a packet and throw it at me and I
would do the right thing whether I had ever heard of you before or
not.  Paul Francis and crew called a "funnel", as opposed to a
"tunnel".

> Agreed.  Moreover, if there is a way out, why do we need this kludge
> as a stepping-stone...

Ignoring "kludge" for the moment because everything is equally grubby
as far as I can tell, this is the big question.  This is why it's
important to evaluate the various approaches to locator/identifier
separation.  Can we solve the routing scaling problem indirectly while
solving locator/identifier split for session control?  

I'm not saying map-and-encap is the most elegant thing we can do.
However, we do have time constraints.  You can only change all the
hosts in a few special markets.  For others you have to do some kind
of "separation", whether NAT, proxying, or map-and-encap.  NAT doesn't
work for sites that are multihomed unless you change the hosts (socket
bindings break when routing changes).  Proxying is desperate.  What
are you left with?  Also, see Lixia's slides from two IETFs ago where
she said that investing in developing map-and-encap loses us little
and may gain us everything.

Scott

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 17:03:42 -0500

Excerpts from Michael Meisel on Sun, Nov 30, 2008 03:33:53PM -0800:
> Hi everyone, one of the APT people here. =) So we actually did an
> analysis of potential default mapper load a while back, based on real
> traffic traces at two POPs that serves mostly academic networks, taking
> into account the RTT to the default mapper and the cache size and
> expiration time. Even in the worst case, the results show that the load
> on the default mappers is pretty small.

I would be concerned about bogons.  I'm told that there are millions
of crap packets that currently get thrown away at site borders.  I
don't know how many of your campus machines are owned bots, but almost
15% of the hosts on the Internet are.  Did you see any of that?

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 17:23:55 -0500

Excerpts from Tony Li on Thu, Nov 27, 2008 01:11:31AM -0800:
> All of the Loc/ID split solutions are precisely adding another layer
> of indirection and thus it behooves us to truly understand what
> problems we are creating.

There are many components to an Internet system, and I think we have
to pick things apart some before we start talking about "adding
another layer of indirection".  Let's look at MIP:

  - There is no new level of indirection to mapping a domain name to a
    locator.

  - There is indirection in initial contact.  That has a number of
    potential advantages.  My favorite is privacy.  Another is
    scalability of the mapping system during mobility.

  - There is indirection for initial authentication for direct
    communication.

  - There is indirection for identifier re-authentication for session
    continuity.  (In neither case would I call this a "layer" of
    indirection.)

HIP optionally adds other indirection and takes away most of the
above.  

Does Trilogy multipath add indirection?  It adds a need for _control_
of existing communication, certainly.

> and I embrace that portion of the goal.  However, all recursion
> starts with a base case, and it troubles me that the host cannot be
> an element of that base case.  It implies that the base is flawed
> and has limitations in its functionality.  Noel likes to say that
> the measure of an architecture is its ability to adapt to new
> requirements that are not yet forseen.  What new requirements will
> arise in the base architecture where the fundamental lack of host
> participation will cripple us?

When you talk to me, you are shielded from seeing what goes on on a
DWDM link in Kansas.  Are you flawed?  :-)

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 19:01:30 -0500

Excerpts from Christian Vogt at 20:36:11 +0200 on Fri 28 Nov 2008:
> Chris -
> 
> I agree with you that the overhead created by a host-based solution
> needs to be carefully looked at.  Yet I don't think that high overhead
> is inevitable for host-based solutions.  As you say, it depends on
> protocol design.  Take, e.g., the overhead that is needed for
> connectivity verification:  Whereas host-based solutions cannot
> aggregate explicit probing messages as efficiently as network-based
> solutions, host-based solutions can potentially use implicit probing
> more efficiently than network-based solutions because hosts have session
> state that may include useful connectivity information.  So the
> overhead-efficiency of a host-based solution depends to a significant
> extent on how effectively the solution exploits such session state.

We need real numbers.  First, probing in general has serious scaling
problems and we need to quantify how much overhead they are (relative
to actual data packets).  Second, you say endpoints have session state
that may include useful information, but they only get that state from
traffic flowing (e.g. a TCP keepalive), and points further into the
network have even more traffic flowing (they would see that TCP
keepalive and many more, so they wouldn't need it).  Endpoints have an
advantage when flows are asymmetric but lose it when they have to
check inactive locators for possible liveness.  In any case, one of
the fundamental goals should be that applications do not get led down
a garden path by either lower layers in the stack or the network.
Whatever we come up with needs to be able to provide robustness in the
face of network changes without overwhelming the network.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 19:07:11 -0500

Excerpts from Mark Handley at 10:47:08 +0000 on Fri 28 Nov 2008:
> However, once you have multi-path capable transport, then this opens
> up the option of doing smarter things for path discovery to try and
> ensure diversity.  The goal then is to specify these mechanisms in
> such a way that they can operate self-contained with whatever
> addresses and paths they're given, or to give better results when
> you add extra path choice mechanisms in the future.
> 
> Overall though, it remains a research question as to how much path
> diversity is needed overall to realize the benefits of resource
> pooling, and this is what we are currently starting to examine.

The first problem is initialization of your knowledge.  You send some
sort of probe to each of the target's locators.  You get some packets
back.  How do you establish that those are not attacks?  Do you need
to have your responses come from the same locators you sent the
initial queries to?

The second problem is maintenance.  What percent of your packets go to
knocking on the door of an apparently dead locator?  Multiply that
times the number of apparently dead locators belonging to endpoints
with active sessions.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Scott Brim <[email-masked]>
Date: Mon, 8 Dec 2008 22:01:01 -0500

Excerpts from Fleischman, Eric at 10:45:43 -0800 on Mon 1 Dec 2008:
> I believe that map-and-encaps should solely be a network
> infrastructure design alternative. To be effective, the hosts should
> not be aware that map-and-encaps (or any other network topology
> tool) exists.
> 
> In my biased opinion this means that claims such as "LISP provides
> an identifier-locator split" is fundamentally unrelated to the fact
> that HIP provides an identifier-locator split. Quite obviously, HIP
> does so in a manner that is real to the host (including
> applications) while LISP does so in a manner that is known to
> routers only. [My own bias is that LISP's claims in this regard are
> marketing while HIP's claims are close to what the ROAD Group
> originally intended.]

After a lot of arguing with others, I've come to see the similarities
wrt "locator/identifier split".  Look at it this way: the difference
is one of granularity.  An endpoint-based LIS separates an identifier
for a stack in that endpoint from its attachment point(s) to the
topology.  A network-based LIS splits a whole chunk of "identifiers"
from its attachment point(s) to the topology.  LISP calls those EID
prefixes.  Vince Fuller suggested having EID stand for "end site
identifiers".  Sometimes the "identifiers" within those chunks are
also used by forwarding functions as locators within the scope of the
final site, but they need not be (see GSE, ILNP etc.).

Just as source endpoints need to keep track of which destination
locators are functional, ITRs maintain liveness checks for an EID
prefix and forward packets to whichever locators for the EID prefix
are up.  When the set of locators for an endpoint identifier changes,
the endpoint updates its correspondents somehow.  An ETR responsible
for for an EID prefix will let sources sending to that identifier know
which locators are up for it (in LISP this is done with loc-reach
bits).  And so on.  Of course there are differences -- in
map-and-encap there are no higher stack layers to utilize identifiers
for higher layer functions -- but I think you see the parallels.

Scott

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 10 Dec 2008 09:41:25 +1300

Scott,

> final site, but they need not be (see GSE, ILNP etc.).

This is true and it shows the risk of trying to use a concept loosely.

I've said this from the beginning but I'll try once more: I get much
less confused about LISP if I use different terminology, because the
LISP "EID" is not an EID. Think of it as a local locator (which also
happens to be globally unique) and call it LOC0 for example. Then the
LISP "RLOC" is a less local locator that just happens to be globally
routable as well as globally unique; call it LOC1 for example. As HAIR
points out (and I pointed out last year) the obvious conclusion is a
recursive model with any number of layers of locator scope.

GSE/ILNP and HIP are deeply different in that they genuinely separate
a unique *non-locator* ID from one or more locators. Trying to use the
phrase "loc/id split" to cover both of these models is certain to
lead to confusion.

    Brian

---

From: Scott Brim <[email-masked]>
Date: Wed, 10 Dec 2008 06:35:29 -0500

Excerpts from Brian E Carpenter at 09:41:25 +1300 on Wed 10 Dec 2008:
> > After a lot of arguing with others, I've come to see the
> > similarities wrt "locator/identifier split".  Look at it this way:
> > the difference is one of granularity.  An endpoint-based LIS
> > separates an identifier for a stack in that endpoint from its
> > attachment point(s) to the topology.  A network-based LIS splits a
> > whole chunk of "identifiers" from its attachment point(s) to the
> > topology.  LISP calls those EID prefixes.  Vince Fuller suggested
> > having EID stand for "end site identifiers".  Sometimes the
> > "identifiers" within those chunks are also used by forwarding
> > functions as locators within the scope of the final site, but they
> > need not be (see GSE, ILNP etc.).
> 
> happens to be globally unique) and call it LOC0 for example.

A LISP EID names a network attachment point.  It is a locator with
global uniqueness but potential use by routing and forwarding is local
(near the beginning and end of a packet's path).

> Then the LISP "RLOC" is a less local locator that just happens to be
> globally routable as well as globally unique;

You need to be specific on "less local locator".  That's a question of
the functions that use it as input, not a property of the thing itself.
 A LISP RLOC is a locator with global uniqueness, just like an EID, but
potential use by routing and forwarding is "less local".

> call it LOC1 for
> example. As HAIR points out (and I pointed out last year) the
> obvious conclusion is a recursive model with any number of layers of
> locator scope.

HAIR has other issues, but let's go there tomorrow :-).  But yes you
can add another level.  The big question there is, if you have enough
of a scaling problem that another level of indirection looks
attractive, is that the right solution?  Perhaps just sticking with
the one level of indirection and using other means to reduce
rate*state is better.  TBD.  First let's get some experience with one
level (or none, depending on how things work out).

> GSE/ILNP and HIP are deeply different in that they genuinely
> separate a unique *non-locator* ID from one or more locators. Trying
> to use the phrase "loc/id split" to cover both of these models is
> certain to lead to confusion.

This seems to be a good time to reiterate a rant.  The names of things
("it's a locator! No, it's a floor wax!") don't matter nearly as much as

  * what they name
  * the scope of that naming
  * how they are used by the functions performed at various points

ILNP and GSE identifiers name devices.  They might name network
attachment points.  They are not guaranteed to be globally unique but
they are carried in every packet and are used by forwarding in the
destination network.  They only way they are not locators is that they
are not topologically aggregatable into forwarding directives, so
routing doesn't use them.  HIP HIs are globally unique but are not
carried in every packet (HITs are).   They are not used by routing or
forwarding anywhere.

I don't think we can divide all of this up into simple categories like
"locator" or "identifier".  We can say "it's a locator" as shorthand for
"a forwarding function uses it as a forwarding directive" but we almost
_always_ get into trouble when we do so.

By the way there are three categories of identification functions that
are relevant here

  * access authentication
  * initial location of a correspondent
  * session authentication and control

Enough mumbling for the moment.  Your point about LISP EIDs is correct
but we have to be careful about trying to classify everything into just
two buckets.

Scott

---

From: Lan Wang <[email-masked]>
Date: Wed, 10 Dec 2008 20:51:23 -0600

> 15% of the hosts on the Internet are.  Did you see any of that?

The traffic traces we use have been anonymized (so we don't know the  
actual destinations).   If there are lots of bogus destinations (from  
the crap packets), they will create more mapping entries at the ITR  
than necessary and probably increase the load at the default  
mappers.  In the worst case, this is similar to a cache overflow  
attack.  There have been proposed counter-measures for that kind of  
attacks (see the paper ""Pollution Attacks and Defenses for Internet  
Caching Systems", L. Deng, Y. Gao, Y. Chen, and A. Kuzmanovic, http:// 
networks.cs.northwestern.edu/publications/cache_JCN.pdf).

Lan

---

From: Robin Whittle <[email-masked]>
Date: Sun, 21 Dec 2008 03:07:37 +1100

Hi Noel,

In your message of 5 December, you wrote some things about map-encap

> it's not; etc, etc. All sounds very familiar, doesn't it? :-)

In Ivip, the ITR only has a single ETR address to send packets to.
The ITR is not involved in reachability testing.

It is up to the end-user network to do reachability testing and to
change the mapping of their micronet to whatever ETR is working now.
  In practice, they will probably pay another company to do this -
and that company will change the mapping of their micronets if one
ETR becomes unreachable, so the packets are sent to another ETR which
is reachable.

The resulting mapping change is propagated to all full database query
servers within a few seconds.  They propagate the change, within a
fraction of a second, to every ITR which recently requested and
received mapping for any address affected by the change.  ("Recent"
means within the caching time specified in the mapping response.)

I don't think the ITR -> ETR aspect of Ivip, LISP, APT or TRRP
constitutes a "new packet-switching system".  These proposals involve
introducing a new architectural system in the routing and address
systems.

 - Robin