Subject: RE: [arch-d] It's the routing, people!

From: [email-masked] (Noel Chiappa)
Date: Sat, 11 Nov 2006 20:03:18 -0500 (EST)

> From: "Tony Li" <[email-masked]>

    > Interestingly, multihoming was about 10% of the routing table in 1992,
    > as well. It would seem to me that if we could reduce the growth due to
    > .. multihoming, .. then we would have an impact on the growth rate.

You raise an interesting point.

I don't think there's any way in the routing alone to support multi-homing in
a way which reduces table growth (at least as long as the only scope we have
for advertisements, other than "local", is "global"). To support multi-homing
without impacting the routing, it seems that the way to go is have multiple
locators for each multi-homed entity (which is the way Multi6 eventually
went, with Shim6, but nobody likes the mechanism). So if we do, say GSE, how
do we make multi-homing work, using multiple addresses?

	Noel

_______________________________________________
Architecture-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/architecture-discuss

---

From: "Tony Li" <[email-masked]>
Date: Sat, 11 Nov 2006 17:13:38 -0800

> do we make multi-homing work, using multiple addresses?

In my mind, that's one of the primary benefits of any loc/id split.
Multiple locators per-site, with locator replacement at the border.
I've always thought of this as being one of the primary applications of
GSE.

Tony

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 13 Nov 2006 11:23:20 -0800

Good answer, Tony!!

Many Internet architects are apparently keen to burden us end users with
multiple addresses. To the extent that Multi6 and Shim6 do this, to that
extent they will be resisted for deployment into real life networks.
People who intend for us to physically assign different addresses for
each of our ISPs to each of our computers have not really understood the
primary end user addressing requirement: no readdressing for something
as unimportant as who our ISPs are.

There are administrative, management, and security implications of
multiple addresses that make them impractical for today's networks
(unless there is a compelling business reason to justify those overhead
expenses and risk). 

Shifting the burden from ISPs to the end user is just not going to cut
it. Why does anybody imagine that end users will willingly bear
significant operational expenses for a problem that isn't even visible
to us? ISPs are a fraction of our size -- and we are the customer!!
Unless a monopoly situation exists, why would anybody maintain business
relationships with an uncooperative ISP? 

Rather, a clean solution that is equally good for everybody has to be
found, which is why I personally have supported the loc/id split
approach for many years now. Alternatively, we can NAT.

---

From: "Schliesser, Benson" <[email-masked]>
Date: Mon, 13 Nov 2006 19:54:20 -0600

Fleischman, Eric [mailto:[email-masked]] wrote:
> multiple addresses.
> [...]
> [...]

I think this is an excellent point, Eric, and I fundamentally agree with
the sentiment. But I'm curious about your perception of multiple
addresses as burdensome. For instance, with some of the
auto-configuration options currently available this could be as simple
as adding a prefix to your gateways for each ISP that you want to
enable.

Is there an acceptable-burden crossover point somewhere between
"physically assign[ing]" addresses and the ideal zero-work "no
readdressing" scenarios?

> to us?

Well, here is my perspective: The scale problems in the global routing
system are economic externalities from the end network's perspective.
This may (probably?) eventually lead to relative rising costs for the
provider. Which would ultimately become a cost to the customer, though
perhaps not until the provider marketplace is turned upside down.

Given the options of 1) higher monetary costs for services, 2) higher
administrative costs for renumbering, or 3) the cost of adopting
whatever cool new technology we come up with, I'm betting on end
networks / customers choosing #3. (unless we totally screw up the
technology, I suppose)

> approach for many years now. Alternatively, we can NAT.

One might argue that loc/id split methods that don't require endpoint
involvement inherently require NAT. Though not all NAT methods are
created equal...

Cheers,
-Benson

---

From: Igor Gashinsky <[email-masked]>
Date: Tue, 14 Nov 2006 00:18:16 -0500 (EST)

On Mon, 13 Nov 2006, Schliesser, Benson wrote:

:: I think this is an excellent point, Eric, and I fundamentally agree with
:: the sentiment. But I'm curious about your perception of multiple
:: addresses as burdensome. For instance, with some of the
:: auto-configuration options currently available this could be as simple
:: as adding a prefix to your gateways for each ISP that you want to
:: enable.
:: 
:: Is there an acceptable-burden crossover point somewhere between
:: "physically assign[ing]" addresses and the ideal zero-work "no
:: readdressing" scenarios?

Because the problem here isn't just getting the address on the end host. 
It's getting all the ACLs/Firewalls configured with the new rules, 
adding those extra netblocks to loadbalancers, adding the routes all over 
the network (and filtered/leaked correctly), adding them into the 
application logic to recognize internal vs external, etc, and you want 
people to do that every time they want to add/remove/change ISP's? Talk 
about lock-in... 

Also, most people don't do dynamic addressing for their servers -- too 
many things that could break, and I'm not going to even going to bring up 
applications written 5-10 years ago with hardcoded IPs that everyone is 
afraid to shut down, much less change -- and that's just some of the 
burdens faced by quite a few enterprise networks (aren't you glad 
you don't work for them? I sure am...) which is why DHCP++ 
magic-auto-numbering stuff just doesn't work here.

Honestly, I think anything that's more complext then "adding statement 
XYZ on the border router(s)" is "too complex" for most enterprises (and 
those who provide services to SMB's know exactly what I'm talking 
about)...

Thanks,
-igor (do not work for an enterprise network, luckily)

----------------------+----------------------+------------------
Igor Gashinsky, CISSP | Network Architecture | Yahoo! Inc.
 [email-masked]   |  cell 917.807.2213   | Do You... Yahoo?
----------------------+----------------------+------------------

---

From: Jefsey_Morfin <[email-masked]>
Date: Mon, 13 Nov 2006 23:07:42 +0100

Dear Eric,
the solution is quite simple. Consider that you have three orthogonal 
concatenated segments in an IP address:

- the ISP address global (X.121, E.164, are example)
- your own address obtained to your favorite registrar
- your own extensions you manage by yourself.

The second address part is the one interesting you. You will keep it 
whatever the ISP (as you will keep your extensions the way you like). 
Now you change ISP (or you rotate ISP for security reasons, or move 
abroad, etc.): you can give your ISP address, or you can have it 
dynamically registered, etc. When you move around the world with your 
mobile, you keep your number, but you change prefixes. Because you 
need far less numbers, and can far better manage them, you have room 
enough for:

     the_type_of_address:the_network_address:the_user_global_address:your_extensions 

and you can easily think ways to have variable length numbers.

May be this iticipate in this, so  
much the better, but we shouldn't make this a requirement for succes.

---

From: Jefsey_Morfin <[email-masked]>
Date: Mon, 13 Nov 2006 23:07:42 +0100

Dear Eric,

May be this is why there are 2 billions mobiles in 10 years and no 
IPv6 in 14? But we just are lead users ...
all the best.
jfc

---

From: "Fleischman, Eric" <[email-masked]>
Date: Tue, 14 Nov 2006 12:35:19 -0800

Benson,

You've mentioned some pretty important concepts, many of which are about
topics that are outside of my visibility. The sole point about which I
can authoritatively speak is that entities are unlikely to willingly pay
financial or operational costs to solve problems about which they have
no visibility.

Concerning the topics you mentioned about which I have poor visibility:

1) We are only beginning to deploy IPv6 and so we don't yet have actual
operational experience concerning how destabilizing multiple addresses
are within huge operational networks. The problem from my knothole isn't
the automated assigning of multiple addresses (i.e., the issue that you
discussed) but it is rather accounting for them and managing devices in
vast networks having many duplicate addresses in a security-viable
manner. Network management is already far too difficult and expensive
without making it significantly harder. Consider, for example, the
increased burden to our intrusion detection systems (to mention only one
affected subsystem). Our experience is that we operationally have
significant problems whenever we have to ensure aligned configuration on
large numbers of devices from multiple vendors enterprise-wide. For
example, theoretically one can deploy many DiffServ PHBs, but in
practice that is not the case. With great difficulty we have deployed 5
PHBs within the enterprise and most networks significantly smaller than
us are only able to operationally deploy 4 (i.e., we have business
reasons to bear the expense of supporting 5 PHBs rather than the more
normal 4 for networks approaching our size).

2) I don't understand why you have concluded that ISPs will have
expenses that will bleed-through to us end users in any significant way
due to routing table explosion. To my simple mind, ISPs will perhaps
have to continually upgrade their Internet-facing ASBRs such that they
could not depreciate at the standard accounting rate but other than that
what are their actual expenses due to this problem? The greater threat
(to my simplistic mind) is that the routing table growth rate may exceed
the capabilities of the ASBRs, which will impact actual Internet
performance, but as long as that doesn't happen, this doesn't seem like
a very large expense to "pass through" to us. For example, depending on
their number of Internet-facing ASBRs, the overhead of their billing
system (for those that have one) could dwarf many possible worse-case
ASBR expense scenarios.

--Eric

-----Original Message-----
From: Schliesser, Benson [mailto:[email-masked]] 
Sent: Monday, November 13, 2006 5:54 PM
To: [email-masked]
Subject: RE: [arch-d] It's the routing, people!

> as unimportant as who our ISPs are. [...]

Cheers,
-Benson

---

From: Geoff Huston <[email-masked]>
Date: Wed, 15 Nov 2006 10:13:00 +1100

>financial or operational costs to solve problems about which they have

Generally pain (cost) and gain (benefits) need to be very directly 
correlated for a deregulated and diverse industry to pick up and run 
with the concept. This topic came up at the Amsterdam workshop and 
the folk there were certainly aware of this - i.e. getting some folk 
to spend time effort and money for somewhat vague and general 
benefits is a deep problem, and the routing space today is an 
eloquent expression of the observation that so far we haven't been 
able to make a routing "economy" work. (that does not mean that it 
cannot work, nor does it imply that efforts to look at ways to scale 
routing by altering the information model are not valid - they are 
both valid and useful activities imho, but adoption is in itself a 
separable issue that requires particular consideration in terms of 
identifying the cost and benefits that ensue.

Geoff

---

From: David Meyer <[email-masked]>
Date: Tue, 14 Nov 2006 15:24:30 -0800

> identifying the cost and benefits that ensue.

	Well said.

	--dmm

---

From: "Schliesser, Benson" <[email-masked]>
Date: Thu, 23 Nov 2006 16:31:20 -0600

(Sorry for my delayed response; too much travel, work, etc, has kept me
busy. I now get to use my holiday to catch up...)

> system (for those that have one) could dwarf many possible worse-case

Overall you are right about the proportion of costs in a service
provider today--administrative costs often predominate, and capital
expenditures tend to be lower than operational expenditures regardless
of the source.

But providers have been competing fiercely, and prices in (parts of) the
market are now approaching the fundamental cost of the service.
Providers are attempting to implement automation and leaner business
process, amongst other things, to reduce their costs further. (and
searching for value-adds to build their business on, since network
access clearly won't satisfy)

And if some of the predictions I've heard about routing system growth
versus Moore's law etc come true, then it's possible that the capacity
of new equipment will not be able to keep up with the growth of the
routing system, leading to a need for more equipment and/or relatively
more expensive equipment. Over time this will mean that prices have to
rise--there's nowhere else for the costs to be absorbed, that I'm aware
of.

Given how low market prices are today, and believing that administrative
costs will decrease relatively over time, I expect that even a small
increase in equipment costs could translate to a rather noticeable
increase in costs to customers. I'd be happy to be wrong, though. :)

Cheers,
-Benson