Subject: RE: Bad routes update

From: Richard Draves <[email-masked]>
Date: Tue, 20 Jul 1999 15:35:04 -0700

(I would like to move the main thrust of the discussion from the 6bone list
to the ipng list.)

Someone expressed the thought that multihoming is being swept under the
rug... I don't think so. We spent considerable time on the subject in Oslo
and I believe there was a consensus to plan an interim WG meeting with
multihoming being a major focus.

Perry said:
> Think of it from a systematic viewpoint. Is there actually a 
> way for a 
> multi-homed host both to get service from "two parts of the tree" and
> yet not announce that into the default free zone? Thinking about it
> for several minutes indicates "no".

I think giving the multi-homed host two addresses (one for each part of the
tree) is the beginning of a "yes" answer. It shifts the problem from the
network backbone (routing scalability) to the network periphery (when to use
which address).

Petri said:
> But having all TCP sessions reset when topology converges is 
> a showstopper for IPv6 deployment.

I was disappointed when there was no controversy around this point in
Oslo... There are a several arguments for why TCP session survival when
topology changes is not crucial:

- Most application use is for the web. Web TCP sessions are very short-lived
and if there is a problem, the user will just use stop/reload.

- Mission-critical applications already have logic for reliability above
TCP. (Transactions, restarting transfers, etc.) The Internet today is just
not reliable enough to rely entirely on TCP's guarantees for
mission-critical applications.

- BGP reacts/converges very slowly to topology changes. I've recently seen
data that indicates convergence usually takes minutes and there is a heavy
tail of very long convergence times. So even if you use a single prefix and
rely on BGP for fail over, you will often see poor results.

That said, one element of the Oslo proposal was an RFC 2260-style solution -
one benefit of that approach is survivability of existing TCP connections.

Rich
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: "Perry E. Metzger" <[email-masked]>
Date: 20 Jul 1999 18:56:12 -0400

> (I would like to move the main thrust of the discussion from the 6bone list

Fine with me.

> network backbone (routing scalability) to the network periphery (when to use

It won't work. It means that if the network routing to one of the
addresses dies, the TCP connections die.

> topology changes is not crucial:

Let me be really blunt.

Right now, I can happily leave a "cvs update" that takes several hours 
running over the nice v4 internet and feel happy knowing that no
matter what happens, the connection will keep on trucking.

If the new v6 internet doesn't work that well, it is broken from my
point of view. I didn't sign up on this to see my internet work less
well than it does right now.

> mission-critical applications.

Again, you just aren't dealing with the reality. Sure, lots of the net
is the web, but it isn't all, and you're essentially saying "network
reliability? we don't need that!" forever. I'm sorry, but having
*less* reliability just isn't acceptable.

People use this *now*. I don't think it is reasonable to break them.

Oh, and

> data that indicates convergence usually takes minutes

sure, but someday, we *will* have much better performing protocols in
place. I don't think it is reasonable to architect things to burn
today's limitations "into the wood", especially when, as I note, I use 
the net for long lived connections every day and I *don't* lose the
way you're telling me I will have to under v6.

Perry

---

From: [email-masked] (Sam Manthorpe)
Date: Tue, 20 Jul 1999 20:17:30 -0700 (PDT)

> matter what happens, the connection will keep on trucking.

If you lose connectivity for more than 10 mins for any BSD derived
stack you'll also lose your connection.  TCP isn't that reliable.
If site renumbering were done on a regular basis this could turn out
to be quite annoying, but that won't happen, will it.  Site renumbering
will be much rarer than your next-hop router going down for > 10 mins
and you losing your connection, IMO.

> People use this *now*. I don't think it is reasonable to break them.

Er, don't exactly the same multi-homing issues exist with v4 right now?
They just aren't an issue because TCP does not change its source and
destination addressing during a connection.  You're talking about a
TCP change rather than a v6 issue, aren't you?

I strongly urge leaving things alone right now.  This is really the
worst time to start talking about changes to the IPv6 API 
(i.e. connect("FQDN")), especially for a `problem' that has been lived
with for a very long time.  This is a completely orthogonal issue to
IPv6 IMO.

Just my .2 cents as an implementor.

-- Sam

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sam Manthorpe, SGI.  tel: (650)933-2856 fax: (650)932-2856  [email-masked]
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Peter Tattam <[email-masked]>
Date: Wed, 21 Jul 1999 13:48:39 +1000 (EST)

This may or may not be relevant to the discussion.

A few months ago, I put some ideas together based on the GSE debate.
I have been reluctant to publish it because of the possible ridicule it may
generate, but nothing ventured, nothing gained, so here it is.

http://jazz.trumpet.com.au/ipv6-draft/GSE-plus-routing-idea-july-21-1999.htm

It's only rough "ideas on the back of a napkin" style, not a rigorous
treatment of the subject, and there may be things that just won't work in it.
Of course the actual numbers I used in the document are flexible, and it should
be read with that in mind.

Anyway it's only as food for thought, and I bet noone will take it seriously.

Peter

--
Peter R. Tattam                            [email-masked]
Managing Director,    Trumpet Software International Pty Ltd
Hobart, Australia,  Ph. +61-3-6245-0220,  Fax +61-3-62450210

---

From: "Perry E. Metzger" <[email-masked]>
Date: 21 Jul 1999 09:30:47 -0400

[email-masked] (Sam Manthorpe) writes:
> stack you'll also lose your connection.

That's a lot better than losing every time the routing changes, which
is often multiple times in one day.

> Er, don't exactly the same multi-homing issues exist with v4 right now?

Not really. People who need multi-homing set up multi-homing and at
the moment the doctrine of "use only the provider God gave you because 
only a sicko would want to use multiple providers" hasn't yet been
burned in the wood.

I know a lot of organizations that do this and are much happier
because of it. People are saying that in v6 land they're going to have 
to "go screw" and I really don't think that's acceptable.

People keep saying "well, we'll engineer around it" but thus far, no
one has actually demonstrated real, deployable protocol to do so.

> TCP change rather than a v6 issue, aren't you?

As I said, right now, this works fine in v4. People announce routes
through two providers and everything is copacetic. The issue is that
people are trying to squash the practice in v6. They're claiming that
one must never do this.

If you guys insist on having people claim that they're bigger than
they are in order to do that, all that's going to happen is that
people will fight to pretend that they need bigger address blocks than 
they do.

Perry

---

From: Jim Bound <[email-masked]>
Date: Wed, 21 Jul 1999 09:34:46 -0400

>with for a very long time.  This is a completely orthogonal issue to

I agree, unless anyone is talking about changing the src/dst address
duriing a renumbering operation the connection will not break.

If we are talking about that change the subject line and start that
discussion.  

We do not need to fix changing src/dst addresses on the fly for
renumbering to deploy IPv6 whats so ever.

Please focus this discussion and lets not yell "wolf" here.

/jim

---

From: "Perry E. Metzger" <[email-masked]>
Date: 21 Jul 1999 09:38:58 -0400

Joe Abley <[email-masked]> writes:
> We routinely re-route inbound traffic through a backup carrier in order
> to do maintenance on our primary inbound (US -> NZ) network without
> customers being affected. It is not acceptable for all TCP sessions
> to break whenever we decide to re-route traffic for a couple of hours.

Lots of people are claiming this is just fine, and the fact that you
are using multiple providers is some sort of disease on your part.

I think they ought to start re-thinking this idea right now.

Multiple homing is existing IPv4 practice and is not going away.

> > You're talking about a
> Nope. We're talking about a route aggregation issue.

For reference, this discussion came up because some people started
setting up multiple tunnels to multiple "providers" on the 6bone, and
others started complaining about the "bogus routes" they were emitting 
-- i.e. more specific routes being announced via some provider to
another provider's address block, i.e. the existing v4 practice.

Lots of people are harboring the illusion that there is some way to
avoid this in v6. "We'll build new protocol!" they say. "We'll use IP
mobility algorithms!" they say. "We'll outlaw the practice!" they say.

Try any of those and you're going to fall flat on your face. The fact
is that this is a real internet with real problems that can't be
wished away via non-real protocols.

Unfortunately, the time for tinkering with v6 is over. You guys had
seven years to come up with protocols to get around the multi-homing
issue and you danced around it. Well, too late. Now you have to accept 
people advertising multiple routes to their multi-homed networks. If
you wanted to do something else, you should have done it over the last 
decade while you had the time and breathing room.

If we actually are going to deploy v6, we can't break existing users
or they'll go on the warpath. If we're going to actually deploy v6, we 
have to do it now.

That means that whatever magic multi-homing solutions people pull out
of the pot of engineering manna are going to have to come "someday"
and we're going to have to deploy using the existing solution now.

Unless, of course, people don't want to actually deploy.

Perry

---

From: Jim Bound <[email-masked]>
Date: Wed, 21 Jul 1999 10:02:23 -0400

>are using multiple providers is some sort of disease on your part.

Who exactly is claiming this is bad can we put names or organizations to
those who think this is a bad idea.

I don't see a problem with doing this and never did.

>I think they ought to start re-thinking this idea right now.

I agree with Perry.

>Multiple homing is existing IPv4 practice and is not going away.

Agreed.

>another provider's address block, i.e. the existing v4 practice.

So whats wrong with that we have to permit it and the routes are not
bogus if they work.

>mobility algorithms!" they say. "We'll outlaw the practice!" they say.

That is not going to work.

>wished away via non-real protocols.

Exactly.

>or they'll go on the warpath. If we're going to actually deploy v6, we 

Agreed.  This is not a problem and existing practice should work as we
did nothing in IPv6 to prevent how it is done in IPv4.

I won't go into the debate if we ignored the multihome problem because
we as designers did not, what we did is define an addressing
architecture that enhances the use of IX's IMO that makes fixing the
multihome problem somewhat less complex.

/jim

Perry

---

From: Joe Abley <[email-masked]>
Date: Wed, 21 Jul 1999 20:38:20 +1200

> stack you'll also lose your connection.  TCP isn't that reliable.

It's incredibly rare that I lose connectivity somewhere for long enough
that TCP sessions time out.

However, it's not _that_ rare for me to lose connectivity for a minute or
so due to a major network event.

The fact of the matter is that TCP usually _is_ that reliable. If we
start to dump all our customer's TCP sessions on a semi-regular basis,
the helpdesk will track me down and kill me while I sleep.

> Er, don't exactly the same multi-homing issues exist with v4 right now?

Nope.

> destination addressing during a connection. 

That's right.

> with for a very long time.  This is a completely orthogonal issue to

This is a completely new problem, and one that does not happen with
IPv4. This is a vital operational issue for us, and one which is surely
best debated (and experimented with) on the 6bone in preference to
any production v6 network. That's what the 6bone is for, right?

> Just my .2 cents as an implementor.

My NZ$0.02 as an operator.

Joe

-- 
Joe Abley <[email-masked]>      Tel +64 9 912-4065, Fax +64 9 912-5008
Te Kaihoahoa Kawei, CLEAR Communications Ltd      http://www.clear.net.nz/

---

From: [email-masked] (Sam Manthorpe)
Date: Wed, 21 Jul 1999 10:05:31 -0700 (PDT)

> to break whenever we decide to re-route traffic for a couple of hours.

Yes but you don't routinely renumber your site.  You just change routes
and that works fine in IPv6 as it does in IPv4.  

Or are you talking about moving your NAT box rom one net to another?

> > Er, don't exactly the same multi-homing issues exist with v4 right now?

So you can change the outgoing interface of your TCP connection,
together with the source address of your TCP 4-tuple and your
connection stays up?  Please tell me more!

Cheers,
Sam.

---

From: Joe Abley <[email-masked]>
Date: Wed, 21 Jul 1999 22:49:50 +1200

> Anyway it's only as food for thought, and I bet noone will take it seriously.

It looks a little like you are using a 32-bit BGP ASN-type-thing (the
visible portion of the globally unique site id) as a key to choose a
label for the datagram, so that it can be pushed through an MPLS
cloud-looking-thing. Doesn't the exchange and dispersal of routing IDs
sound like the label distribution protocol?

Perhaps MPLS is a general solution for the core, and will collapse
an arbitrary complex routing problem in the centre of the network into
a generally simpler mesh?

Is MPLS an alternative approach to avoiding routing collapse in the core?

Joe

---

From: [email-masked]
Date: Thu, 22 Jul 1999 02:29:22 +0900

Couldn't we just calm down until Oslo IETF minutes is made available?
	Most of the items are tackled in presentation there, of course with
	clarification of problem domains.
	Most of the emails we saw these days seems to not sharing clear
	problem domain.  Multihoming issue includes policy issues so problem
	domain can be very different by person to person.

	So, again, let us wait till minutes is made available.

itojun

---

From: Jim Bound <[email-masked]>
Date: Wed, 21 Jul 1999 13:44:17 -0400

I agree.  I was in the dark as I was not at Oslo.  Can we please listen
to Itojun on this one.  Then we can discuss it based on the draft and
also clearly articulate the problem space technically and political
issues as I think there are some.  Also I think Brian was telling us the
same thing.  How about we chill out till the draft is available.

/jim

---

From: Joe Abley <[email-masked]>
Date: Thu, 22 Jul 1999 09:13:15 +1200

> and that works fine in IPv6 as it does in IPv4.  

No, it won't. Since we are not a (p)TLA, the current rules forbid us to
advertise our prefixes to all our transit ASes. This effectively means
that TCP endpoints need to renumber to recover from a network event such
as that I described.

> Or are you talking about moving your NAT box rom one net to another?

No NAT here. We're a wholesale supplier of IPv4 transit to about twenty
ISPs, and have several hundred dedicated-access (i.e. permenantly-
connected) retail customers, and around 100,000 dial-up users.

> connection stays up?  Please tell me more!

No. I don't need to, since all my IPv4 transit ASes are happy for me to
announce the (relatively) long-prefix routes that describe all possible
TCP endpoints within my network to them.

No TCP endpoint renumbering or other bolt-on TCP stability techniques
required. That was my point.

Joe