Subject: Re: [rrg] Summary of architectural solution space

From: "William Herrin" <[email-masked]>
Date: Thu, 4 Dec 2008 10:08:42 -0500

On Mon, Nov 24, 2008 at 4:21 PM, Brian E Carpenter
<[email-masked]> wrote:
> On 2008-11-25 02:03, William Herrin wrote:
>> 2. In strategy B, locator assignment is dynamic.
>
> Why is that a necessary property?

Because administrative management of of the locator assignments is too
slow to react and too manpower intensive. The strategy is founded on
renumbering to achieve topological aggregation of the locators. IMO,
the only way such a strategy can succeed is if the administrative cost
of renumbering is zero or so close to zero as makes no difference.

> If that *is* a necessary property
> of strategy B, then you have a missing strategy which is essentially
> B with administratively assigned locators

Fair point. This will be strategy H in the fourth draft.

The fourth draft is now online. Do you or anyone else have additional
comments on http://bill.herrin.us/network/rrgarchitectures.html before
I close the document to additions and move on to exclusions?

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: "Teco Boot" <[email-masked]>
Date: Thu, 4 Dec 2008 17:36:42 +0100

Hi Bill,

Some comments on rrgarchitectures.html.

On Strategy G:
>> Make everybody attach to only one ISP using IPv6 and the ISP's single set
of PA addresses, as originally intended for IPv6.
Can you provide a xref for this statement?
RFC1887 describes 4 solutions for Multi-homed Routing Domains (section 4.4).
It seems that users like the solution described in 4.4.1 (PI addresses) and
thus we are facing the disadvantage:
>> The disadvantage of this approach is that since the IPv6
   address for that organization has no relationship to the addresses of
   any particular TRD, the TRDs to which this organization is attached
   will need to advertise the prefix for this organization to other
   providers.  Other providers (potentially worldwide) will need to
   maintain an explicit entry for that organization in their routing
>> tables.

On Strategy B and H:
I think the difference of the two strategies is minor. Make methods A2a and
A2b for these?
And on the criticisms:
With B: Why would LOCs be constantly in flux? Fixing this should be part of
the strategy.
With H: I think renumbering has succeeded sometimes also. I would agree it
could be costly and / or error prone.

Teco.

---

From: "William Herrin" <[email-masked]>
Date: Thu, 4 Dec 2008 12:20:25 -0500

> RFC1887 describes 4 solutions for Multi-homed Routing Domains (section 4.4).

Hi Teco,

I was being sarcastic and a little condescending. I should probably
rewrite it more like RFC 1987 4.4.3  and move the sarcasm to the
criticisms section.

RFC 1887 4.4.1 is strategy F: do nothing.

RFC 1887 4.4.2 is obviously not functional with *currently deployed
network stacks*. It would need the kind of software described in
strategy B or H in order to be viable. Or do you disagree?

RFC 1887 4.4.3 turns into strategy G once you actually fill in the
details. Hence my condescension.

RFC 1887 4.4.4 is knuckleheaded. You want to avoid a dependence on one
ISP so you get another ISP who won't work except by sending traffic
through the first ISP? Seriously? It doesn't belong in the solution
domain.

> I think the difference of the two strategies is minor. Make methods A2a and

I vacillated on that for a while and eventually came down on the side
of calling it a different strategy. I think if the difference was
truly minor, IPv6 would have worked out a whole lot better than it
has.

> With B: Why would LOCs be constantly in flux? Fixing this should be part of

Because link state changes on the nearby upstream path are satisfied
by renumbering instead of rerouting. When I said "dynamic," I really
really meant it.

Actually, that's not strictly true. You'd reroute for a few minutes,
long enough to be sure the state change wasn't ephemeral. Then you'd
renumber and on completion discontinue the exception route. That's
because renumbering looks like a distance-vector protocol; I can't
think of a way to describe it as a faster link-state protocol.

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Thu, 4 Dec 2008 19:52:31 +0100

Hi Bill,

/ RFC 1887 4.4.2 is obviously not functional with *currently deployed
/ network stacks*. It would need the kind of software described in
/ strategy B or H in order to be viable. Or do you disagree?

It depends.

This solution needs SA routing towards DFZ (like my BRDP Based Routing
proposal). Maybe lack of easy_to_manage_PBR / automatic PBR is the main
reason that this option is not used that much. Easy to fix!!

As mentioned earlier, I experience renumbering each day. And I *am*
reachable. So examples of "kind of software" is available and massively
deployed.
www.ietf.org / www.irtf.org are accessible using PA addresses:-). Hosted
servers can provide high availability. And it is not that difficult to make
servers PA multi-homing. I see two concerns:
 1: Support for changing ISP (multi-homing with slow renumbering)
 2: Support for session continuity when link to ISP fails 
Only 2 is hard to fulfill without upgrading host stacks. But I doubt if many
of us need this. 

I do not think mobility should be handled by the routing system. We have
protocols for this (MIP6).

So I would say: some of us need an upgraded network stack. We have already
two solutions (HIP & Shim6) (use your sshagentd for this?).
Problem here is, solutions need updated stack on corresponding node.

/ > On Strategy B and H:
/ > I think the difference of the two strategies is minor. Make methods
/ A2a and
/ > A2b for these?
/ 
/ I vacillated on that for a while and eventually came down on the side
/ of calling it a different strategy. I think if the difference was
/ truly minor, IPv6 would have worked out a whole lot better than it
/ has.

Maybe this can be corrected. 
Do you really think the Internet would work a whole lot better if we go for
this strategy?
I am with you!!

/ > And on the criticisms:
/ > With B: Why would LOCs be constantly in flux? Fixing this should be
/ part of
/ > the strategy.
/ 
/ Because link state changes on the nearby upstream path are satisfied
/ by renumbering instead of rerouting. When I said "dynamic," I really
/ really meant it.
/ 
/ Actually, that's not strictly true. You'd reroute for a few minutes,
/ long enough to be sure the state change wasn't ephemeral. Then you'd
/ renumber and on completion discontinue the exception route. That's
/ because renumbering looks like a distance-vector protocol; I can't
/ think of a way to describe it as a faster link-state protocol.

DV can distribute triggered updates quite fast. BRDP supports this. In some
cases, DV is faster than LS.

When a link is flapping, why using the prefix that comes along with it? We
need metrics & hysteresis for address assignment and SA selection, just as
we have in routing. All part of BRDP (better described in next versions). In
a MANET, this is mandatory for obvious reasons.

Teco.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 05 Dec 2008 09:37:54 +1300

> On Thu, Dec 4, 2008 at 11:36 AM, Teco Boot <[email-masked]> wrote:

...
> truly minor, IPv6 would have worked out a whole lot better than it

The difference is very much related to the practicality of automatic
(i.e. non-administered) renumbering. Which is why renumbering still needs
work. We (the authors) will revise draft-carpenter-renum-needs-work-00.txt
shortly, so comments are still welcome. I'm thinking we should be
clearer about the distinction between automated and administrative
actions in the draft. If we can show that some aspects of renumbering
are intrinsically impossible to automate, some classes of solution
will vanish for sure.

   Brian

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 00:44:24 -0500

Hi Teco,

If it's conditionally functional and the conditions aren't met then
it's not functional. Right?

> Do you really think the Internet would work a whole lot better if we go for

In my opinion, IPv6 layer 3 is salvageable intact for a strategy B
system, but layer 4 (TCP, UDP) will prove to be inadequate. Time will
tell.

Regards,
Bill Herrin

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 00:44:46 -0500

On Thu, Dec 4, 2008 at 3:37 PM, Brian E Carpenter
<[email-masked]> wrote:
> If we can show that some aspects of renumbering

Brian,

I've been playing with that question for a while from a clean-slate
perspective. Given a system dependent on map servers and certain
choices about the layer-4 structure, there don't seem to be any LOC
assignment issues that are especially hard to automate, not even in
the core.

On the other hand, renumbering in TCP/UDP networks? Fugeddaboutit.

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Fri, 5 Dec 2008 10:09:21 +0100

Hi Bill,

|> / RFC 1887 4.4.2 is obviously not functional with *currently deployed
|> / network stacks*. It would need the kind of software described in
|> / strategy B or H in order to be viable. Or do you disagree?
|>
|> It depends.
|
|If it's conditionally functional and the conditions aren't met then
|it's not functional. Right?

Not agreed at all!!
Why block a good solution for certain use cases, because there are (few?)
other use cases that cannot use it?
Why force an imperfect solution that has (strong?) disadvantages for some
(few?) use cases?

IMHO some of us are hunting for the holy grail. Let's keep in mind that the
first *small* steps of a migration to a new infrastructure MUST provide
benefits (Thaler - draft-iab-protocol-success).

By the way, I think most conditions against (IPv6) renumbering are caused by
violations on BCPs. I have strong objections against dictating a less
optimal architecture that works in all cases, including the BCP violations.

I think the new routing architecture should span both host / edge based
solutions (using PA addresses, supporting host based multipath transport)
and core based solutions (taking PI prefixes out of FIB of core routers).
They are orthogonal.

|> / I vacillated on that for a while and eventually came down on the
|side
|> / of calling it a different strategy. I think if the difference was
|> / truly minor, IPv6 would have worked out a whole lot better than it
|> / has.
|>
|> Maybe this can be corrected.
|> Do you really think the Internet would work a whole lot better if we
|go for
|> this strategy?
|> I am with you!!
|
|In my opinion, IPv6 layer 3 is salvageable intact for a strategy B
|system, but layer 4 (TCP, UDP) will prove to be inadequate. Time will
|tell.

It is happening right now. The Internet has more and more mobile nodes. Many
applications support lossy links and ever changing IP addresses. The TCP/IP
model itself hasn't a session layer, but there is absolutely no reason not
to implement something like that. We have SIP for good reasons. SIP provides
session management spanning multiple hosts!!! Just an example.

By the way, the term SID could be somewhat misleading. Lets distinguish
transport layer ID (connection ID) from session ID.

Teco.

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 08:28:54 -0500

> Why force an imperfect solution that has (strong?) disadvantages for some

Teco,

I think you might be missing my point. I wasn't making a value
judgment on the reasonableness of using multiple addresses in order to
multihome. I was simply observing that the users have spoken and what
they've said is that the currently available tools and protocols are
unacceptable.

Multiple-PA multihoming is conditional on a number of factors which,
in current practice, aren't true. So if we want a multiple-PA solution
like strategy B or H, its necessary to rearrange things so that those
conditions -are- true. Saying "go use it" when there's not an adequate
toolset to go use is really no different than the "do nothing"
strategy.

Anyway, if you really want to rehash the "why's" in detail, that's a
subject for another thread.

> transport layer ID (connection ID) from session ID.

Would you expand on that? I'm thinking of a session as an ID that
conceptually speaking, lasts from the TCP SYN to the TCP FIN. Is there
a better definition of session? What's the definition of transport
layer ID?

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Fri, 5 Dec 2008 18:07:12 +0100

Hi Bill,

|> |> / RFC 1887 4.4.2 is obviously not functional with *currently
|deployed
|> |> / network stacks*. It would need the kind of software described in
|> |> / strategy B or H in order to be viable. Or do you disagree?
|> |>
|> |> It depends.
|> |
|> |If it's conditionally functional and the conditions aren't met then
|> |it's not functional. Right?
|>
|> Not agreed at all!!
|> Why block a good solution for certain use cases, because there are
|(few?)
|> other use cases that cannot use it?
|> Why force an imperfect solution that has (strong?) disadvantages for
|some
|> (few?) use cases?
|
|Teco,
|
|I think you might be missing my point. I wasn't making a value
|judgment on the reasonableness of using multiple addresses in order to
|multihome. I was simply observing that the users have spoken and what
|they've said is that the currently available tools and protocols are
|unacceptable.
|
|Multiple-PA multihoming is conditional on a number of factors which,
|in current practice, aren't true. So if we want a multiple-PA solution
|like strategy B or H, its necessary to rearrange things so that those
|conditions -are- true. Saying "go use it" when there's not an adequate
|toolset to go use is really no different than the "do nothing"
|strategy.

I say "go on with strategy B / H".
I am not saying users should have used PA multi-homing, even if there was no
adequate toolset.
If something is missing, we can fix it.

|Anyway, if you really want to rehash the "why's" in detail, that's a
|subject for another thread.

I just want the "Summary of architectural solution space" to be neutral and
to be based on facts.

|> By the way, the term SID could be somewhat misleading. Lets
|distinguish
|> transport layer ID (connection ID) from session ID.
|
|Would you expand on that? I'm thinking of a session as an ID that
|conceptually speaking, lasts from the TCP SYN to the TCP FIN. Is there
|a better definition of session? What's the definition of transport
|layer ID?

For transport protocols, the correct term is connection.
A TCP connection is identified by a pair of sockets (RFC793).
One could say UDP is connectionless.

In the TCP/IP model (e.g. RFC1122), there is no concept of a _session layer_
nor _session_.

Although there are IETF protocols that act on a session layer (e.g. SIP),
for discussion on TCP (and even UDP) I prefer the term connection. Use CID
for this?

(Shim6 uses ULID: Upper Layer ID).
(SIP: Recently, there is I-D on Session-ID posted, with xref to UUID
(RFC4122). In this proposal, IP addresses are NOT used for Session_IDs)

Teco.

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 13:47:55 -0500

> I am not saying users should have used PA multi-homing, even if there was no

Then where do we disagree?

> One could say UDP is connectionless.

Connection isn't the right term since communication could well consist
of a single stateless query and the response to that query. It could
even consist of a unidirectional burst of packets notifying the
receiver of some event.

Reading between the lines, it sounds to me like your real complaint is
that the word "session" could take on unintended connotations from
SIP.

I'll ask the crowd on that. Should I subsititue "transport ID" or
something like that to make it clear that we're not talking about
SIP-like sessions?

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Fri, 5 Dec 2008 20:10:08 +0100

Hi Bill,

|-----Oorspronkelijk bericht-----
|Van: [email-masked] [mailto:[email-masked]] Namens William Herrin
|Verzonden: vrijdag 5 december 2008 19:48
|Aan: Teco Boot
|CC: RRG
|Onderwerp: Re: [rrg] Summary of architectural solution space
|
|On Fri, Dec 5, 2008 at 12:07 PM, Teco Boot <[email-masked]> wrote:
|> I say "go on with strategy B / H".
|> I am not saying users should have used PA multi-homing, even if there
|was no
|> adequate toolset.
|
|Then where do we disagree?

It started with comments on for example "LOCs are constantly in flux?"
And that renumbering "consistently failed in the past".
On both, I cannot agree.

|> |> By the way, the term SID could be somewhat misleading. Lets
|> |distinguish
|> |> transport layer ID (connection ID) from session ID.
|> |
|> |Would you expand on that? I'm thinking of a session as an ID that
|> |conceptually speaking, lasts from the TCP SYN to the TCP FIN. Is
|there
|> |a better definition of session? What's the definition of transport
|> |layer ID?
|>
|> For transport protocols, the correct term is connection.
|> A TCP connection is identified by a pair of sockets (RFC793).
|> One could say UDP is connectionless.
|
|Connection isn't the right term since communication could well consist
|of a single stateless query and the response to that query. It could
|even consist of a unidirectional burst of packets notifying the
|receiver of some event.
|
|Reading between the lines, it sounds to me like your real complaint is
|that the word "session" could take on unintended connotations from
|SIP.
|
|I'll ask the crowd on that. Should I subsititue "transport ID" or
|something like that to make it clear that we're not talking about
|SIP-like sessions?

My understanding re-reading the summary: It depends on the context one have
in mind.
If a session ID is based on addresses in flux, there are problems and the
summary describes just that.
One could write down session IDs shall not be based on temporary addresses
and we are done.
This is not that easy for transport IDs, those have addresses as part of
their ID.

Add "transport" in the introduction and rename SID to TSID?
>>  2. a component of the transport / session identity (TSID)

Teco.

|
|Regards,
|Bill Herrin
|
|
|--
|William D. Herrin ................ [email-masked]  [email-masked]
|3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
|Falls Church, VA 22042-3004

---

From: "William Herrin" <[email-masked]>
Date: Fri, 5 Dec 2008 14:37:49 -0500

Hi Teco,

For strategy B they fluctuate as renumbering (instead of rerouting)
resolves nearby link failures. For strategy H changes are more
planned.

> And that renumbering "consistently failed in the past".

On that we'll have to agree to disagree. I have too much operations
experience to consider renumbering as implemented now and in the past
as anything but an unmitigated failure.

At any rate, that's why the comment is in the criticisms section.
Whether you consider it valid or not, past operational experience with
host-level renumbering will form a major criticism for strategies
based on host-level renumbering. Proponents of those strategies should
expect to say, "This is why it didn't work out in the past and here's
what we've done about that in our new protocol."

> Add "transport" in the introduction and rename SID to TSID?

I'm not opposed to it, but I'd like to see if anyone else wants to
weigh in first. Does Transport Session IDentity (TSID) imply a more
clear purpose to which that identify is put than just calling it a
SID? Or Communication Session IDentity (CSID)? Or is just plain old
Session IDentity clear enough?

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Sat, 6 Dec 2008 12:42:34 +0100

Hi Bill,

|> It started with comments on for example "LOCs are constantly in flux?"
|
|For strategy B they fluctuate as renumbering (instead of rerouting)
|resolves nearby link failures. For strategy H changes are more
|planned.

OK, this is B & H as separate strategies.
For me, H is not acceptable. I work on solutions for ad hoc networking.
Ad hoc not equals planning.

|> And that renumbering "consistently failed in the past".
|> On both, I cannot agree.
|
|On that we'll have to agree to disagree. I have too much operations
|experience to consider renumbering as implemented now and in the past
|as anything but an unmitigated failure.

OK, I agree that *current* IPv6 is not able to fulfill needs for automatic
renumbering.
But who said that the required enhancement is not part of the approach?
I say solutions MUST support ingress filtering, as ingress filtering SHOULD
be there.
So it is part of the approach. You could specify so, if you want. It is
already in Autoconf PS (6.1 R11) and RFC5220 (2.1.2).
And supported by BRDP Based Routing.

By the way, I think the firewall criticism applies to strategy A as well.
In split load scenarios, firewalls see part of the traffic and would block
flows.
See also http://www.ietf.org/proceedings/08nov/slides/behave-14.pdf slide 31
for a problem description.
Maybe approach B performs the best, as (transport) connections would be
symmetric.

|At any rate, that's why the comment is in the criticisms section.
|Whether you consider it valid or not, past operational experience with
|host-level renumbering will form a major criticism for strategies
|based on host-level renumbering. Proponents of those strategies should
|expect to say, "This is why it didn't work out in the past and here's
|what we've done about that in our new protocol."

Fully agree. This is why I put some cycles in BRDP / BRDP Based Routing.

Teco.

---

From: "Teco Boot" <[email-masked]>
Date: Tue, 9 Dec 2008 08:58:25 +0100

Hi Bill,

Another question on the summary: Which strategy matches the "8+8" class of
solutions?

Regards, Teco

---

From: "William Herrin" <[email-masked]>
Date: Tue, 9 Dec 2008 10:06:27 -0500

Hi Teco,

Strategy A (A4e), B or H depending on the details of the plan. Six/One
router, for example, is A4e. A variant the modified the host so it
understood its session identity component to only reside in the lower
8 would fall under B or H depending on how the routing was organized.

As I wrote the document, I found to my surprise that the 8+8
approaches didn't really fit a category unto themselves but rather
worked as variants of several others.

Regards,
Bill Herrin

---

From: "William Herrin" <[email-masked]>
Date: Fri, 12 Dec 2008 00:01:36 -0500

> I think the difference of the two strategies is minor. Make methods A2a and

Turns out you were right about this...

The fifth draft is now available at
http://bill.herrin.us/network/rrgarchitectures.html

Major changes include:

Strategy H is folded in to strategy B as variant B1c. Original
strategy B is B1a.
Anja's loose source routes (from HAIR) is now B1b.

So, here's my master plan. I'm playing it by ear so this can still
change, but this is what I'm thinking:

Step 1. Where we are right now. In step 1, I'll make additions based
on this criteria: You propose something architecturally different than
what's already there and I understand it well enough to form a mental
picture of how that would work. I don't have to agree; I just have to
understand.

If I can get to a draft where there are only cosmetic changes from the
last one, I'm going to "close" the document to additions. I put close
in quotes because if something genuinely different and obviously valid
comes along I'll still add it but that'll be it for anything more
minor.

Step 2. I ask each of you to propose eliminating one or more of the
strategies or variants on the grounds that they are in some way
unworthy of further attention. If I get a strong consensus (meaning
lots of you say yes and no more than a couple say no) then I'll remove
that strategy.

Step 3. I'll ask for weak consensus on forming IETF working groups to
pursue specific strategy variants. Weak consensus means lots have
something to say and more agree than disagree. I'll mark them
accordingly in the document. My hunch is that two or three will pass
this hurdle, but we'll see when the time comes.

Step 4. I'll reformat the draft as an I-D and offer it to the group as
Bill Herrin's recommendation. I'll propose that it become the RRG's
recommendation to the IETF and be published as an informational RFC.

Beyond that, it's in your hands. If you find some other, better basis
for forming a consensus recommendation, great. If not, well, it'll be
your call.

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Fri, 12 Dec 2008 09:16:16 +0100

Hi Bill,

BRDP and supplementary mechanisms support:

B1d. Semi-hierarchical numeric LOCs are dynamically assigned. Local
reconnection during link state changes is accomplished with rerouting and /
or renumbering.

I think this variant is missing, and it is an important one (e.g. ad hoc
networks).

FYI: BRDP supports other strategies as well.

Just repeating myself on "session identity (SID) used by the layer 4". Layer
4 is OSI terminology, and this is NOT the session layer. Counting TCP/IP
model layers: the fourth is the application layer. Please reword the
introduction and I am fine.

Teco.

|-----Oorspronkelijk bericht-----
|Van: [email-masked] [mailto:[email-masked]] Namens William
|Herrin
|Verzonden: vrijdag 12 december 2008 6:02
|Aan: RRG
|Onderwerp: Re: [rrg] Summary of architectural solution space
|
|On Thu, Dec 4, 2008 at 11:36 AM, Teco Boot <[email-masked]> wrote:
|> On Strategy B and H:
|> I think the difference of the two strategies is minor. Make methods
|A2a and
|> A2b for these?
|
|Turns out you were right about this...
|
|The fifth draft is now available at
|http://bill.herrin.us/network/rrgarchitectures.html
|
|
|Major changes include:
|
|Strategy H is folded in to strategy B as variant B1c. Original
|strategy B is B1a.
|Anja's loose source routes (from HAIR) is now B1b.
|
|
|
|So, here's my master plan. I'm playing it by ear so this can still
|change, but this is what I'm thinking:
|
|Step 1. Where we are right now. In step 1, I'll make additions based
|on this criteria: You propose something architecturally different than
|what's already there and I understand it well enough to form a mental
|picture of how that would work. I don't have to agree; I just have to
|understand.
|
|If I can get to a draft where there are only cosmetic changes from the
|last one, I'm going to "close" the document to additions. I put close
|in quotes because if something genuinely different and obviously valid
|comes along I'll still add it but that'll be it for anything more
|minor.
|
|Step 2. I ask each of you to propose eliminating one or more of the
|strategies or variants on the grounds that they are in some way
|unworthy of further attention. If I get a strong consensus (meaning
|lots of you say yes and no more than a couple say no) then I'll remove
|that strategy.
|
|Step 3. I'll ask for weak consensus on forming IETF working groups to
|pursue specific strategy variants. Weak consensus means lots have
|something to say and more agree than disagree. I'll mark them
|accordingly in the document. My hunch is that two or three will pass
|this hurdle, but we'll see when the time comes.
|
|Step 4. I'll reformat the draft as an I-D and offer it to the group as
|Bill Herrin's recommendation. I'll propose that it become the RRG's
|recommendation to the IETF and be published as an informational RFC.
|
|Beyond that, it's in your hands. If you find some other, better basis
|for forming a consensus recommendation, great. If not, well, it'll be
|your call.
|
|Regards,
|Bill Herrin
|
|
|--
|William D. Herrin ................ [email-masked]  [email-masked]
|3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
|Falls Church, VA 22042-3004
|_______________________________________________
|rrg mailing list
[email-masked]
|https://www.irtf.org/mailman/listinfo/rrg

---

From: "William Herrin" <[email-masked]>
Date: Fri, 12 Dec 2008 09:53:51 -0500

> I think this variant is missing, and it is an important one (e.g. ad hoc

Is it fully dynamic (routers and subnets are dynamically assigned in
some kind of hierarchy on up to the core) or is it a combination of
dynamic and administrative (LAN addresses are dynamically assigned
while router addresses and subnets are fixed)?

If the former, what makes it different from B1a? Or to put it another
way, could B1a be reasonably rewritten to include it?

If the latter, what makes it different from B1c?

> introduction and I am fine.

Your point is well taken. I'm still looking at this one.

Another viewpoint would be to stop inaccurately describing TCP as a
layer-4 protocol and start inaccurately describing it as a layer 4/5
protocol. I used "session ID" because the plain-english meaning is
appropriate.

Regards,
Bill Herrin

---

From: "Teco Boot" <[email-masked]>
Date: Fri, 12 Dec 2008 19:55:05 +0100

Hi Bill,
|> B1d. Semi-hierarchical numeric LOCs are dynamically assigned. Local
|> reconnection during link state changes is accomplished with rerouting
|and /
|> or renumbering.
|>
|> I think this variant is missing, and it is an important one (e.g. ad
|hoc
|> networks).
|
|Is it fully dynamic (routers and subnets are dynamically assigned in
|some kind of hierarchy on up to the core) or is it a combination of
|dynamic and administrative (LAN addresses are dynamically assigned
|while router addresses and subnets are fixed)?
|
|If the former, what makes it different from B1a? Or to put it another
|way, could B1a be reasonably rewritten to include it?

B1a mentions "supernet" upstream and "subnet" downstream. This is definitely
not the case in B1d. In B1d, the IGP can route for any assigned prefix in
the edge network. Failures in the path to the BR that corresponds with the
SA is solved by rerouting.
Rerouting could be used if one of a set links to an ISP fails, or one of a
set middleboxes fails. Renumbering is needed when no path to an ISP is
available anymore. Smooth renumbering (using metrics) is supported to take
access to ISP out of service, or path becomes less optimal, e.g. when
wireless links are used.
The model has 2 layers: the path is inner-edge or edge-BR-core-BR-edge.
Edge-core traffic is allowed (e.g. ICMP).

|If the latter, what makes it different from B1c?

I am not sure what is meant with "administratively assigned". In the B1d
strategy, nothing in the edge needs manual configuration. The BR (part of
edge network, attached to ISP) gets prefixes from ISP with some protocol
(e.g. DHCP-PD or AAA). All routers autoconfigure addresses (e.g. for
management) and get prefixes. Routers come and go. This is the ad hoc type
of operation, to be addressed in Autoconf.

On rewording B1a: Maybe some proposals suggest hierarchical prefix
delegation. This is very different from the PD with IGP approach. If B1a is
reworded for PD with IGP, one could say a variant of a strategy is missing
(again...).

Teco.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 17 Dec 2008 11:25:56 +1300

Bill,

Under strategy B you say:

> 1. Probably not compatible with UDP or TCP 

I think you mean "not compatible with UDP or TCP sessions
surviving all topology changes". There's no incompatibility
as such unless the address pair changes.

(I hate to sing the same song again, but this is exactly
the issue that shim6 fixes, for the subset of hosts that chooses
to implement it.)

Oh, and you could add as criticism 4:

4. Unrealistic for IPv4 due to address depletion.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 17 Dec 2008 11:35:59 +1300

Bill,

On 2008-12-12 18:01, William Herrin wrote:
...
> lots of you say yes and no more than a couple say no) then I'll remove

I don't think that "remove" is the right thing to do. I think that such
strategies should continue to be documented in a "roads not followed"
section of the draft, with a succinct explanation. Otherwise, you can be
sure, they will come back again and again.

> this hurdle, but we'll see when the time comes.

Wouldn't you also want to see at least one proof-of-concept proposal
for each? We have plenty of those around, so enumerating the ones that
match each strategy variant would probably make the weak consensus
more explicit.

> recommendation to the IETF and be published as an informational RFC.

This seems like a reasonable series of steps to me.

   Brian

---

From: "William Herrin" <[email-masked]>
Date: Thu, 18 Dec 2008 21:26:27 -0500

On Tue, Dec 16, 2008 at 5:35 PM, Brian E Carpenter
<[email-masked]> wrote:
>> strategies. If I get a strong consensus then I'll remove
>> that strategy.
>

Hi Brian,

I concur. Rather than delete them, I'll move them to a section titled
something along the lines of "rejected strategies," with perhaps
sentence or two composed from the choicer comments.

> match each strategy variant would probably make the weak consensus

For the moment I want to err on the side of being inclusive. If that
yields a consensus to move too many things forward, I'll back up and
try a different approach.

Also, bear in mind that I won't accept proposals in the form "Let's
recommend a LISP working group." Rather, I'll ask folks to specify,
"let's recommend a working group for a Strategy A system based on
variants 1,2,3." That the variants were selected to match LISP is
fine. And if the resuling WG's first order of business is to adopt
LISP rather than consider other map-encap protocols with the same
criteria, that's fine too.

Regards,
Bill Herrin

---

From: "William Herrin" <[email-masked]>
Date: Thu, 18 Dec 2008 21:46:40 -0500

Hi Brian,

I meant the more general way. Authors of strategy B solutions should
be prepared to speak to this point. If they didn't abandon TCP and
UDP, how *did* they manage to cover the multihoming corner cases? Or
did they just abandom some of them?

Especially on the UDP side of things, Shim6 has more than a few
scenarios in which it can't re-establish the session during an address
change event. And it suffers ambiguities with protocols like FTP and
HTTP that use more than one TCP connection to complete their
communication session.

One could argue that Shim6 is at best an incomplete strategy B system.
It fails to select a legitimate B2 variant. It isn't B2a because once
the original address is given up as a locator, the GUID presented at
layer 6 is no longer globally unique.

I don't have better answers given Shim6's constraints. Rather I see
that as part of the steep price of maintaining TCP and UDP
compatibility while adding multiple address multihoming. For better
answers, I rope the DNS into the system design and let the name serve
as the GUID.

> 4. Unrealistic for IPv4 due to address depletion.

In one sense that's correct. In another, it's demonstrably untrue:
anyone with two commodity broadband connections today is holding at
least one public IP address from each. Do you really expect that
product offering to disappear completely in the next two years merely
because of the address shortage? Become a by-request offering sure,
but disappear?

Let me play devil's advocate for a moment: if we add the class E
addresses to the RFC1918 pool and go with something along the lines of
B1b, we could make the 32-bit addresses last a long, long time. Those
formerly private IP addresses could be reused for each global exit
that is the first hop of the loose source route.

Strategy B has some real hidden power to it.

Regards,
Bill Herrin

---

From: Robin Whittle <[email-masked]>
Date: Mon, 22 Dec 2008 01:38:44 +1100

Hi Bill,

I will write in a separate message about fine-tuning your page:

  http://bill.herrin.us/network/rrgarchitectures.html  (Fifth Draft)

to properly cover Ivip.

Here are some thoughts on your Item 1: "Routing table size problem,
root cause:".  This explanation seems to assume a lot of knowledge of
why multihoming, TE and portability in the current system creates an
undesirable number of BGP advertised prefixes.

I think that the real problem is that with current techniques, there
is a serious mismatch between the number (millions or in the future,
perhaps hundreds of millions or billions) of end-user networks which
want multihoming, TE and/or portability and the only way this can
currently be provided - by globally routed prefixes for every such
end-user network, which every DFZ router needs to handle in its RIB
and generally in its FIB.

It is not open to us to say the problem lies in millions of end-user
networks wanting multihoming, TE and/or portability.  So the problem
is that the current architecture can't scale to meet this demand.

That is the guts of the problem.  It is only one interpretation of
the problem to see it in terms of IP addresses being used for too
many purposes all at once, which is your page's current definition of
the root cause of the scaling problem.

There are potential host-based solutions - with new protocols and
changes to stacks and applications - which pushes the responsibility
for multihoming, TE and portability to each host, by way of
separating out the various functions into physically separate
entities, such as for GUID, SID and LOC.  But that doesn't mean the
only or best definition of the problem is in these terms.

Such a solution necessarily involves every host in fussing around
with changes in the core of the Net.  I see many problems with this:

  Fundamental objections to a host-based scalable routing solution
  http://www.irtf.org/pipermail/rrg/2008-November/000271.html

An alternative viewpoint to your current "root cause" problem
definition is that the current use of IP addresses and the relatively
stable expectations hosts have of them, is just fine.

In this viewpoint the root cause of the routing scaling problem is
the current limitations of the BGP system on its own.  So the core of
the Internet needs a new architectural enhancement so it can provide
multihoming, TE and portability to millions or billions of end-user
networks, while meeting the current expectations of host stacks and
applications that an IP address remains stable, including during
multihoming service restoration.

Such architectural enhancements don't necessarily involve any
locator-ID stuff - for instance some major souping up of BGP in
principle could do the trick - but this is agreed to be impractical.

It so happens that (in my opinion) the best way to do it is with a
map-encap, or map-forward, or whatever, which does do some locator-ID
stuff internally, although invisibly to the hosts.  In this model,
the problem is not in the hosts, or the way they use IP addresses.
It is in the limited capacity of the BGP system to provide
multihomable, portable, prefixes to a billion or so end-user
networks.  LISP, APT, Ivip and TRRP all attempt to provide this
number of prefixes for end-user networks, by means of an additional
architectural layer over BGP.

  - Robin

---

From: "William Herrin" <[email-masked]>
Date: Sun, 21 Dec 2008 17:32:31 -0500

> and generally in its FIB.

Hi Robin,

I think I follow what you're saying. My answer is that the potentially
millions or billions of folks who want/will want to multihome is one
cause of the scaling problem, but it's not the root cause. The root
cause is the described design defects in the protocols which prevent
such multihoming from occurring in a resource-light manner.

If the folks who want to multihome were the root cause of the problem
then the solution would obviously be, "Tough luck." That would be
precisely the opposite of productive, so that can't be the correct
root cause.

Regards,
Bill Herrin

---

From: Robin Whittle <[email-masked]>
Date: Mon, 22 Dec 2008 10:38:29 +1100

Hi Bill,

I wasn't suggesting that the large number of end-users who want
multihoming etc. was the problem.  I suggested the problem was a
mismatch between this number and the scaling limits the BGP based
routing system, which is the only current method of providing this.

That problem could be seen in at least two ways:

  1 - That the current system's scaling limits are the problem.

      Assumes: The routing system should be responsible for
      providing multihoming, TE and portability.

      Solution: Soup up or replace the routing system so it can
      cope with the larger numbers.  (Since we can't replace
      the BGP system, and no-one thinks it could be souped up
      to scale to hundreds of millions of prefixes, we have
      the network-based core-edge separation schemes, primarily
      based on map-encap: LISP, APT, TRRP and Ivip, although in
      the long term, Ivip does not use encapsulation.)

  2 - That current host functionality is deficient, since hosts
      themselves are incapable of providing multihoming, TE and
      portability.

      Assumes: 1 - The routing system is not capable of, or should
                   not be required to, provide these things, at
                   least to such numbers of end-user networks.

               2 - Hosts should be required to do this for
                   themselves.

      Solution:  Change Internet protocols, involving changes to
                 host stacks, APIs and applications so that hosts
                 can provide all this, without burdening the
                 existing BGP-based routing system.

Your current definition of the root cause of the routing scaling
problem:  http://bill.herrin.us/network/rrgarchitectures.html (5th)
is entirely along the lines of 2 above.  However the underlying
assumptions are not stated.

I disagree with the assumptions on which 2 is based.  (I also think
it will be impossible to attract enough end-users to any such system
which involves application changes.)

I think the proper place to handle multihoming, TE and portability
for end-user networks is in the "routing system" - that is in the
DFZ, in ISP routing systems and in at least one of the routers in
end-user networks.  (Not, as Noel discussed, in routers at the edge
of end-user networks.) I wrote about this in:

Adopting the 2 assumptions and solution will burden all hosts with
extra complexity, greater reliance on fast and reliable
communications to and from each host etc. to provide these things.

Yet multihoming, TE and portability are all things which concern the
operators of end-user networks, and are not of any direct interest to
hosts at all.

Multihoming involves real-time response to network outages.  This is
nothing to do with hosts and has traditionally been handled by the
routing system.

Inbound Traffic Engineering is also a network-based, real-time,
concern.  The network operator wants to balance loads across multiple
links, handle some kinds of packets on one link or the other for
cost, QoS, or some other policy reasons.  This has nothing to do with
hosts.  Hosts can't easily determine what the total load of packets
is across any one link, or what other factors the network operators
consider important when choosing which packets should come in via
which links.

Portability of address space is the only way a network can change
ISPs without the excessive costs, disruption etc. of renumbering.
This is never going to change, since the addresses turn up in all
sorts of places, including in config files of hosts, routers etc.
well beyond the network itself.

These things are best controlled from a few levers directly operated
by the network administrator.  I think the mechanisms should be in
the routing system too - and should not involve hosts at all.

Implementing the actual mechanisms in hosts would require the network
administrator to be sure that all hosts have the right mechanisms and
respond to the network administrator's desires.  Yet it is
commonplace for hosts which are not owned by, or controlled by, the
end-user network operator to connect to these networks in an ad-hoc
manner, such as plugging in a laptop, or using a Wi-Fi link for a PC,
Wi-Fi phone etc.

A further problem is the complexity and extra packets which must flow
back and forth to hosts in order to do this work.  This would be a
serious burden on any small device, significantly increasing the
complexity, costs, operating currents etc of the simplest possible
host.

Over a slow, expensive, unreliable, radio link the extra traffic
required for hosts to perform these functions would be a real
problem.  The current unreliability of radio links impacts service
quality directly.  In addition, if hosts had to do multihoming, TE
and portability within themselves, radio link delays and
unreliability for packets concerning these functions would further
degrade the final service quality.

I am completely opposed to host-based solutions to the routing
scaling problem.  I think the current  "root cause" problem assumes
the problem is in the hosts, and therefore implies that the solution
should lie there - despite the fact that some of the solutions are
network-based, rather than host-based.

I could write some new words for your page, but you would probably do
a better job of writing with the terseness I think you desire.

  Regards

  - Robin

---

From: "William Herrin" <[email-masked]>
Date: Sun, 21 Dec 2008 19:25:54 -0500

Hi Robin,

I don't see it. I describe deficiencies with the protocol where
functionality improperly overlaps. I make no claims in item #1 about
where in the system the problem should be solved or whether tackling
it head on is the best choice.

If you feel that the language is biased towards host-level changes,
I'm open to making small tweaks to remove that bias. What would you
suggest?

Regards,
Bill Herrin

---

From: Robin Whittle <[email-masked]>
Date: Mon, 22 Dec 2008 15:18:45 +1100

Hi Bill,

I do think the current statement is biased towards seeing the problem
as a deficiency in hosts, and therefore towards solving the problem
in the hosts too.

> I'm open to making small tweaks to remove that bias. What would you

I can't think of a small tweak which also presents the problem from
the point of view that multihoming, TE and portability (or some
alternative method of painless ISP choice) can and should be provided
by the network, rather than by hosts.

My attempt to rewrite your section is to insert the following text
between the "Item 1: Routing table size problem, root cause:" heading
and your current text.

I make certain statements about RRG consensus, based on my perception
 of this.  I guess all the stuff I suggest below needs to be
evaluated and debated.

  Regards

    - Robin

The routing scaling problem involves addressing and the provision of
multihoming, inbound traffic engineering (TE) and portability of
address space for end-user networks.

"Portability of address space" is one term for the problem of not
being able to select another ISP, without the unreasonable costs and
disruption of having to renumber the network.  Currently, the only
solution to this problem is portability, in the form of one or more
PI prefixes.

An alternative position is that this problem can and should be solved
by making it easy to quickly and reliably renumber networks.

A critique of this is that automated renumbering is unlikely to ever
meet the needs of end-user networks, due to factors such as:
addresses appearing in config files which are hard to identify and to
securely and automatically change; addresses appearing in various
forms in hosts and routers outside the network; stability problems in
any network undergoing any kind of renumbering; and difficulties
securely and reliably testing renumbering, without causing disruption.

No techniques come close to making "painless renumbering" possible
for IPv4, and attempts to achieve this for IPv6 are still far from
sufficient to convince most administrators of end-user networks of
any significant size that automated renumbering and PA space suits
their needs.

The routing scaling problem we are trying to solve is due to a
mismatch between the increasingly large numbers of end-user networks
which want or need multihoming, TE and/or address portability and the
capacity of the current Internet architecture to provide this.

At present, the only way of providing any of these, with either IPv4
or IPv6, is for each end-user network to have one or more PI prefixes
for each of its one or more sites.  The current interdomain routing
system, which uses BGP, requires every DFZ router to maintain state,
and engage in communications with its neighbours, for every
advertised prefix, including each PI prefix of any end-user network.
  Generally, the FIB also needs an entry for each such PI prefix too.
  Most concern is about the scaling difficulties of the BGP control
plane, rather than the FIB of each router.  The scaling problem is
due to the increasing number of end-user network PI prefixes placing
unacceptable demands on CPU and RAM resources.  Furthermore, the BGP
network is widely regarded as becoming increasingly unstable and
unresponsive to network outages, as the number of such prefixes grows.

Also, the practice of achieving TE by dynamically changing the
advertisement of these prefixes places burdens on potentially all DFZ
routers.

In addition to the technical concerns just mentioned, there is a
fundamental problem of the beneficiaries of the increasingly large
number of end-user PI prefixes (the end-user networks themselves) not
paying for the costs they impose on others: every ISP which runs one
or more DFZ routers.

There is consensus in the RRG that the desires and needs of end-user
networks for multihoming and TE are reasonable.  However, opinions
vary as to how much there is a need these in the smallest networks,
such as a residential or SOHO service via DSL / cable modem / fibre /
WiMax etc. or a mobile device, via wired Ethernet and one or more
radio links.

There is less consensus that the desire or need for portable address
space is reasonable, but it is clear that many or most substantial
end-user networks need freedom of choice of ISP, and that they
believe there is no method of achieving this other than portable
address space.

There is little consensus in the RRG on how many end-user networks
will want or need multihoming, TE and portability in the future, but
it is agreed that a scalable routing solution, to be successful,
should scale to very large numbers of end-user networks, such as
hundreds of millions.  There is a widely held belief that the
scalable routing solution should ideally scale well to a situation
where billions, perhaps 10 billion, separate mobile devices each
constitute an end-user network, with multihoming, TE and address
portability - though this scenario could not develop in IPv4.

There is no clear consensus that a scalable routing solution needs to
appeal to every type and size of end-user network.  For instance, if
some of the larger networks continued with their current PI
practices, the scaling problem could probably be solved as long as
most or all of the medium sized and smaller networks adopted the new
system.  A critique of this is that the new system needs to be
attractive, in the short term, for the vast majority of end-user
networks of all sizes, for reasons including it being necessary to
attract smaller end-user networks which aspire to become "large" in
the future.  So a perception that the scalable routing solution is
not suitable for some class of end-user networks, such as the
largest, will deter many other smaller networks from adopting it.

There is complete consensus that we have no power to force adoption
of the scalable routing solution - and that we need to make it
attractive in the short term, for solving end-user networks'
immediate needs, irrespective of how concerned they are about
worsening or solving the routing scaling problem.

There is no consensus on how rapidly IPv6 will be adopted, or about
the urgency in solving the routing scaling problems for IPv4 or for
IPv6 - which is a long way from having such a problem.  Ideally, one
system would be optimal for both Internets.  However, there are
significant differences between the two networks which might result
in differences between the optimal solutions for each.

The statement of the problem can be developed further in at least two
generally incompatible ways - by locating the problem in the routing
system (1), or in the hosts (2).

1 - That the mismatch is due to a deficiency in the interdomain
routing system which can and should be corrected.  This is based on
the position that it is impractical and/or undesirable to expect
hosts to implement multihoming, TE and address portability (or
something equivalent which makes it easy to choose another ISP).

Reasons for this position include: not wanting to increase the
minimum complexity of hosts; the belief that multihoming, TE and
portability are responses to network-centric changes, and that they
should be handled in the "network" (DFZ, ISP routers and end-user
network PE routers), rather than in hosts or end-user network edge
routers; concern about the extra traffic costs for mobile devices due
to the extra management traffic if hosts have to implement
multihoming, TE and portability; and concerns about further delays
and robustness problems with user communications caused by an
increasing dependency on unreliable carriage of this management
traffic, particularly over mobile radio links.  There are at least
two approaches to defining the problem in this "network" sense, each
with its own implied solution:

1a - The problem is the BGP system's inability to handle millions, or
perhaps billions, of PI prefixes.  Therefore, the solution is to
improve BGP, or to replace it with something else which can scale
well to such numbers.  However, there is complete consensus that
firstly there is no potential for improving the BGP system
significantly in this respect and secondly that there is no prospect
for introducing a replacement for BGP without prohibitive levels of
disruption.

1b - The deficiency is in the interdomain routing system itself, due
to it currently consisting only of the BGP system.  Consequently, the
solution is to progressively augment the BGP system with another
architectural system which handles many, most or ideally all end-user
networks, providing them with portable address space suitable for
multihoming and TE in a manner which burdens the BGP system very
lightly per end-user network prefix, or ideally not at all.  This
gives rise to the core-edge separation class of scalable routing
solutions.  These are mainly based on map-encap (LISP, APT and TRRP).
 One core-edge separation scheme, (Six-One Router, for IPv6 only)
uses translation, and another (Ivip) uses different types of
forwarding for IPv4 and IPv6, using modified DFZ routers, with the
option of introducing the system with map-encap if it must be
introduced before all the DFZ routers can be upgraded.

2 - That the mismatch is not due to a deficiency in the interdomain
routing system, but due to the current architecture not expecting
hosts to provide multihoming, TE and portability on their own, with a
routing system similar or identical to today's BGP system.

The argument from this point of view is that it is more desirable for
hosts to provide the functionality required for end-user network
multihoming, TE and portability, than to add a new architectural
layer to the interdomain routing system.

The solutions suggested by this point of view involve changes to host
stacks and usually to APIs and applications.  These solutions
generally work only for a modified form of IPv6, of for a clean-slate
design with fresh protocols - which means that a mass migration to
this new Internet needs to be achieved before such time as the
routing scaling problem in the IPv4 Internet becomes "unacceptable",
however this is defined.

These solutions aim to equip all hosts with the capabilities to
communicate reliably - especially in the presence of failures of
links to one of the multiple ISPs - while using two or more
addresses, each drawn from two or more PA prefixes provided by two or
more upstream ISPs.  This class of solutions solves the routing
scaling problem by ensuring that all participating end-user networks
- ideally all such networks of all sizes - are able to meet their
needs for multihoming, TE and easy selection of new ISPs (termed
"portability" in the alternative viewpoint which considers this can
only be achieved with portable address space) with the only prefixes
being handled by BGP being those of the ISPs.

A description of the problem in these terms is:

---

From: Robin Whittle <[email-masked]>
Date: Mon, 22 Dec 2008 17:50:51 +1100

Hi Bill,

Thanks for your response on 24 November:

  http://www.irtf.org/pipermail/rrg/2008-November/000261.html

I will concentrate on the text in the current 5th draft.

The paragraph directly after "Strategy A." is applicable to Ivip.

A1a.  This doesn't seem to apply to any proposal I know of.

A1b.  This applies to Ivip and I guess to LISP, APT and TRRP.

A1c.  I don't clearly understand this.

A2a.  LISP NERD.

A2b.  This somewhat applies to Ivip, which is why I suggested:

>>   A2d.  GUIDs dynamically mapped to each RLOC are pushed towards a
>>         central or distributed registry as they change. The registry
>>         pushes all incremental changes in near-real time to all
>>         full database query servers in ISP and/or end-user networks.
>>         The encoders (ITRs) request mapping from these local
>>         query servers.  The response has a caching time and the
>>         local query server will push any changed mapping to an
>>         encoder if it receives such a change for mapping which
>>         matches a recent query which is still within its caching
>>         time.  There may be one or more full database query
>>         servers in each ISP and there may be one or more layers of
>>         caching query servers between these and the encoders.

You responded:

> Hi Robin,
> 
> When I read this, I see A2b in the large with some local
> optimizations. Let me ask others on the group to offer an opinion on
> this: is Robin's hybrid method described above distinctive enough from
> the other three mapping methods that it should be listed separately in
> the document?

There being no feedback to the contrary, as the Ivip designer I
assert that A2d. above is necessary to properly represent Ivip's fast
hybrid push/pull mapping system.

I don't think A2b. matches an existing proposal.  It is fast,
real-time, push to every ITR.  No-one is suggesting that this would
scale well.  So I would be happy if the text above replaced A2b.

A2c.  I think this matches LISP ALT and TRRP.

# Failure handling approaches include:
#
# Link failures in the Internet core cause the RLOCs to be rerouted
# with no change to the address-RLOC map.

OK - this applies to Ivip, as well as to LISP, APT and TRRP.

A3a.  I think this describes the approach of LISP, ALT and I guess
      TRRP to reachability testing and multihoming service
      restoration.  It involves the ITRs ("RLOC encoders") having
      multiple RLOCs to choose between, doing their own reachability
      testing and making their own decisions about which RLOC to use.

The previous A3b didn't suit Ivip, so I suggested:

>>    A3c. End-user networks are responsible for controlling the
>>         mapping of their EID address space.  Each EID prefix -
>>         in the case of Ivip, not necessarily a prefix, since it
>>         can be one or more contiguous IPv4 addresses or IPv6
>>         /64s - is mapped to a single RLOC address.   For reasons
>>         such as a link failure, to implement inbound Traffic
>>         Engineering, or to implement address portability when
>>         moving to another ISP, the end-user network causes
>>         the mapping to change to the new RLOC address, and this
>>         is conveyed to all full database query servers in
>>         near real time.  These push the changes to any encoders
>>         (ITRs) which may need them, based on previous queries and
>>         the caching times of the responses.
> 
> This is really what A3b is all about.
> 
> I'll adjust it to:
> 
> A3b. Link failures which prevent parts of the RLOC's network from
> reaching a destination host or set of hosts it serves cause an
> external analysis element to make a dynamic change to the address-RLOC
> map, depreferencing or removing the affected RLOC.
> 
> Better?

It doesn't really describe Ivip's approach in sufficient detail.
There is no "deprefencing" of an RLOC, since Ivip only has one RLOC
in the mapping.  With a real-time hybrid push/pull mapping
distribution system, there is no reason to have multiple RLOCs and
the ITR choosing between them for multihoming.

This raises questions in the reader's mind about how TE is achieved,
which is what my suggested text also describes.

Here is another attempt.  I think it is a bad idea to avoid mention
of ITRs and the name of the proposal etc. but I am trying to follow
this style.

    A3b. End-user networks are responsible for controlling the
         mapping of their EID address space.  Each EID prefix
         (or non-prefix, contiguous, range of address space: one or
         more contiguous IPv4 IPv6 /64s) is mapped to a single RLOC
         address.   For reasons such as a link failure, to implement
         inbound Traffic Engineering, or to implement address
         portability when moving to another ISP, the end-user network
         causes the mapping to change to the new RLOC address, and
         this is conveyed to all full database query servers in
         near real time.  These push the changes to any RLOC encoders
         which may need them, based on previous queries and the
         caching times of the responses.

>> On second thoughts, A4d is not a good description at all.
>>
>>   A4f. The encoder encapsulates all packets which are short enough
>>        that once encapsulated, they will be no longer than some
>>        constant N, where it is assured that the MTU between all
>>        encoders (ITRs) and ETRs is at least N.  Longer packets
>>        are either encapsulated and sent, or rejected with a
>>        packet too big message, depending on how their length
>>        if encapsulated, compares to two variables the encoder
>>        maintains for each particular RLOC (ETR) address.  The
>>        encapsulated packets function as probes, and depending
>>        on whether the encoder receives PTBs for them, it
>>        adjusts its two variables.  These represent the
>>        encoders upper and lower limits to its uncertainty about
>>        the true MTU to this RLOC.
> 
> I think you were right the first time: this is a fancy version of A4d
> but architecturally its still A4d.

OK, the current A4d is a terse description.  Folks will need to refer
to http://www.firstpr.com.au/ip/ivip/pmtud-frag/ to see how Ivip does
this.  I think no other proposal currently incorporates proper
handling of the PMTUD problems caused by encapsulation.

>>  A4h  For IPv6, all DFZ routers are upgraded to recognise an
>>       alternative format of the IPv6 header in which 19 or
>>       20 bits are used to determine the packet's forwarding.
>>       These bits map to a predefined set of advertised prefixes
>>       of ISP networks, and so the packet is delivered across the
>>       DFZ, to a border router of the correct ISP network, without
>>       PMTUD problems.  If there is more than one ETR at that
>>       network, a second lookup will be required and a similar, or
>>       different, mechanism internal to the ISP network will be
>>       required to forward the packet to the correct ETR.
> 
> Whoops, I missed that one. How about:
> 
> A4f. The IPv6 flow label or some other component(s) of the IPv6 header
> are used to contain the RLOC. The flow label is set before the packet
> enters the core. Non-local packets are routed based on the flow label.
> IPv4 is abandoned.

This, your current A4f, doesn't describe Ivip's Prefix Label
Forwarding at all:  http://www.firstpr.com.au/ip/ivip/ivip6/

There is no concept of abandoning IPv4 in this.

The full RLOC can't fit in the IPv6 header, but 19 bits of it can,
which is sufficient to forward the packet across the DFZ.

I would appreciate it if you replaced the current A4f with the text
above, converting "ITR" and "ETR" into other terms, if you like.

> A4g. Steal bits from other functions in the IPv4 header (e.g.
> checksum) to make space for an RLOC. Discard those components and set
> the RLOC when the packet enters the core. Restore the original bits
> when the packet leaves the core. Use another strategy for IPv6.

That is OK if you want to keep it so terse that it is only a cursory
explanation.  Ideally, people would know this means:

  ETR Address Forwarding (EAF) - for IPv4
  http://tools.ietf.org/html/draft-whittle-ivip4-etr-addr-forw

It is somewhat more complex than this, since there are 30 bits
available, rather than 32, and the system does not handle fragments,
or DF=0 packets longer than some TBD length.

>> I think it would be helpful to include pointers to actual proposals.
>>  I understand you want to use a more generalised language to describe
>> the various proposals, but for the benefit of anyone who is not
>> totally up with RRG discussions (or who is, but is tired), I think it
>> would help to have such pointers.
> 
> I'm trying to keep this document higher level than that. I think a
> second document which maps from the strategies to the proposals and
> vice versa would be very helpful.

OK - When it is finalised I will write a message trying map your
descriptions to the proposals as best I can.

Regarding your assessment of the Strategy A:

#  Major criticisms:
#
#  There don't appear to be any genuinely clean ways of implementing
#  strategy A.

"Genuinely clean" could mean almost anything.

A major characteristic of Strategy A your page doesn't mention is
that it can be introduced smoothly, without requiring changes in
hosts.  With Ivip's OITRDs, LISP's PTRs and with APT's and TRRP's
similar arrangements, it can provide multihoming, TE and address
portability in a scalable fashion, including for all packets sent
from hosts in networks without ITRs.  So they can, in principle,
provide immediate and significant benefits to the end-user networks
which adopt the new kind of address space each proposal requires.
Doing this without disruption, cost to others, and without host
changes anywhere seems pretty "clean" to me compared to trying to
convince people to adopt a new system which only provides benefits
when they upgrade their hosts, and applications - and then only for
communications with other  hosts with these upgrades and new apps.
(Also, those host-change proposals generally only work on IPv6, so
you have to coax everyone over to an Internet where they can't
contact all the hosts they can with IPv4.)

# Handling path-MTU is a usually problem since the packets in the
# core are different than the origin host would recognize.

I think it is a solvable problem:

  http://www.firstpr.com.au/ip/ivip/pmtud-frag/

but those concerns, and the need for protocols, ITR and ETR functions
to support this etc. do not exist in Ivip's two "Forwarding" approaches.

I think it may be easier to upgrade the DFZ routers with firmware
updates, and replace any which can't be upgraded (I would think they
are probably all be firmware upgradeable in these respects) than to
develop and implement the encapsulation and PMTUD stuff.

# Extra bandwidth is consumed by the ITR figuring out whether the ETR
# is still available and functioning.

This a problem for LISP, APT and TRRP - but not at all for Ivip.  The
reachability testing system of the end-user network (probably run by
some company they hire to do this) does whatever testing the end-user
wants, and makes decisions in which ever way they require, without
involving ITRs at all.

# Border filtering of source addresses becomes problematic. It's a
# mess.

This is a problem for LISP, APT and TRRP, but not at all for Ivip.
Ivip's "forwarding" modes work normally with source address
filtering, for instance at the border routers of ISPs, when checking
packets arriving from the DFZ.   Likewise for Ivip's encapsulation
modes, since the outer header source address is that of the sending
host.  The filtering happens naturally and normally at the ISP border
routers - and this is extended to the inner packet by the simple
process of the ETRs dropping any packet which has an inner source
address which is different from its outer source address.

# Deployment may require heavy weight "for the public good" relays in
# the non-upgraded part of the Internet to facilitate migration.

This is a reference to Ivip's OITRDs (Open ITRs in the DFZ) or LISP's
PTRs (Proxy Tunnel Routers).  The LISP team seem to have no feasible
business plan for this, but there is a good Ivip business plan which
doesn't at all rely on charity, someone paying for the benefit of
others etc.

  Business incentives for LISP PTRs and Ivip OITRDs  2008-07-29
  http://psg.com/lists/rrg/2008/msg02021.html

The OITRDs for any MAB (Mapped Address Block) are paid for by the
RUAS (Root Update Authorisation System) organisations which operate
each MAB and rent parts of each MAB out to end-user networks to use
as micronets.  The RUASes pay for the MABs by sampling their traffic
patterns and charging their end-user network customers according to
which network the traffic is destined to.

Ideally, I think, the document you are creating would contain:

1 - Fuller descriptions of the proposals, with the appropriate
    terminology, ETR, ITR, QSD, PTR, OITRD etc.

2 - Clear labelling of which proposals match which descriptions.

3 - Either a much more fully developed set of critiques,
    appraisals etc. than you have at present - or none at all.

The "Major Criticisms" of Strategy A do not apply to Ivip, yet Ivip
is one of the Strategy A proposals.

I haven't really followed Strategy B, but I think it involves hosts
implementing multihoming, TE and whatever takes the place of address
portability.  If you are going to include major criticisms, then I
think there should be some attempt to mention or link to the concerns
about pushing these responsibilities onto hosts.  I discussed this in:

  Fundamental objections to a host-based scalable routing solution
  http://www.irtf.org/pipermail/rrg/2008-November/000233.html

and today in the text I suggest you add to your problem definition:

  Summary of architectural solution space - problem definition
  http://www.irtf.org/pipermail/rrg/2008-December/000525.html

My view is that any proposal involving such host changes is a
non-starter firstly for the fundamental reasons mentioned in the
above messages and secondly because there is no way we can convince
enough end-user networks to adopt this, since benefits only accrue in
proportion to the number of other end-user networks they communicate
with who also adopt it.

Rewriting apps is just not going to happen on the scale which would
make this sort of thing attractive to most end-user networks.  This
latter argument also applies to any proposal which assumes there will
be widespread adoption of IPv6 without the need for IPv4 any time
soon.  See these messages from Thomas Narten and Keith Moore:

  http://www.ietf.org/mail-archive/web/ietf/current/msg54376.html
  http://www.ietf.org/mail-archive/web/ietf/current/msg54392.html
  http://www.ietf.org/mail-archive/web/ietf/current/msg54409.html

I think these Strategy B host upgrade schemes (such as ILNP) are
generally only intended for IPv6.

What are the proposals covered by Strategy B?

Strategy C is "replacing BGP with something different and better".  I
agree with your "Major Criticisms" that there is no proposal for this
which matches the needs of ISPs.  Nor is there any proposal which has
a chance of being introduced without a complete switch-over to a new
system - and there is 0% probability of this becoming feasible in the
foreseeable future.

Strategy D doesn't address the routing scaling problem as I
understand it.  FIBs are not the major problem - the RIBs and the
overall BGP distributed control plane is the problem, at least when
we consider using it for tens or hundreds of millions of prefixes.

Strategy E is probably impractical and does not provide the
multihoming etc. needed by tens or hundreds of millions of end-user
networks.  In point 2 of your critique, perhaps you can add, after
"users": "to have multihoming, TE and portable address space".

Strategy F doesn't solve the problem - but that is what we will do
until a lot of people become much more enthusiastic than they
currently are about one or more of the proposals!

Strategy G doesn't solve the problem either.

  Regards

    - Robin

---

From: "William Herrin" <[email-masked]>
Date: Mon, 22 Dec 2008 18:39:10 -0500

On Mon, Dec 22, 2008 at 1:50 AM, Robin Whittle <[email-masked]> wrote:
>> http://bill.herrin.us/network/rrgarchitectures6.html
>

A1a. the RLOC = the ISP's AS#

A1b. Just like A1a except each ISP gets two or three AS#'s that they
can use and they sort which GUIDs go under which RLOC and the priority
at which each RLOC is announced to each peer to optimize their
routing.

A1c. Each ISP gets a big block of locators, such as 251.1.0.0 -
251.1.255.255. They don't announce each one to the core; they announce
the aggregated set, such as 251.1.0.0/16.

I haven't really checked but I suspect most of the proposals these
days look like A1c. We can probably reject A1a and A1b on the grounds
that A1c has the same or lower implementation price and is obviously
better.

>         mapping of their EID address space.  Each EID prefix

I'll add the sentence, "The external analysis element may be under the
control of the end-user destination network, the RLOC network or a
third party under contract to one of them."

> this.  I think no other proposal currently incorporates proper

I'd prefer "concise," but I guess I'll settle for terse. :)

> There is no concept of abandoning IPv4 in this.

I'll replace "IPv4 is abandoned" with "Use a different A4 variant for IPv4."

> which is sufficient to forward the packet across the DFZ.

I don't follow the distinction. If what's in the header gets the
packet to the ISP and then the ISP does something internally to move
it to it's destination then that number in the header was the RLOC.

> hosts.

A4a. A4b. Proposals based on e and f may call for deprecating IPv4.

Tell you what, if A4c and A4d are the only compatibility variants to
survive the rejection phase, I'll add a note to the strategy A
description to the effect that host changes are not required.

>    terminology, ETR, ITR, QSD, PTR, OITRD etc.

This terminology only applies to specific strategy-A proposals and
only some of them. It does not apply, for example, to Six/One Router.

> 2 - Clear labelling of which proposals match which descriptions.

That's a job for another document.

> The "Major Criticisms" of Strategy A do not apply to Ivip

If correct, you're ahead of the game. You're still gonna have to sell it though.

Regards,
Bill Herrin

---

From: Robin Whittle <[email-masked]>
Date: Tue, 30 Dec 2008 15:47:59 +1100

Short version:  Discussion of variants of Strategy A.

                Further suggestions of text which would better
                describe the Ivip variants - though it may be
                too late to change the page now.

Hi Bill,

> A1a. the RLOC = the ISP's AS#

I don't clearly understand this.  Do you mean the ISP's Autonomous
System number is used by the existing BGP interdomain routing system
in some way which relates to addressing?  Can you give some examples
or point to any solutions which this describes?

# A1b. Each core ISP has a small number of RLOCs for traffic
#      engineering (TE). The RLOCs' existence and reachability is
#      flood-propagated to the rest of the core.

> A1b. Just like A1a except each ISP gets two or three AS#'s that
> they can use and they sort which GUIDs go under which RLOC and the
> priority at which each RLOC is announced to each peer to optimize
> their routing.

I don't interpret A1b in this way at all.  My interpretation is that
each ISP has one or more prefixes advertised in the DFZ, typically
one or more per site (such as the ISP's operations in a different
city or country).  All ETRs have addresses within these prefixes, so
amongst other things, all RLOC addresses are within these prefixes.

The numbers of these prefixes are something the BGP interdomain
routing system can cop with.  ISPs do inbound TE for their various
prefixes by changing how they advertise them to upstream ISPs.

Meanwhile, the Strategy A solution provides globally reachable
address space for most or all end-user networks (for multihoming,
inbound TE and portability) in a form which does not place such
burdens on the BGP interdomain routing system (the DFZ) as does the
only current way of doing it: individually advertising each such
end-user prefix in the DFZ.  Specifically, Ivip has a relatively
small number of Mapped Address Blocks, each of which is advertised in
the DFZ, but each of which provides thousands to hundreds of
thousands of separate micronets for end-user networks.

There is nothing in my description concerning AS numbers.  It is
basically the current DFZ, with the addition of Ivip and more and
more end-user networks using micronet address space to provide
multihoming, TE and portability.

> the aggregated set, such as 251.1.0.0/16.

OK - that is one way of doing my understanding of A1b.  The ISP
advertises as many prefixes as it needs for its sites to have the
degree of TE they require.

If most end-user networks are on micronet spaces, then these do not
need to be particularly large blocks of addresses, since the ISP will
 primarily use them for RLOCs (ETR addresses) - and presumably ETRs
won't need to be anywhere as numerous as the IP addresses of the
end-user networks.

Still, ISPs will probably have large blocks of address space, because
in addition to using this space for ETR addresses (RLOCs) and for
their own servers etc. they would also presumably still have large
numbers of customers who are not multihomed and are using PA
prefixes.  This includes especially the large number of residential
and small business customers who get a single IPv4 address via DSL,
cable modem or fibre.

> that A1c has the same or lower implementation price and is obviously

I don't see A1c as having any important architectural distinctions
from A1b.

> third party under contract to one of them."

OK.  Hopefully people will recognise this includes Ivip and that the
"depreferencing or removing the affected RLOC" is not a good
description of Ivip's approach, which is:

  To have only one RLOC in the  mapping information: the one that all
   ITRs should currently use, since it is the one chosen directly or
   indirectly by the end-user network, and sent in real time
   to all the ITRs which need this mapping information.

> I'd prefer "concise," but I guess I'll settle for terse. :)

OK, but there is more to it, such as being able to cope gracefully
with the transition to jumboframes in the DFZ, and the lack of
support for fragmentable packets longer than a certain length.

> I'll replace "IPv4 is abandoned" with "Use a different A4 variant for IPv4."

OK - for IPv6 there is a broadly similar system, but with some
important distinctions.

> it to it's destination then that number in the header was the RLOC.

It is an RLOC only across the DFZ as far as the ISP's border router.
 With the IPv4 approach:

the RLOC is the ETR address, so with upgraded internal routers, the
modified packet is forwarded as far into the ISP network as
necessary, without any further lookups, until it reaches the ETR.

A4f and A4g roughly describe my IPv6 and IPv4 forwarding approaches.

In A4f, the text:

  "The flow label is set before the packet enters the core. Non-local
   packets are routed based on the flow label."

is not quite right.  A better description would be:

  "Internal routers upstream of the RLOC encoder, and all DFZ
   routers, forward the packet according to this new RLOC address to
   an ISP border router which advertises the matching prefix.  There,
   the packet is either sent to a single RLOC decoder, or a second
   lookup is performed so the ISP can use its own methods of
   transporting the packet to the correct one of multiple RLOC
   decoders."

In A4g, the text:

  "Discard those components and set the RLOC when the packet enters
   the core. Restore the original bits when the packet leaves the
   core."

is not quite right.  This would be better:

  "Discard those components and set the RLOC when the packet is
   forwarded from the RLOC encoder [ITR].  Restore the original bits
   when the packet arrives at the RLOC decoder [ETR].  Upgraded
   routers between RLOC encoder and decoder - DFZ and internal
   routers - forward the packet according to the RLOC address
   rather than the destination address."

> A4a. A4b. Proposals based on e and f may call for deprecating IPv4.

OK - A4e (Six/One Router) involves abandoning IPv4, so this does
involve major host changes.  Apart from that, the map-encap
approaches (LISP, APT, Ivip and TRRP) and the forwarding approaches
(Ivip) are for both IPv4 and IPv6 and involve no host changes.  This
is a major distinction between Strategy A and Strategy B, which can
only be implemented with changing hosts to a modified form of IPv6,
with new host stacks, API and applications.

> description to the effect that host changes are not required.

OK - but the nature of the host changes is my primary argument for
why Strategy B should be rejected.  The difficulty of having such
changes developed and widely adopted is a secondary argument.

> only some of them. It does not apply, for example, to Six/One Router.

Yes - Six/One Router is the exception.  I am not sure how much
Christian is working on it now.  I understand your need to generalise
to cover all the approaches.

> That's a job for another document.

I will write a message to do this.

> If correct, you're ahead of the game. You're still gonna have to sell it though.

Those major criticisms don't apply to Ivip.  I am doing my best to
promote understanding of Ivip and to encourage critiques.

  - Robin

---

From: "William Herrin" <[email-masked]>
Date: Tue, 30 Dec 2008 00:50:37 -0500

On Mon, Dec 29, 2008 at 11:47 PM, Robin Whittle <[email-masked]> wrote:
> in some way which relates to addressing?

Something along those lines, yes.

> Can you give some examples

http://bill.herrin.us/network/ermpi6.html

> I don't interpret A1b in this way at all.

You interpret A1b the way I intended A1c. I probably got a little too
fancy with the language in A1 but I'm not quite sure how to correct
it.

Regards,
Bill Herrin