Subject: [mif] Some thoughts

From: William Herrin <[email-masked]>
Date: Mon, 9 Nov 2009 13:14:16 -0500

Hi Folks,

I'm just joining the list. I would join you at the meeting but I'm
about half a world away right now...

I've read the charter and the draft problem statement at
http://www.ietf.org/id/draft-ietf-mif-problem-statement-01.txt and
I've leafed through the list archives. I have some comments I'd like
to share as I try to get up to speed. Feedback would be helpful.

1. The draft seems pretty focused on end-hosts, that is systems which
originate or consume but don't forward IP packets.

Routers are hosts too and figuring out how to handle packets to
multiple upstreams in different administrative domains is important
there as well. Otherwise we limit ourselves to only the simplest case:
where no component larger than a single host is a member of two
networks. As often as not, that isn't the case. The entire local LAN
can be a member of two upstream networks and it'd be nice if entire
routed subsystems could be members of two networks.

Even if we don't want to address the router case explicitly, it might
be helpful to look at whether the router case offers any insight into
the grouping and relationship between the issues in the host case.

Source address selection, for example, could be viewed through a
routing lens: find the destination in the routing tables associated
with each of the hosts's source addresses, return a list of source
addresses for which the destination is reachable and weight the list
based on the specificity and metric of the respective route.

Looking at the problem from that angle, all hosts are MIF hosts. They
have a unique loopback network which is administratively separate from
any physical wire to which they attach. Put another way: from the
perspective of MIF, loopback doesn't look like a special case.

2. DNS strikes me as a subset of a bigger problem: naming services.
Other naming services such as the windows browser, windows domains,
LDAP and NIS will be impacted in more or less the same way that the
DNS is.

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004

---

From: Jan Melen <[email-masked]>
Date: Tue, 10 Nov 2009 00:45:14 +0200

Hi folks,

> perspective of MIF, loopback doesn't look like a special case.

In the edge router products I think that there exists solutions how do  
they handle multiple interfaces and metrics of those. So if you have  
two interfaces through which you can reach the network the router  
usually makes the decision based on metrics and some times even based  
on policy based routing rules.

But then again if the intent is to bring up issue on how a host (e.g.  
BSD) acts as a router or when using static addressing (not DHCP) that  
is whole different story. E.g. in all BSDs it is possible to use  
multiple defaultroutes based on the source address the application  
select using firewall rules. Almost all firewall implementations allow  
you to specify next hop for the packet based on source address,  
destination address, outgoing or incoming interface or by any  
combination of these. So if your BSD/Linux host acts as an router or  
you use static addresses in which case you know the addresses of  
defaultroutes before hand you can do a lot more than what ISC-DHCP can  
do. Even in the case of ISC-DHCP you could by creating dhclien-scripts  
change the default behavior of address lease for example to modify  
your firewall rules according to the address lease information. In the  
recent versions of FreeBSD 7.x-> there is introduced a functionality  
that allows you to have multiple routing tables and you can for  
example limit that certain process may access only certain routing  
table thus mandating that packets coming from that process are routed  
according to that table. This is useful for example in the cases where  
you have multiple virtual hosts or jails (sandboxes) running in one  
physical hardware thus you can limit and control how the packet are  
routed from that entity.

So all in all. I think that most of the features that are needed to  
actually route packets based on source address or interface basis  
exist in the kernel but usually when you take the system out of the  
box (CD) they are not used. You have to really be a geek to get them  
to work as you would like to and not even then are things easy unless  
you really know how the kernel works.

> LDAP and NIS will be impacted in more or less the same way that the

Agreed. DNS is really a problem as the first resolving of the address  
already mandates which interface you choose or if you use multiple and  
get multiple different answers how are you supposed to know which  
address is more appropriate.....

     Jan

---

From: liu dapeng <[email-masked]>
Date: Tue, 10 Nov 2009 11:11:55 +0800

Hi William,

> routed subsystems could be members of two networks.

In my understanding, router normally does not have the multiple
interfaces issue, since router is designed to forward packets across
different interfaces and there is no default route issue in router. so
what problems are you thinking that need to be solved?

> _______________________________________________
> mif mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mif
>

---

From: William Herrin <[email-masked]>
Date: Mon, 9 Nov 2009 23:03:04 -0500

> what problems are you thinking that need to be solved?

Hi,

Starting from a post-multipath TCP world...

The situation that jumps to mind is:

3 offices, A B and C.

B connects to A with a T1.
C connects to A with a T1.
B connects to C with a T1. So they're connected in a triangle.

A connects to the Internet with a DSL.
B also connects to the Internet with a WiMax link.

Server located at C called CS. It has an address from the DSL ISP and
an address from the WiMax ISP.

CS(DSL)->CA router->AC Router->DSL Router->Internet
CS(WiMax)->CB router->BC Router->WiMax Router->Internet

But if the CA T1 is down then:
CS(DSL)->CA router->CB Router->BC Router->BA Router->AB Router->DSL
Router->Internet

And while CS has two addresses and network blocks, it has only one
ethernet interface on which it talks to both the CA and CB routers.

Following me? The routers in the picture have two different routing
domains depending on the source address.

Add an Internet cable modem to site C and now you have to handle 3
routing domains on every router in the system.

But if you keep thinking about it, maybe you don't try to introduce
routing domains at all. Maybe you introduce source+destation CIDR
pairs instead of just destination CIDRs. So, the DSL at A offers
1.2.3.0/24+0.0.0.0/0, the WiMax at B announces 4.5.6.0/24+0.0.0.0/0
and the cable modem at C announces 7.8.9.0/24+0.0.0.0/0. On the flip
side, the site-C routers announce 0.0.0.0/0+1.2.3.32/28,
0.0.0.0/0+4.5.6.32/28 and 0.0.0.0/0+7.8.9.32/28. Now there's just one
routing domain and six routes which work sensibly in a dynamic routing
protocol.

And if CS gets hacked and tries to send a packet to 9.9.9.9 from the
forged source address 8.8.8.8, the packet dies at the first router
because there's no route for 8.8.8.8/32+9.9.9.9/32.

Now if you go back and think of host CS the same way as the routers,
CS has the following routes:
1.2.3.35/32+1.2.3.32/28->eth0
1.2.3.35/32+0.0.0.0/0->Router CA metric 2
1.2.3.35/32+0.0.0.0/0->Router CB metric 3
4.5.6.35/32+4.5.6.32/28->eth0
4.5.6.35/32+0.0.0.0/0->Router CB metric 2
4.5.6.35/32+0.0.0.0/0->Router CA metric 3
7.8.9.35/32+7.8.9.32/28->eth0
7.8.9.35/32+0.0.0.0/0->Cable Modem metric 1
127.0.0.0/8+127.0.0.0/8->loopback

Or so goes my train of thought anyway...

Regards,
Bill Herrin

---

From: Jan Melen <[email-masked]>
Date: Tue, 10 Nov 2009 07:00:21 +0200

Hi William,

See inline.....

> CS(DSL)->CA router->CB Router->BC Router->BA Router->AB Router->DSL

The CA router can't for the packet that has the source address of the  
DSL ISP to the CB router. The source address filtering in the WiMax  
ISP would drop the packet. If the CA router would do translation of  
addresses it would somewhat work but still the end-host mif function  
would not have any control. If then again the CA router would replace  
the source address with the address CS(WiMax) the return packet would  
come back on a socket that is not open.

Or maybe I misunderstood something from this...

> domains depending on the source address.

So far....

> 127.0.0.0/8+127.0.0.0/8->loopback

If any of the default routes fail in such configuration it should be  
the routing protocol that stops advertising default route through that  
interface at that point in which case it would result that default  
route through that interface deprecates in few minutes and the traffic  
will be routed through another router. Existing connections will break  
if there is no translation in place in the network see the earlier  
comments about source address filtering.

DHCP has very poor or non-existing mechanisms to inform the clients  
about changes in the network routing information as the protocol is  
designed to be pull instead of push where as all routing protocols are  
much more appropriate for this type of operation.

    Regards,
      Jan

---

From: Jan Melen <[email-masked]>
Date: Tue, 10 Nov 2009 07:17:01 +0200

Hi William again,

I'm replying to my own mail sorry for that but forgot one thing from  
the previous mail. If we forget about the failure case then I think  
that the scenario you proposed is valid for MIF meaning that what ever  
mechanism we use to select the source address we should also select  
the default router accordingly regardless whether the routers are on  
the same link or different links of the host. That I think was already  
in the scope of MIF problem statement if I understood correctly Marc  
and Margaret yesterday at WG meeting.

Marc or Margaret may correct me if I've misunderstood something.....

    Regards,
       Jan

On Nov 10, 2009, at 7:00 AM, Jan Melen wrote:

> The CA router can't for the packet that has the source address of  
> the DSL ISP to the CB router. The source address filtering in the  
> WiMax ISP would drop the packet. If the CA router would do  
> translation of addresses it would somewhat work but still the end- 
> host mif function would not have any control. If then again the CA  
> router would replace the source address with the address CS(WiMax)  
> the return packet would come back on a socket that is not open.
>
> the routing protocol that stops advertising default route through  
> that interface at that point in which case it would result that  
> default route through that interface deprecates in few minutes and  
> the traffic will be routed through another router. Existing  
> connections will break if there is no translation in place in the  
> network see the earlier comments about source address filtering.
>
> designed to be pull instead of push where as all routing protocols  
> are much more appropriate for this type of operation.
>
>   Regards,
>     Jan
>
>
>

---

From: William Herrin <[email-masked]>
Date: Tue, 10 Nov 2009 11:44:56 -0500

On Tue, Nov 10, 2009 at 12:00 AM, Jan Melen <[email-masked]> wrote:
> The CA router can't for the packet that has the source address of the DSL
> ISP to the CB router.

It should. If we remove the WiMax connection so there's only one
network domain, we would certainly expect a packet for the Internet at
router CA to find it's way to the DSL router at site A regardless of
whether the T1 from site C to site A was up. Adding the WiMax internet
link at site B should in no way impact CS(DSL)'s ability to reach the
Internet via the DSL line at site A.

That's among the reasons I think its important to consider the router
case in addition to the host case.

> If any of the default routes fail in such configuration it should be the
> routing protocol that stops advertising default route through that interface
> at that point in which case it would result that default route through that
> interface deprecates in few minutes and the traffic will be routed through
> another router. Existing connections will break if there is no translation
> in place in the network see the earlier comments about source address
> filtering.

Yes. That's the desired behavior: if the internet link is unavailable
for a given source address then depending on the nature of the failure
packets to the Internet from that source address either vanish into
the ether or return a destination unreachable. It's then up to higher
level protocols (like Multipath TCP) to detect and deal with the path
failure, shifting the failed source address out of use.

On the other hand, just because the Internet link was lost doesn't
mean the administrative domain goes away. In the scenario I painted,
the IP addresses from the failed Internet link are still valid for
communication within three sites; they just aren't workable for a
default route any more.

> DHCP has very poor or non-existing mechanisms to inform the clients about
> changes in the network routing information as the protocol is designed to be
> pull instead of push where as all routing protocols are much more
> appropriate for this type of operation.

A good point to make in the problem statement if it isn't already there.

Regards,
Bill Herrin

---

From: Jan Melen <[email-masked]>
Date: Tue, 10 Nov 2009 20:44:47 +0200

Hi,

On Nov 10, 2009, at 6:44 PM, William Herrin wrote:

>> the DSL

Oops sorry my mistake I though you were talking about that the  
connection to ISP went down but obviously you meant that the T1  
connection between C and A dies. In that case, yes, the routing  
protocol should resolve this situation. As probably the routers A, B,  
C run some routing protocol (OSPF, BGP, RIP or whatever) that will  
have to resolve the situation and get the packets routed correctly  
through although using route that is less preferred than the route  
normally used due to higher metric value. Of course as we now are  
talking about multiple default routes that are advertised between the  
routers A, B, C so we need to add source address prefix in to the  
default route advertisements. Right? Then the C router should see that  
router A is advertising (default, prefix:1.2.3.0/24, metric 1)  and B  
is advertising (default, prefix:1.2.3.0/24, metric 2). This  
information doesn't necessarily need to go all the way to client. If  
it does, e.g. there are two physically different routers that CS may  
use to reach networks A and B, then we need that CS listens to these  
advertisements and makes up his mind and if it is the CS that has to  
select the correct next hop router then DHCP is not the protocol you  
want to use to configure routes as it is designed to deliver more or  
less static routes that do not change (at least) during the address  
lease time.

> communication within three sites; they just aren't workable for a

Yep, the addresses are valid for internal traffic but the default  
route is not any more valid.

>> about

   Regards,
     Jan

---

From: Xiangsong Cui <[email-masked]>
Date: Mon, 16 Nov 2009 15:14:30 +0800

Hi William,

Sorry for jumping into the discussion,
I just want to remind that Router and Host are different concept.

>>> Routers are hosts too and figuring out ......

Take the definition from RFC2460:

   router      - a node that forwards IPv6 packets not explicitly
                 addressed to itself.  [See Note below].

   host        - any node that is not a router.  [See Note below].

Although they are both node  - a device that implements IPv6.

Regards
Xiangsong

---

From: William Herrin <[email-masked]>
Date: Mon, 16 Nov 2009 11:06:16 -0500

> Take the definition from RFC2460:

Hi Xiangsong,

Two observations:

1. MIF's charter is not IPv6 specific and the distinction you draw was
not made for IPv4.

2. Both the IPv4 and IPv6 network stacks for PC's and routers have
stayed more or less converged for quite a while. Higher end routers
have some hardware assists that don't tend to show up in PCs but
they're otherwise just close to identical. If we're to recommend that
change, we ought to have better reasoning than, "we didn't want to
deal with the router case" and we should very thoroughly explain that
reasoning.

Regards,
Bill Herrin

---

From: William Herrin <[email-masked]>
Date: Mon, 16 Nov 2009 11:08:42 -0500

On Tue, Nov 10, 2009 at 1:44 PM, Jan Melen <[email-masked]> wrote:
> Of course as we now are talking
> about multiple default routes that are advertised between the routers A, B,
> C so we need to add source address prefix in to the default route
> advertisements. Right?

Hi Jan,

Do we? What about routes other than the default? Does it ever make
sense for a small ISP to propogate three source address ranges on each
link downstream instead of introducing DFZ routes of its own?

We've identified at least two approaches here, tying source and
destination together in the route and establishing entirely separate
routing domains for each source range.  Are there more ways to look at
the problem?

I think that's a worthwhile question for the problem statement.

> we need that CS listens to these advertisements and makes up his mind and if
> it is the CS that has to select the correct next hop router then DHCP is not
> the protocol you want to use to configure routes as it is designed to
> deliver more or less static routes that do not change (at least) during the
> address lease time.

How big a concern is that? Won't the higher-level protocols have to
deal with dead path detection anyway? Even if you have a legitimate
default route, there's no guarantee that all of your ISPs are fully
connected right now.

Regards,
Bill Herrin

---

From: Xiangsong Cui <[email-masked]>
Date: Tue, 17 Nov 2009 09:57:59 +0800

Hi,

This is just a comment but not objection to the discussion,

About the difference between host and router,
Wiki may also be taken as a reference, 
http://en.wikipedia.org/wiki/Host_(network)
Yes, there are software routers, the configuration option in Linux PCs and 
others...
but the operation of *pure host* and router are different practice.

Regards
Xiangsong

---

From: Scott Brim <[email-masked]>
Date: Wed, 18 Nov 2009 03:31:22 -0600

Bill, I don't understand exactly what you're saying but it seems to
be something like:

  - An endpoint may have multiple addresses assigned to it on a single
    interface.  It could even have addresses from different PA
    prefixes from upstream providers.
  - If source address and upstream provider path don't match, packets
    could be dropped.
  - Address selection itself can't help here, so you suggest that
    routing take source address into account until packets get beyond
    where uRPF is enforced.

Is that right?

swb

---

From: William Herrin <[email-masked]>
Date: Wed, 18 Nov 2009 17:50:34 -0500

>    routing take source address into account until packets get beyond

Hi Scott,

That too, but I'm trying to convey what I think is a more important
notion: Address selection is classically a simple function on the
routing table: pick the primary address of the interface the
destination routes out of. With multiple administrative domains,
address selection may *still* be a simple function on the routing
table... the more significant difference may be in the routing table
itself.

That notion is suggested by the following example:

(A,B,C)-D-(E,F)

Where A, B, C are hosts, D is a router and E, F are ISPs supplying two
different administrative domains with unique blocks of PA addresses.
The two domains propogate from E and F down through D to hosts A, B
and C. A, B, C and D *all* need to do the right thing with respect to
source address selection and route selection based on source address.
*AND* if D is a complex network instead of a simple router, that
complex network needs a *dynamic routing protocol* that helps the
routers in D do the right thing, not just static policy routing.

I'm not sure if MIF is correctly scoped for that, but offer the
thought that maybe it should be.

Regards,
Bill Herrin

---

From: Rémi Després <[email-masked]>
Date: Thu, 19 Nov 2009 10:06:02 +0100

> source address selection and route selection based on source address.

Yes, that's the key.

> routers in D do the right thing, not just static policy routing.

In a complex D, an alternative to a dynamic routing protocol consists  
in a stateless address mapping, in border hosts and routers of D,  
with encapsulation of global packets to traverse D.

This is feasible with the SAM approach described in draft-despres- 
softwire-mesh-sam-01.
A customer-side border node, when it has to forward a global packet  
toward the global Internet, selects the destination of the  
encapsulating header based on which PA prefix it finds in the source  
global address. This destination is the address of the provider-side  
border router to which this PA prefix has been delegated (known by  
some parameters received in DHCP ).

If you have a look at this draft, maybe you can tell whether you  
agree or not on the relationship with the point you make.

Regards,

RD

---

From: Jan Melen <[email-masked]>
Date: Fri, 20 Nov 2009 08:27:30 +0200

Hi,

> link downstream instead of introducing DFZ routes of its own?

Yep, you're right one could also introduce DFZ routes.

> default route, there's no guarantee that all of your ISPs are fully

Well it would be nice if one could avoid even trying such a route  
which is known to be dead, but of course through higher level  
protocols you can get the the information through time outs but that  
usually is not desirable as it introduces poor user experience.

      Jan

---

From: Jan Melen <[email-masked]>
Date: Fri, 20 Nov 2009 08:40:57 +0200

Hi,

On Nov 19, 2009, at 11:06 AM, Rémi Després wrote:

> In a complex D, an alternative to a dynamic routing protocol  
> consists in a stateless address mapping, in border hosts and routers  
> of D, with encapsulation of global packets to traverse D.
>

Remi could you elaborate a bit more why would we need a encapsulation  
inside D to solve the routing problem? Why don't we use existing  
routing protocols instead to exchange the information and even do DFZ  
routes inside D as was pointed out earlier? I don't think we want to  
introduce something like SAM in to the end-hosts for doing this and  
I'm not even convinced that we need for D either?

      Jan

---

From: Rémi Després <[email-masked]>
Date: Fri, 20 Nov 2009 14:03:01 +0100

Le 20 nov. 2009 à 07:40, Jan Melen a écrit :

>> In a complex D, an alternative to a dynamic routing protocol consists in a stateless address mapping, in border hosts and routers of D, with encapsulation of global packets to traverse D.
>> 
>> This is feasible with the SAM approach described in draft-despres-softwire-mesh-sam-01.
>> A customer-side border node, when it has to forward a global packet toward the global Internet, selects the destination of the encapsulating header based on which PA prefix it finds in the source global address. This destination is the address of the provider-side border router to which this PA prefix has been delegated (known by some parameters received in DHCP ).
>> 
>> If you have a look at this draft, maybe you can tell whether you agree or not on the relationship with the point you make.
>> 
> 
> Remi could you elaborate a bit more why would we need a encapsulation inside D to solve the routing problem?

Thanks to encapsulation, an end-to-end packet can be routed across a local routing domain with a forced local destination. This one can depend on which ISP delegated to the domain the prefix matched by the E2E source address.    

> Why don't we use existing routing protocols instead to exchange the information and even do DFZ routes inside D as was pointed out earlier?

Note that nodes that support a dynamic routing protocol compatible with one supported by their ISPs don't *need* SAM. It is just an alternative.
In favor of SAM is its simplicity (no protocol, no AS number...).

> I don't think we want to introduce something like SAM in to the end-hosts for doing this and I'm not even convinced that we need for D either?

In multihomed sites having several CPEs, a host that supports neither a routing protocol nor SAM cannot ensure that a packet it sends send goes to the right ISP (that which provided the PA prefix matched by the source address).

RD