Subject: [Tsvwg] SCTP source address routing

From: Mark Butler <[email-masked]>
Date: Wed, 15 Mar 2006 22:55:18 -0700

Suppose I have two hosts A and B that are multihomed, each to a pair of 
different providers.  Each host has a source address based routing 
policy so that traffic can be routed through either connected provider 
by selecting the source address appropriately.

Now each host has a choice of four paths to reach the other host.  
However, the standard procedure described in RFC2960 does not allow each 
of these four paths to be tested independently, resulting in cases where 
associations fail even though viable connectivity is still available 
just by switching source addresses. 

Moreover, among the four paths, even (especially) between two hosts in 
the same city, one is likely to be much better than the other three, and 
this path will often go missing if only two paths are used.  Most 
providers do not peer locally in most cities, so traffic typically gets 
routed through several nearby states and back even if the destination is 
across the street. But if two of the four possible providers peer 
locally, or if the endpoints share a common provider, that path will 
usually be the best by far, in terms of RTT, jitter, packet loss, etc.

So I suggest a scheme where failover and congestion control is done by 
source / destination address pair rather than by destination address 
alone.  This is particularly important in the case where one host is 
multihomed, and the other is not.  Without path based (instead of 
destination address based) failover support, the multihoming of the 
first host is relatively pointless, because there is only one path used 
the first host to the second, even though two are used in the reverse 
direction.

The standard problem with doing this is N^2 path explosion.  I suggest 
the following techniques to mitigate this problem:

(1) Advertise only one address per interface per next hop when bound to 
INADDR_ANY.

(2) When generating local address lists for INIT and INIT ACK, check to 
see if routes to any of the peer addresses exist from a given local 
address when bound to the interface of that address.  If not, eliminate 
the address.

(3) Whenever transmitting with an INADDR_ANY derived source address, use 
the same interface as the address.  If a destination cannot be reached 
through that interface, for whatever reason, let the routing layer 
select both the source address and interface.

(4) Track whether each path is confirmed or unconfirmed, where the 
default is unconfirmed.

(5) When replying to an INIT or an INIT ACK (with an INIT ACK or a 
COOKIE ECHO) use the original source address as the destination address, 
and the original destination address as the source address.

(6) Mark the (dest address, source address) pair of a valid packet 
bearing a valid INIT ACK or COOKIE ECHO as a confirmed path.

(7) Use a heartbeat info structure that includes forward path information.

(8) Upon receiving  a HEARTBEAT ACK, mark the forward path confirmed 
using the information in the heartbeat info structure.

(9) Send DATA only through confirmed paths

Rationale:

(R1) More addresses than that is pointless except with very unusual 
configurations

(R2) Avoid creating degenerate (completely overlapping) paths, e.g. for 
addresses on interfaces that will never be used by the association, such 
as stub network interfaces.

(R3)  Avoid using degenerate paths.

(R5) Necessary for initial forward path confirmation. The reverse path 
is most likely to be up as well.

(R6)  The confirmation procedure eliminates invalid cross paths between 
two redundant networks, as well as paths that are reverse routable, but 
not forward routable.

(R8) Same as R6.

(R9) Sending data only on confirmed paths reduces failover latency. 
There will always be one confirmed path after a valid INIT ACK or COOKIE 
ECHO is received.  The others will normally be confirmed by initial 
heartbeats.

---

A further refinement would standardize the heartbeat info prefix to 
include information about which path the heartbeat ack should be sent 
on, and the time elapsed between HEARTBEAT arrival and HEARTBEAT ACK 
departure (similar to DCCP).  That would allow proper forward path 
transit time estimation (where proper means adjusted for asymmetric 
forward and reverse routes, bandwidth, transit time, and congestion)  
with the objective of selecting the path with the lowest forward transit 
time for DATA and SACK transmissions. 

By any chance has anyone implemented a scheme similar to this, with 
(source address, dest address) based paths, heartbeats, and congestion 
control instead of just (dest address) based paths?

 - Mark

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 16 Mar 2006 02:09:05 -0700

Mark,

Mark Butler wrote:                              (Wed, 15 Mar 2006 22:55:18)
> by selecting the source address appropriately.

I was told years ago that source address routing was the stuff of research.
Nevertheless, implementations are encouranged to vary the source address
if possible and when necessary.  It does not seem to cause any problems in
practice.  I believe that source address routing can break some widespread
TCP implementations as well.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: <[email-masked]>
Date: Thu, 16 Mar 2006 14:02:48 +0200

Hi!

>control instead of just (dest address) based paths?

	In fact, my implementation does basically exactly the same you were stating in your email. Using paths, validating them, and so on...

	Seems that somebody has been spying somebody :-)=

	BR Iván Arias Rodríguez

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 16 Mar 2006 06:42:32 -0700

Mark,

There's a pretty good read on the SIGTRAN mail archive from March 2000
entitled: Source and destination address selecting strategy

Randall Stewart wrote:    (Tue, 28 Mar 2000 04:52:06 -0600)
>
> For sure it will very but I don't think any O/S I know of uses the source
> address in choosing the interface to fire the datagram out. This, according
> to Scott, is a research matter for the IRTF ...
>

Randall Stewart wrote:    (Wed, 29 Mar 2000 22:26:20 +0000)
> 
> To requote Scott Bradner, go back and see my
> forwarded email from him.
> 
> Anything that has to do with source/destination routing
> is the subject of research and should NOT be in
> the SCTP draft.
> 

That is pretty much why it wasn't addressed.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Thu, 16 Mar 2006 08:53:50 -0700

>TCP implementations as well.
First of all, the real goal here is not source address routing per se, 
but choosing an outbound interface or nexthop router appropriate for a 
path.   If one does not do something like that, the only way to get SCTP 
path failover at a multihomed site (in general) is to either put entries 
in the routing table for every peer destination address, or carry full 
BGP route information at the gateway router, so that (if you are lucky) 
different peer addresses will use different outbound paths. 

In general, path failover at the transport layer requires path selection 
at the transport layer.  That policy is _much_ more scalable (not to 
mention more widely deployable) than the alternatives.

 - Mark

---

From: Mark Butler <[email-masked]>
Date: Thu, 16 Mar 2006 09:27:13 -0700

Brian F. G. Bidulock wrote:

>>address in choosing the interface to fire the datagram out. 
>>
Linux can do this for both IPv4 and IPv6. The administrator has to 
specify an an appropriate routing policy of course.  One should not have 
to run a routing protocol on every SCTP host just to get rudimentary 
failover between two different providers.

>>This, according

I do not know whether that is still the current sentiment, but it should 
be relatively obvious that SCTP implements capabilities at the transport 
layer traditionally left to the network layer, and for very good 
reason.   In particular network layer rerouting is slow and does not scale.

>>is the subject of research and should NOT be in
Well, if it wasn't in 2000, source address routing is in pretty 
widespread use now.

- Mark

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 16 Mar 2006 14:30:39 -0700

Mark,

It was widely available for routers and Linux boxen in 2000 too.

--brian

Mark Butler wrote:                           (Thu, 16 Mar 2006 09:27:13)
>    specify  an  an  appropriate routing policy of course.  One should not
>    have  to  run  a  routing  protocol  on  every  SCTP  host just to get
>    rudimentary failover between two different providers.
> 
> This, according
>    I  do  not  know  whether  that is still the current sentiment, but it
>    should  be relatively obvious that SCTP implements capabilities at the
>    transport  layer traditionally left to the network layer, and for very
>    good  reason.   In particular network layer rerouting is slow and does
>    not scale.
> 

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Xiaoming Fu <[email-masked]>
Date: Thu, 16 Mar 2006 23:48:41 +0100

What if there is one targeted destination address while there are 
several SCTP source addresses (corresponding to several links/paths)?
Xiaoming