Subject: [rbridge] PS issue #4

From: spencer at mcsr-labs.org (Spencer Dawkins)
Date: Wed, 7 Dec 2005 13:36:02 -0600

While looking at

#4 delivery requirements:
   a) is in-order required?
   b) is no-duplication required?

I tripped over some unexpected good news: IEEE 802.1D-2004 does seem to 
understand that trading off in-order and no-duplication guarantees for 
performance makes sense (ex. Section 6.3.3, "NOTE 2-Frame misordering and 
duplication (6.3.4) does not occur during normal operation. When RSTP is 
configuring or reconfiguring the network (see Clause 17), there is an 
increased and implementation-dependent probability that frames that are in 
transit will be misordered or duplicated as network paths change, since a 
Bridge can buffer frames awaiting transmission through its Ports. Since the 
probability of duplication or misordering occurring as a result of 
reconfiguration is small, and the frequency of physical network failures 
leading to reconfiguration is also generally small, the degradation of the 
properties of the MAC service is considered to be negligible."

This was a huge improvement over the situation in IEEE 802.17/RPR several 
years ago, when we assumed (IIRC, because we were explicitly told) that no 
tradeoff was possible.

As the note goes on to say, there are LLC-level protocols that don't 
tolerate out-of-order delivery, but anything running over TCP is protected 
from misordering, and anything running over UDP should already be dealing 
with reordering, so I think that for our purposes, we can say that meeting 
these requirements in normal operation is "good enough".

Although I'm getting off-topic, I note that letting TCPs see reordering 
during path changes is not a bad thing, since it's a clue that available 
path bandwidth may be changing...

Spencer

---

From: touch at ISI.EDU (Joe Touch)
Date: Wed, 07 Dec 2005 12:12:14 -0800

> these requirements in normal operation is "good enough".

Which requirement:

a) that reordering is a transient possibility

b) that persistent reordering is acceptable

The latter affects whether multipath forwarding is supported inside the
rbridge and whether it also requires reordering facilities at the egress.

> path bandwidth may be changing...

The Internet doesn't tend to want to think a link layer has volatile
properties in general, though.

Joe

---

From: harald at alvestrand.no (Harald Tveit Alvestrand)
Date: Wed, 07 Dec 2005 21:36:44 +0100

--On onsdag, desember 07, 2005 12:12:14 -0800 Joe Touch <touch at ISI.EDU> 

> rbridge and whether it also requires reordering facilities at the egress.

multipath between a single source/destination pair may be troublesome 
enough that we should not attempt it..... I don't think there is an 
ordering requirement for packets between different source/destination pairs.

I don't think that persistent reordering (that is, reordering when the 
network is not being perturbed) is acceptable - if severe, it 
*significantly*  degrades TCP performance, if nothing else.

---

From: touch at ISI.EDU (Joe Touch)
Date: Wed, 07 Dec 2005 12:45:19 -0800

Harald Tveit Alvestrand wrote:
> enough that we should not attempt it.....

Isn't that one of the opportunities we wanted to support, though?

It's already supported by the Internet protocols we're considering
(e.g., IS-IS - see RFC2991). As per below, do we want to prohibit it
outright, or assert that IF equal-cost multipath is used then reordering
mitigation (and perhaps latency smoothing) needs to occur?

> I don't think there is an

---

From: spencer at mcsr-labs.org (Spencer Dawkins)
Date: Wed, 7 Dec 2005 14:46:50 -0600

Hi, Joe,

> rbridge and whether it also requires reordering facilities at the egress.

"Reordering is a transient possibility". I think that persistent reordering 
will not be acceptable (for both technical and political reasons).

> properties in general, though.

(I hope no one ever runs IP over wireless link layers :-) but I do 
understand your point)

---

From: harald at alvestrand.no (Harald Tveit Alvestrand)
Date: Thu, 08 Dec 2005 07:45:13 +0100

--On onsdag, desember 07, 2005 12:45:19 -0800 Joe Touch <touch at ISI.EDU> 

> mitigation (and perhaps latency smoothing) needs to occur?

from RFC 2991:

   All of the problems outlined in the previous section arise when
   packets in the same unicast or multicast "flow" are split among
   multiple paths.  The natural solution is therefore to ensure that
   packets for the same flow always use the same path.

I think it makes sense to say:

- we don't require that multipath for one source-destination pair be 
supported
- IF multipath for one source-destination pair is available, and can cause 
persistent reordering, we need reordering mitigation in order to be 
consistent with the Ethernet service model

I'm troubled by the implicit layer violation in the multipath algorithm 
trying to detect "flows" at a higher level such as TCP sessions.
(for extra complexity, add IPSec or encryption at Ethernet layer.... both 
would make most flow-detection algorithms go haywire....)

                   Harald

---

From: gibanez at it.uc3m.es (Guillermo Ibáñez)
Date: Thu, 08 Dec 2005 08:26:07 +0100

>will not be acceptable (for both technical and political reasons).
I fully agree. Reordering  is only acceptable as transitory event.

>_______________________________________________
>rbridge mailing list
>rbridge at postel.org
>http://www.postel.org/mailman/listinfo/rbridge
>
>  
>

-- 
Guillermo Ib??ez
Departamento de Ingenier?a Telem?tica
Universidad Carlos III de Madrid
1.1.B.11 Colmenarejo 91-6241393
4.1.F.13 Legan?s 91-6248794

---

From: touch at ISI.EDU (Joe Touch)
Date: Thu, 08 Dec 2005 09:04:30 -0800

> would make most flow-detection algorithms go haywire....)

This is an easy position to adopt, but I'm wondering whether anyone
considers the result overly restrictive - esp. where addressing the
limitations of STPs (even MSTP) using Internet-style routing is the
issue (where Internet routing is still embracing multipath).

An alternative would be to allow multipath but limit it for a flow. Here
a flow would be for a single source or destination ethernet address
('external' address). That would allow for multipath for internal
destination addresses (multipath to destination) but retain ordering per
'flow' (in this case the internal addresses act as ports).

Joe

---

From: harald at alvestrand.no (Harald Tveit Alvestrand)
Date: Thu, 08 Dec 2005 23:06:57 +0100

--On torsdag, desember 08, 2005 09:04:30 -0800 Joe Touch <touch at ISI.EDU> 
wrote:

>> (for extra complexity, add IPSec or encryption at Ethernet layer....
>> both  would make most flow-detection algorithms go haywire....)
>>
>>                    Harald
>

I considered 2991 to be saying "it's wise to be somewhat restrictive"... 
and think that our specs should say "we SHOULD be restrictive", or perhaps 
even "we MUST"...

> 'flow' (in this case the internal addresses act as ports).

when you say "limit it for a flow", do you mean "do not allow persistent 
reordering within a flow"?
and when you say "flow", do you mean "all packets with the same (external) 
Source address and the same (extrernal) destination address"?
If that's what you mean, I think you're trying to say the same thing I am 
trying to say.... that's progress...

---

From: touch at ISI.EDU (Joe Touch)
Date: Thu, 08 Dec 2005 18:17:24 -0800

...
> reordering within a flow"?

Yes.

> trying to say.... that's progress...

Basically. Could also mean VLAN group; anything that doesn't affect TCP
;-) I.e., what Eric said (trying to be general enough to avoid IPR).

Joe