Subject: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

From: Rui Costa <[email-masked]>
Date: Fri, 26 Feb 2010 11:55:15 +0000

Greetings,	

Once this draft has some content related to ETH interfaces (carrying MPLS-TP packets), I'd like to put the question below.	
 
If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is impredictability of BW distribution among LSPs while maintaining frame ordering (distributors' algorithms need to maintain "each conversation's frame order" and that implies hard BW distribution predictability and hence provisioning).	

I imagine Ethernet link aggregation distributor algorithms to be very alike to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has the same implication for ETH link aggregation and hence it won't be applicable to MPLS-TP networks?	

Thanks for your time. Regards,	
Rui

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 26 Feb 2010 12:04:23 +0000

> If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is impredictability of BW distribution among LSPs while maintaining frame ordering (distributors' algorithms need to maintain…
.. and the inability for the OAM to fate share with the LSP.
> I imagine Ethernet link aggregation distributor algorithms to be very alike to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has the same implicition for ETH link aggregation and hence it won't be applicable to MPLS-TP networks?	
>   
I would think that if the operator wanted the bandwidth more than they 
wanted the fate sharing, they will deploy the technology.

- Stewart

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 26 Feb 2010 09:24:19 -0800

Hi Rui,

I have a slightly different view on this.

If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dictate
how it is to be done, if it is shown as one interface to the MPLS-TP layer.
MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet link
level will be visible to MPLS-TP as a link down and not which of the
component links is down.

Thanks,
Vishwas
On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]> wrote:

> Rui Costa wrote:
>
>> Greetings,
>>
>> Once this draft has some content related to ETH interfaces (carrying
>> MPLS-TP packets), I'd like to put the question below.
>>  If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason
>> is impredictability of BW distribution among LSPs while maintaining frame
>> ordering (distributors' algorithms need to maintain "each conversation's
>> frame order" and that implies hard BW distribution predictability and hence
>> provisioning).
>>
>>
>> I imagine Ethernet link aggregation distributor algorithms to be very
>> alike to MPLS ECMP's. If so, can we say that the similarity between ECMP
>> (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has the same
>> implicition for ETH link aggregation and hence it won't be applicable to
>> MPLS-TP networks?
>>
>>
> _______________________________________________
> mpls-tp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mpls-tp
>

---

From: Peng He <[email-masked]>
Date: Fri, 26 Feb 2010 13:00:56 -0500

Hi Vishwas,

I agree that an Ethernet Link Aggregation group can be view as one
'virtual' interface to the MPLS-TP layer. Just a little confused here
that suppose a 4-link Ethernet AGG group, tnow one member/component
link is down then the traffic will be re-distributed to the remain 3
memner/component links of the AGG based on the AGG distribution
algorithm and the traffic can still  go through. At this moment,
should MPLS-TP view the (whole) 'virtual' interface down? or the
MPLS-TP views the 'virtual' interface as well as each individual
member/component link and consider it as a normal link down?

Regards,
Peng

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 26 Feb 2010 10:38:54 -0800

Hi Peng,

This is what should happen.

In the ideal case the detection time of the aggregate link going down at the
Ethernet Layer should be faster than that in the MPLS-TP layer. In such a
case when a link goes down, it would probably just a change in properties of
the link (bandwidth etc) and no link down event to MPLS-TP.

However if the detection at the higher layer MPLS-TP is faster it will treat
the whole virtual link down and then UP with new properties when the lower
layer detection happens.

Thanks,
Vishwas

---

From: David Allan I <[email-masked]>
Date: Fri, 26 Feb 2010 15:07:52 -0500

The one complication being that for CAC'd traffic, the aggregate capacity of the LAG in failure scenarios can drop below the level of the bandwidth committments associated with the paths that transit the LAG. In which case you want to move or protection switch SOME of the traffic.

The other complication is due to flow based assignment, the individual links in the LAG may not be being uniformly loaded...a dilation factor is required. And moving the traffic off the LAG as a response to partial failure reduces the statistical sample size for load spreading :-(....

I do not have solutions to this, I merely note the problems...

cheers
D

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Vishwas Manral
Sent: Friday, February 26, 2010 12:24 PM
To: [email-masked]
Cc: Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

Hi Rui,

If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dictate how it is to be done, if it is shown as one interface to the MPLS-TP layer. MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet link level will be visible to MPLS-TP as a link down and not which of the component links is down.

Thanks,
Vishwas
On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]<mailto:[email-masked]>> wrote:
Rui Costa wrote:
Greetings,

I would think that if the operator wanted the bandwidth more than they wanted the fate sharing, they will deploy the technology.

- Stewart

_______________________________________________
mpls-tp mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls-tp

---

From: Peng He <[email-masked]>
Date: Fri, 26 Feb 2010 16:40:16 -0500

need something called 'LAG-TE', like 'PBB-TE'..

Regards,
Peng

On Fri, Feb 26, 2010 at 3:07 PM, David Allan I
<[email-masked]> wrote:
> The one complication being that for CAC'd traffic, the aggregate capacity of
> the LAG in failure scenarios can drop below the level of the bandwidth
> committments associated with the paths that transit the LAG. In which case
> you want to move or protection switch SOME of the traffic.
>
> The other complication is due to flow based assignment, the individual links
> in the LAG may not be being uniformly loaded...a dilation factor is
> required. And moving the traffic off the LAG as a response to partial
> failure reduces the statistical sample size for load spreading :-(....
>

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 26 Feb 2010 15:04:01 -0800

Hi David,

On Fri, Feb 26, 2010 at 12:07 PM, David Allan I
<[email-masked]>wrote:

>  The one complication being that for CAC'd traffic, the aggregate capacity
> of the LAG in failure scenarios can drop below the level of the bandwidth
Isn't this similar to the case where a higher priority LSP needs to preempt
the lower piority one because the bandwidth on a link is exhausted?

>
> The other complication is due to flow based assignment, the individual
> links in the LAG may not be being uniformly loaded...a dilation factor is
That is correct. Thats a known limitation of LAGged interfaces with out
without MPLS-TP.

Thanks,
Vishwas

>  ------------------------------
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Vishwas Manral
> *Sent:* Friday, February 26, 2010 12:24 PM
> *To:* [email-masked]
> *Cc:* Rui Costa; [email-masked]
> *Subject:* Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00:
> ~ECMP => ~ETH aggregation?
>
>   Hi Rui,
>
> On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]>wrote:
>
>> Rui Costa wrote:
>>
>>> Greetings,
>>>

---

From: Phil Bedard <[email-masked]>
Date: Fri, 26 Feb 2010 19:53:12 -0500

Generally the behavior when a member link (or links) fail is a configurable option.  If you drop below a minimum link count or bandwidth threshold. the whole interface is marked as down.   With most equipment today lower priority sessions will be preempted if the bandwidth on the LAG cannot satisfy the BW requirements prior to the failure, I would think MPLS-TP would behave similarly. 

I don't think LAG can be ruled out as an option for interconnecting two MPLS-TP devices.  At the MPLS-TP function it SHOULD see the lag as a single virtual interface so as far as the MPLS-TP function knows OAM and data came across the same path.

The problem of unfair hashing across a LAG is something an operator who chooses to deploy LAG has to deal with in any case.   

Phil

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 27 Feb 2010 00:32:41 -0500

In message <[email-masked]>
> wanted the fate sharing, they will deploy the technology.

I fully agree.

Providers today are claiming to run LAGs or 8-32 10G members so 100GbE
will require LAGs in provider networks from day one and the LAG
balance is identical in practice to the ECMP load balance.

Another place for a knob to turn off an aspect of MPLS-TP purity for
real networks.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 27 Feb 2010 00:41:56 -0500

In message <[email-masked]>
> level will be visible to MPLS-TP as a link down and not which of the

Vishwas,

Please define you use of the technical term "works".

If the OAM packets have a fixed label stack, then they take only one
member of the LAG.  The other member's data plane are untested.

As I recall from the work on OAM a few years back a goal was to be
able to determin when the data plane did not match what the control
plane thought was there or was otherwise broken in an otherwise
undetected way.

OTOH - If you intend to use PW with an Entropy label ala the fat-pw
draft, then it will work.

There is no analogous Entropy aka fat-pw label for GAL AFAIK.  Maybe
there needs to be one.  It wouldn't be at all hard to put one there
but requires relaxing the GAL at bottom of stack only rule (if that's
not already a SHOULD it needs to be).

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 27 Feb 2010 00:52:06 -0500

> member/component link and consider it as a normal link down?

FYI - Independent of MPLS-TP, what existing MPLS implementations do is
drop the reservable bandwidth figures on the advertised link and if
necessary start preempting LSP according to their holding priority.
This is where soft-preempt helps in a packet networks but I don't know
who's followed through on that.  There is generally a config knob to
allow the entire LAG to be considered down if a certain number of LAG
members are lost.

The loss of a LAG mamber also results in a potential reordering event
but it hasn't seemed to hurt anything at least relative to the packet
loss during the fault detect time.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 27 Feb 2010 01:22:00 -0500

In message <[email-masked]>
Phil Bedard writes:
>  
> Generally the behavior when a member link (or links) fail is a
> configurable option.  If you drop below a minimum link count or
> bandwidth threshold. the whole interface is marked as down.  With most
> equipment today lower priority sessions will be preempted if the
> bandwidth on the LAG cannot satisfy the BW requirements prior to the
> failure, I would think MPLS-TP would behave similarly.
>  
> I don't think LAG can be ruled out as an option for interconnecting
> two MPLS-TP devices.  At the MPLS-TP function it SHOULD see the lag as
> a single virtual interface so as far as the MPLS-TP function knows OAM
> and data came across the same path.
>  
> The problem of unfair hashing across a LAG is something an operator
> who chooses to deploy LAG has to deal with in any case.
>  
> Phil 

Since very few providers don't have parts of their network running
over 40G and the vast majority use 10GbE anyway, they are all familiar
with LAG limitations, but are also aware that it works quite well in
practice.

An alternate is not to use LAG but use link bunding.  If relative
amount of deployment is any indication its not a very good
alternative.  For one thing you can not carry an LSP greater in size
that a LAG member even if that LSP is in turn carrying lots of smaller
LSP (or heaven forbid, an IP payload somewhere under there).
If the unspeakable should happen and there is an IP packet or entropy
label, then link bundling causes a bin packing problem that didn't
exist without link bundling.

Regardless, the "large microflow" problem is real and does upset the
load balance if all that is used is a simple hash and modulo.

The more MPLS-TP pure way to do this would be to use link bundling
rather than LAG and never attempt an LSP larger than a member of the
bundle.  Again, I doubt link bundling will be the common usage even in
networks that make use of parts of MPLS-TP, but speculation of that
sort could trigger a religious argument, so disregard it.

Curtis

---

From: Alexander Vainshtein <[email-masked]>
Date: Sat, 27 Feb 2010 10:17:20 +0200

Curtis and all,

Just to remind you that Section 4.2 of RFC 5586 explicitly requires GAL to be at the 
bottom of the label stack in MPLS-TP environment. And 5586 has been published less than 9 months ago...

My 2c,
     Sasha

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Curtis Villamizar [[email-masked]]
Sent: Saturday, February 27, 2010 7:41 AM
To: Vishwas Manral
Cc: Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP =>    ~ETH aggregation?

Vishwas,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 27 Feb 2010 20:39:56 -0500

In message <[email-masked]>
Alexander Vainshtein writes:
>  
> Curtis and all,
>  
> Just to remind you that Section 4.2 of RFC 5586 explicitly requires
> GAL to be at the bottom of the label stack in MPLS-TP environment. And
> 5586 has been published less than 9 months ago...
>  
> My 2c,
>      Sasha

Yes.  and TTL handling in RFC3032 was updated by RFC3443.  There is
plenty of precedence for a new RFC that updates an RFC or clarifies a
particular point.

Besides it all ready says in Section 4.2:

   However, in other MPLS environments, this document places no
   restrictions on where the GAL may appear within the label stack or
   its use with PWs.

If LAG is used, this says don't call it MPLS-TP, and just do it.  So
for MPLS Ping, it would be useful to put and entropy label under the
GAL label, just in case a LAG is crossed.  Implementations can do this
and just make it a non-default.

Perhaps a "mpls-tp pedantic" config option might disable doing this
and other useful things forbidden by MPLS-TP.

It would be good to get this into the MPLS Ping draft as an option for
use in extended MPLS Ping in mpls-tp non-pedantic mode.  If anyone is
using LAG they may already be in non-pedantic mode.

Curtis

---

From: Alexander Vainshtein <[email-masked]>
Date: Sun, 28 Feb 2010 14:32:57 +0200

Curtis,
Lots of thanks for a prompt response.

I agree that "non-pedantic MPLS-TP mode" will be probably quite useful. 
However, for the purpose of interoperability, it would be nice to understand which of the MPLS-TP restrictions are "pedantic' and which are essential...

As for 3443 updating 3032: it took 2 years (3032 has been published in January 2001, 3442 - in January 2003).
So somebody should probably start writing an update to 5586 now...

Regards,
     Sasha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Sunday, February 28, 2010 3:40 AM
To: Alexander Vainshtein
Cc: [email-masked]; Rui Costa; [email-masked]; Vishwas Manral
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation? 

Curtis

---

From: Maarten Vissers <[email-masked]>
Date: Mon, 01 Mar 2010 00:19:31 +0100

Rui,

There is nothing against running LAG on the basis of labels. 
This is a variation defined as ACL-SNCG/I in G.808.1. 
It is a traffic engineered deployment of LAG (ACL-SNCG/I = "TE-LAG").

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Rui Costa
Sent: vrijdag 26 februari 2010 12:55
To: [email-masked]
Subject: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP =>
~ETH aggregation?

Greetings,	

Once this draft has some content related to ETH interfaces (carrying MPLS-TP
packets), I'd like to put the question below.	
 
If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is
impredictability of BW distribution among LSPs while maintaining frame
ordering (distributors' algorithms need to maintain "each conversation's
frame order" and that implies hard BW distribution predictability and hence
provisioning).	

I imagine Ethernet link aggregation distributor algorithms to be very alike
to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs)
and ETH aggregation (ETH interfaces; 802.3, c43) has the same implication
for ETH link aggregation and hence it won't be applicable to MPLS-TP
networks?