Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

From: Rui Costa <[email-masked]>
Date: Wed, 3 Mar 2010 06:18:40 +0000

Thanks for answering, Stewart.	

"and the inability for the OAM to fate share with the LSP"	
I thought LAG's triggers would be PHY-alarm driven and not OAM. You mean "whole LAG" OAM, right? Wouldn't it still be quicker to infere from PHYs' alarms? (Just possible for LAG - not ECMP...)	

Regards,	
Rui

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Stewart Bryant
Sent: sexta-feira, 26 de Fevereiro de 2010 12:04
To: Rui Costa
Cc: [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

Rui Costa wrote:
> Greetings,	
>
> Once this draft has some content related to ETH interfaces (carrying MPLS-TP packets), I'd like to put the question below.	
>  
> If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is impredictability of BW distribution among LSPs while maintaining frame ordering (distributors' algorithms need to maintain "each conversation's frame order" and that implies hard BW distribution predictability and hence provisioning).	
>   
.. and the inability for the OAM to fate share with the LSP.
> I imagine Ethernet link aggregation distributor algorithms to be very alike to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has the same implicition for ETH link aggregation and hence it won't be applicable to MPLS-TP networks?	
>   
I would think that if the operator wanted the bandwidth more than they 
wanted the fate sharing, they will deploy the technology.

- Stewart

---

From: Rui Costa <[email-masked]>
Date: Wed, 3 Mar 2010 06:18:41 +0000

"In the ideal case the detection time of the aggregate link going down at the Ethernet Layer should be faster than that in the MPLS-TP layer. In such a case when a link goes down, it would probably just a change in properties of the link (bandwidth etc) and no link down event to MPLS-TP"

[Rui] By "LAG" we don't then mean 802.3 c43 / 802.3ad LACP (timers multiple of 1s) but just the aggregation/distribution algorithms "PHY alarm triggered", right? (Protocol and hence Key/aggregatability_check being ruled out)

From: Vishwas Manral [mailto:[email-masked]]
Sent: sexta-feira, 26 de Fevereiro de 2010 18:39
To: Peng He
Cc: [email-masked]; Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

Hi Peng,

This is what should happen.

In the ideal case the detection time of the aggregate link going down at the Ethernet Layer should be faster than that in the MPLS-TP layer. In such a case when a link goes down, it would probably just a change in properties of the link (bandwidth etc) and no link down event to MPLS-TP.

However if the detection at the higher layer MPLS-TP is faster it will treat the whole virtual link down and then UP with new properties when the lower layer detection happens.

Thanks,
Vishwas
On Fri, Feb 26, 2010 at 10:00 AM, Peng He <[email-masked]<mailto:[email-masked]>> wrote:
Hi Vishwas,

I agree that an Ethernet Link Aggregation group can be view as one
'virtual' interface to the MPLS-TP layer. Just a little confused here
that suppose a 4-link Ethernet AGG group, tnow one member/component
link is down then the traffic will be re-distributed to the remain 3
memner/component links of the AGG based on the AGG distribution
algorithm and the traffic can still  go through. At this moment,
should MPLS-TP view the (whole) 'virtual' interface down? or the
MPLS-TP views the 'virtual' interface as well as each individual
member/component link and consider it as a normal link down?

Regards,
Peng

On Fri, Feb 26, 2010 at 12:24 PM, Vishwas Manral <[email-masked]<mailto:[email-masked]>> wrote:
> Hi Rui,
>
> I have a slightly different view on this.
>
> If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dictate
> how it is to be done, if it is shown as one interface to the MPLS-TP layer.
> MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet link
> level will be visible to MPLS-TP as a link down and not which of the
> component links is down.
>
> Thanks,
> Vishwas
> On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]<mailto:[email-masked]>> wrote:
>>
>> Rui Costa wrote:
>>>
>>> Greetings,
>>>
>>> Once this draft has some content related to ETH interfaces (carrying
>>> MPLS-TP packets), I'd like to put the question below.
>>>  If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason
>>> is impredictability of BW distribution among LSPs while maintaining frame
>>> ordering (distributors' algorithms need to maintain "each conversation's
>>> frame order" and that implies hard BW distribution predictability and hence
>>> provisioning).
>>>
>>
>>> I imagine Ethernet link aggregation distributor algorithms to be very
>>> alike to MPLS ECMP's. If so, can we say that the similarity between ECMP
>>> (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has the same
>>> implicition for ETH link aggregation and hence it won't be applicable to
>>> MPLS-TP networks?
>>>
>>
>> _______________________________________________
>> mpls-tp mailing list
>> [email-masked]<mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/mpls-tp
>
>
> _______________________________________________
> mpls-tp mailing list
> [email-masked]<mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/mpls-tp
>
>

---

From: Rui Costa <[email-masked]>
Date: Wed, 3 Mar 2010 06:18:41 +0000

Thanks for answering,

I'm not sure i understood the "higher priority LSP". If we are essentially transporting/selling BW, do you mean priorities different for someone asking for 10G and someone asking for 100Mbps? Or, another way to put it: what if all customers have the same priority and pay for protection? This isn't the "Titanic's life boats" principle, is it? Do you mean "extra traffic" (i.e., customers that don't pay for protection)?

As David wrote, "a dilation factor is required": the planning reasoning 2x10G=20G is not true.

We could also run several distribution algorithms, simultaneously to the one in operation, and switch to the best one each time one of the links is reaching its top. Complex, there is always a no-use case, and each switch would potentiate frame misordering, being a violation to the only distributor requirement in 802.3 c43.

Regards,

Rui

From: Vishwas Manral [mailto:[email-masked]]
Sent: sexta-feira, 26 de Fevereiro de 2010 23:04
To: David Allan I
Cc: [email-masked]; Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

Hi David,
On Fri, Feb 26, 2010 at 12:07 PM, David Allan I <[email-masked]<mailto:[email-masked]>> wrote:
The one complication being that for CAC'd traffic, the aggregate capacity of the LAG in failure scenarios can drop below the level of the bandwidth committments associated with the paths that transit the LAG. In which case you want to move or protection switch SOME of the traffic.
Isn't this similar to the case where a higher priority LSP needs to preempt the lower piority one because the bandwidth on a link is exhausted?

The other complication is due to flow based assignment, the individual links in the LAG may not be being uniformly loaded...a dilation factor is required. And moving the traffic off the LAG as a response to partial failure reduces the statistical sample size for load spreading :-(....
That is correct. Thats a known limitation of LAGged interfaces with out without MPLS-TP.

Thanks,
Vishwas

I do not have solutions to this, I merely note the problems...

cheers
D

________________________________
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Vishwas Manral
Sent: Friday, February 26, 2010 12:24 PM
To: [email-masked]<mailto:[email-masked]>
Cc: Rui Costa; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?
Hi Rui,

If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dictate how it is to be done, if it is shown as one interface to the MPLS-TP layer. MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet link level will be visible to MPLS-TP as a link down and not which of the component links is down.

Thanks,
Vishwas

I would think that if the operator wanted the bandwidth more than they wanted the fate sharing, they will deploy the technology.

- Stewart

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 03 Mar 2010 03:30:22 -0500

In message <[email-masked]>
> wanted the fate sharing, they will deploy the technology.

There is no reason not to put some entropy in the extension to MPLS
Ping so that there was fate sharing over LAG.

Curtis

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2010 10:57:26 +0000

> I thought LAG's triggers would be PHY-alarm driven and not OAM. You mean "whole LAG" OAM, right? Wouldn't it still be quicker to infere from PHYs' alarms? (Just possible for LAG - not ECMP...)	
I am trying to understand how this works for a moment.

One purpose of client OAM is to audit the datapath.

How would a LAG client independently detect that that an element of the 
LAG was malfunctioning?

- Stewart

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 03 Mar 2010 10:59:39 +0000

Curtis

Do you have some text in mind?

- Stewart

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 3 Mar 2010 11:26:51 -0000

Surely the formation of the server layer and the use of OAM within the 
server layer is entirely an issue for the server layer.

It is a mistake for the MPLS-TP layer to be aware of the underlying 
technology that provides the link connectivity between to adjacent LSRs. The 
links provided by the server layer have certain resiliency and recovery 
properties, as well as properties associated with in-order (or not) packet 
delivery that are announced to the MPLS-TP network (usually through 
configuration) and the MPLS-TP network can choose which links to use 
accordingly.

LAG at a server Ethernet layer is a way of providing a composite link to the 
MPLS-TP layer. That link has specific properties, but the MPLS-TP layer 
cannot be expected to take specific measures to operate the link. That is 
the job of the server layer itself (noting that the adaptation into the LAG 
is a function of the server layer, just as the operation).

So, the MPLS-TP layer runs OAM across the link. It doesn't know how the link 
is formed. It is the responsibility of adaptation function to either 
distribute the MPLS-TP packets across the group members such that link 
degradation will be noticed by the MPLS-TP layer (this could be noted as a 
requirement that the MPLS-TP layer puts on the server layer that link 
degradation should be detectable by any measure of MPLS-TP packets); or the 
server layer must perform its own OAM to detect link degradation and report 
it to the MPLS-TP layer at the link end points.

LAG is not the only "complex" way of forming links in the MPLS-TP network 
from multiple links in the server network, and we really don't want to 
embark on making MPLS-TP understand each and every server technology.

Cheers,
Adrian

----- Original Message ----- 
From: "Stewart Bryant" <[email-masked]>
To: <[email-masked]>
Cc: "Rui Costa" <[email-masked]>; <[email-masked]>
Sent: Wednesday, March 03, 2010 10:59 AM
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => 
~ETH aggregation?

> [email-masked]

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 3 Mar 2010 07:25:39 -0800

Hi Rui

  I’m not sure i understood the "higher priority LSP". If we are essentially
> transporting/selling BW, do you mean priorities different for someone asking
> for 10G and someone asking for 100Mbps? Or, another way to put it: what if
> all customers have the same priority and pay for protection? This isn't the
> "Titanic's life boats" principle, is it? Do you mean "extra traffic" (i.e.,
> customers that don’t pay for protection)?
>
> I agree a dilation factor is required but that is a different discussion
altogather. What I meant was preemption because of a link going down or
because a new LSP needs to be transported is dealt in the same way
internally in the device.

By higher priority I meant reservations done at a particular priority where
a higher priority reservation can preempt a lower priority one. Have a look
at RFC3209 for RSVP-TE.

Thanks,
Vishwas

> We could also run several distribution algorithms, simultaneously to the
> one in operation, and switch to the best one each time one of the links is
> reaching its top. Complex, there is always a no-use case, and each switch
> would potentiate frame misordering, being a violation to the only
> distributor requirement in 802.3 c43.
>
>
>
> Regards,
>
> Rui
>
>
>
>
>
>
>
> *From:* Vishwas Manral [mailto:[email-masked]]
> *Sent:* sexta-feira, 26 de Fevereiro de 2010 23:04
> *To:* David Allan I
> *Cc:* [email-masked]; Rui Costa; [email-masked]
>
> *Subject:* Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00:
> ~ECMP => ~ETH aggregation?
>
>
>
> Hi David,
>
> On Fri, Feb 26, 2010 at 12:07 PM, David Allan I <
> [email-masked]> wrote:
>
> The one complication being that for CAC'd traffic, the aggregate capacity
> of the LAG in failure scenarios can drop below the level of the bandwidth
> committments associated with the paths that transit the LAG. In which case
> you want to move or protection switch SOME of the traffic.
>
> Isn't this similar to the case where a higher priority LSP needs to preempt
> the lower piority one because the bandwidth on a link is exhausted?
>
>
>
>
>
> The other complication is due to flow based assignment, the individual
> links in the LAG may not be being uniformly loaded...a dilation factor is
> required. And moving the traffic off the LAG as a response to partial
> failure reduces the statistical sample size for load spreading :-(....
>
>  ------------------------------
>
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Vishwas Manral
> *Sent:* Friday, February 26, 2010 12:24 PM
> *To:* [email-masked]
> *Cc:* Rui Costa; [email-masked]
> *Subject:* Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00:
> ~ECMP => ~ETH aggregation?
>
> Hi Rui,
>
>
>
> On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]>
> wrote:
>
> Rui Costa wrote:
>
> Greetings,
>
>  If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is
> impredictability of BW distribution among LSPs while maintaining frame
> I imagine Ethernet link aggregation distributor algorithms to be very alike
> to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs)
> and ETH aggregation (ETH interfaces; 802.3, c43) has the same implicition
> for ETH link aggregation and hence it won't be applicable to MPLS-TP
> networks?
>
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 3 Mar 2010 18:00:13 +0200

Adrian, Rui, and all,

I think that LAG presents two aspects of behavior that should be addressed (or at least noted) if we're going to use it in MPLS-TP.

1. Effective BW of LAG (treated as a single link) depends on actual client traffic carried across and on the way this traffic is adapted into LAG client layer. E.g., if distribution uses hashing on the label stack, the same client traffic pattern could result in different effective BW depending on actual label allocation for specific flows. This effect exists even when the number of "live" links comprising a LAG does not change. And IMHO this behavior is (more or less) specific for LAG. (Well, effective BW of an Ethernet link also depends on actual traffic due to IPG and Preamble overhead; but the effect is not so noticeable, and does not depend on the way client is adapted into LAG).
As a consequence, traffic engineering that involves LAG-based links looks more complicated.

2. Effective BW of LAG (treated as a single link) can change as its comprising links go down and up.
This behavior can be found in many other situations; VCAT with LCAS is one obvious example, and radio links with adaptive code modulation present another example (more difficult since link BW becomes a function of the weather conditions). 

This does not say that LAG cannot be used with MPLS-TP; only that its usage involves new challenges.

My 2c,
     Sasha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Wednesday, March 03, 2010 1:27 PM
To: [email-masked]; [email-masked]
Cc: Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

Cheers,
Adrian

----- Original Message ----- 
From: "Stewart Bryant" <[email-masked]>
To: <[email-masked]>
Cc: "Rui Costa" <[email-masked]>; <[email-masked]>
Sent: Wednesday, March 03, 2010 10:59 AM
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => 
~ETH aggregation?

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 00:30:28 -0500

In message <[email-masked]>
> Do you have some text in mind?

Remove from section 3.1.1:

   Per-packet Equal-Cost Multi-Path (ECMP) load-balancing is outside the
   scope of the MPLS-TP.

BTW- per packet load balance was the name given to the technique in
the 1990s of just round robin distribution across ECMP paths that
resulted in horrible performace problems if applied to WAN.  The IP
src/dst hash techniques were the "other technique".

Add a subsection (somewhere, your choice):

  3.x.x

   While Equal-Cost Multi-Path (ECMP) load-balancing is outside the
   scope of the MPLS-TP, its use is common particularly in link
   aggregation applications and may continue to be common.

   MPLS-TP implementations MAY support optional mechanisms to add
   entropy to the MPLS label stack when sending OAM traffic as defined
   for PW [draft-ietf-pwe3-fat-pw].  Inserting an additional entropy
   label under the GAL label would serve a similar purpose as the use
   as provided by the 127.x sender address space in MPLS Ping.

   If future MPLS implementations ignore the GAL label in EMCP
   hashing, but compute a hash based labels before and after this
   label, as if the GAL label was not present, then MPLS Ping can be
   used to disagnose the exact path, including the exact link
   aggregation members taken for a given client label stack.

Note that this would be better if we also had a draft that specified
using labels under GAL only for entropy.  The intermediate nodes that
see a lable above the GAL label (above being further from the BOS)
SHOULD ignore the GAL label in hashing.  When the GAL label is
encountered, all labels below the GAL label would be disposed of or
optionally delivered to the application.  I'm a little short on spare
time but maybe someone else might try.

In both PW there is never MPLS labels below the PW label except the
flow label used in flow label defined in draft-ietf-pwe3-fat-pw-03.
In GAL there is currently nothing below the GAL label but adding an
entropy label for OAM purposes just in case there is a LAG in the
middle would be useful, and at this stage not harmful since
implementations of GAL mostly just kick GAL along with everything else
in the reserved range up to the processor and no GAL specific
processing is yet cast in silicon.

Curtis

btw- you could probably word this better.

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 00:50:59 -0500

In message <21AFCCB59EFE4816B26E280DF3355CE2@your029b8cecfe>
> configuration) and the MPLS-TP network can choose which links to use 

The goal is to make OAM work by adding entropy.  If there is no
underlying LAG, then the entropy serves no purpose but does no harm.

If there is an underlying LAG then not having entropy allows the OAM
traffic to test only one member.  For example, if an MPLS LSP hop
(section) is a LAG, traffic within that MPLS LSP would take different
LAG members if the entire label stack was not the same for all traffic
or if the LSP carried IP traffic from more than one host pair.  If the
OAM traffic had a fixed label stack it would test only one LAG member.

> is a function of the server layer, just as the operation).

If CL is allowed for MPLS-TP, then OAM must work for CL.  We should
not knowingly just specify something that is broken.

> it to the MPLS-TP layer at the link end points.

For plain old MPLS, it doesn't know about any LAG either, but the
127.x source addresses add the entropy needed to provide effective
OAM.  That is why IETF specified MPLS Ping and and rejected 1711 and
why MPLS Ping works and 1711 doesn't work.

> embark on making MPLS-TP understand each and every server technology.

However, any method which is capable of supporting an LSP that is
greater in capacity than any one component link needs to look below
the top label to do so.  I think that is trivailly provable given that
the top level is always the same for that LSP.

Therefore no matter what we define CL to be, if this CL supports LSP
of greater than a LAG member size, a capability we currently have with
Ethernet LAG or ECMP over parallel links (another form of link
aggregation that predates Ethernet LAG by over a decade), then OAM
needs to provide entropy to insure that any CL/LAG/ECMP along the path
is exercised.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 01:05:44 -0500

In message <[email-masked]>
> As a consequence, traffic engineering that involves LAG-based links looks more complicated.

No it isn't.  TE has been used over LAG for over a decade.

If LAG balance is imperfect by some small percentage due to imperfect
distribution across the hash space, then the stated "reservable
bandwidth" can be similarly reduced.  In practice it isn't becuase LSP
reseervations in practice are predictions of expected load, not
guarentees not to exceed that load and are far less accurate than the
has inaccuracy.

If there is reason to believe that large microflows are present or can
come and go, then the total "reservable bandwidth" of the LAG is
reduced.  I've heard at least one provider refer to this as "derating
the LAG".  I'm not sure how common that term is, but the practice is
common.  It is not an operational hardship by any means.

Then there are adaptive link aggregation techniques of the type Avici
had used beginning over a decade ago.  These reduce the inaccuracy due
to large microflows to a minimum so you are back to the situation
where the LSP bandwidths are far less accurate and there is no need to
change the "reservable bandwidth" on the link.

> This behavior can be found in many other situations; VCAT with LCAS is one obvious example, and radio links with adaptive code modulation present another example (more difficult since link BW becom…

This is also not a big deal.  The reservable bandwidth is changed and
the preemption is run.  It is exactly what would happen if a high
priority LSP came along that was exactly the size of one link member.
Trust me on that - I designed and coded this for Avici.  It wasn't hard.

> This does not say that LAG cannot be used with MPLS-TP; only that its usage involves new challenges.

New to who?  For some, these were challenges in the late 1990s.

MPLS is not new and link aggregation has bee used with MPLS almost
since MPLS started out, though early on with a few parallel OC48,
unlike today with lots of parallel 10GbE.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 01:20:23 -0500

> I thought LAG's triggers would be PHY-alarm driven and not OAM. You mean "whole LAG" OAM, right? Wouldn't it still be quicker to infere from PHYs' alarms? (Just possible for LAG - not ECMP...)	

The issue is an LSP carried over the LAG where the LAG is a midpoint
hop.  If the forwarding fault is in the MPLS insegment, the OAM over
the LAG (section OAM for the LSP) would not pick up the error.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 01:31:45 -0500

Maybe the term LAG is confusing.  Link aggregation in practice is not
done just using Ethernet.  It is a common trick to advertise (into the
IGP) only the sum of a bundle of OC192 as well as 10GbE and with ISIS
in particular this isn't much of a change to what is advertised.

Avici had a protocol similar to LACP for POS back when Ethernet was
stuck at 1GbE with easy way to quickly detect a link fault and
therefore unusable by ISP.  I'm not sure of the details of how C&J
handle the non-Ethernet link aggregation.

It is therefore common to hear the term LAG used for both Ethernet and
non-Ethernet link aggregation.  Also providers use BFD or MEF OAM over
each member rather than rely on IEEE multiple second timers in LACP.
Once the fault is detected by BFD or MEF OAN, LACP can be used to
negociate the reduced LAG membership.

Curtis

In message <[email-masked]>
Rui Costa writes:

--===============1058375631==
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558EAINOAVREX11pti_"

--_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558EAINOAVREX11pti_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

"In the ideal case the detection time of the aggregate link going down at t=
he Ethernet Layer should be faster than that in the MPLS-TP layer. In such =
a case when a link goes down, it would probably just a change in properties=
 of the link (bandwidth etc) and no link down event to MPLS-TP"

[Rui] By "LAG" we don't then mean 802.3 c43 / 802.3ad LACP (timers multiple=
 of 1s) but just the aggregation/distribution algorithms "PHY alarm trigger=
ed", right? (Protocol and hence Key/aggregatability_check being ruled out)

From: Vishwas Manral [mailto:[email-masked]]
Sent: sexta-feira, 26 de Fevereiro de 2010 18:39
To: Peng He
Cc: [email-masked]; Rui Costa; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP =
=3D> ~ETH aggregation?

Hi Peng,

In the ideal case the detection time of the aggregate link going down at th=
e Ethernet Layer should be faster than that in the MPLS-TP layer. In such a=
 case when a link goes down, it would probably just a change in properties =
of the link (bandwidth etc) and no link down event to MPLS-TP.

However if the detection at the higher layer MPLS-TP is faster it will trea=
t the whole virtual link down and then UP with new properties when the lowe=
r layer detection happens.

Thanks,
Vishwas
On Fri, Feb 26, 2010 at 10:00 AM, Peng He <[email-masked]<mailto:pe=
[email-masked]>> wrote:
Hi Vishwas,

Regards,
Peng

On Fri, Feb 26, 2010 at 12:24 PM, Vishwas Manral <[email-masked]<ma=
ilto:[email-masked]>> wrote:
> how it is to be done, if it is shown as one interface to the MPLS-TP laye=
r.
> On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]<mailt=
o:[email-masked]>> wrote:
>>>  If I understood correctly, ECMP is out of MPLS-TP. I suppose the reaso=
n
>>> is impredictability of BW distribution among LSPs while maintaining fra=
me
>>> ordering (distributors' algorithms need to maintain "each conversation'=
s
>>> frame order" and that implies hard BW distribution predictability and h=
ence
>>> alike to MPLS ECMP's. If so, can we say that the similarity between ECM=
P
>>> implicition for ETH link aggregation and hence it won't be applicable t=
o
>> wanted the fate sharing, they will deploy the technology.

--_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558EAINOAVREX11pti_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
	{mso-style-priority:99;
	mso-style-link:"Balloon Text Char";
	margin:0cm;
	margin-bottom:.0001pt;
	font-size:8.0pt;
	font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
	{mso-style-name:"Balloon Text Char";
	mso-style-priority:99;
	mso-style-link:"Balloon Text";
	font-family:"Tahoma","sans-serif";}
span.EmailStyle19
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page Section1
	{size:612.0pt 792.0pt;
	margin:70.85pt 3.0cm 70.85pt 3.0cm;}
div.Section1
	{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext=3D"edit">
  <o:idmap v:ext=3D"edit" data=3D"1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=3DPT link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:11.0pt;font-fami=
ly:"Calibri","sans-serif";
color:#1F497D'>&#8220;</span><span lang=3DEN-US>In the ideal case the detec=
tion
time of the aggregate link going down at the Ethernet Layer should be faste=
r
than that in the MPLS-TP layer. In such a case when a link goes down, it wo=
uld probably
just a change in properties of the link (bandwidth etc) and no link down ev=
ent
to MPLS-TP</span><span lang=3DEN-US style=3D'font-size:11.0pt;font-family:"=
Calibri","sans-serif";
color:#1F497D'>&#8221;</span><span lang=3DEN-US style=3D'color:#1F497D'><o:=
p></o:p></span></p>

<p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:11.0pt;font-fami=
ly:"Calibri","sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

<p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:11.0pt;font-fami=
ly:"Calibri","sans-serif";
color:#1F497D'>[Rui] By &#8220;LAG&#8221; we don&#8217;t then mean 802.3 c4=
3 /
802.3ad LACP (timers multiple of 1s) but just the aggregation/distribution
algorithms &#8220;PHY alarm triggered&#8221;, right? (Protocol and hence
Key/aggregatability_check being ruled
out)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <o:p></o:p></span></p>

<p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:11.0pt;font-fami=
ly:"Calibri","sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

<div style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm =
0cm 0cm'>

<p class=3DMsoNormal><b><span lang=3DEN-US style=3D'font-size:10.0pt;font-f=
amily:
"Tahoma","sans-serif"'>From:</span></b><span lang=3DEN-US style=3D'font-siz=
e:10.0pt;
font-family:"Tahoma","sans-serif"'> Vishwas Manral
[mailto:[email-masked]] <br>
<b>Sent:</b> sexta-feira, 26 de Fevereiro de 2010 18:39<br>
<b>To:</b> Peng He<br>
<b>Cc:</b> [email-masked]; Rui Costa; [email-masked]<br>
<b>Subject:</b> Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00:
~ECMP =3D&gt; ~ETH aggregation?<o:p></o:p></span></p>

</div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

<div>

<p class=3DMsoNormal>Hi Peng,<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>This is what should happen.<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>In the ideal case the detection time of the aggregate =
link
going down at the Ethernet Layer should be faster than that in the MPLS-TP
layer. In such a case when a link goes down, it would probably just a chang=
e in
properties of the link (bandwidth etc) and no link down event to MPLS-TP.<o=
:p></o:p></p>

<p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:11.0pt;font-fami=
ly:"Calibri","sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

</div>

<div>

<p class=3DMsoNormal><span lang=3DEN-US>&nbsp;<o:p></o:p></span></p>

</div>

<div>

<p class=3DMsoNormal>However if the detection at the higher layer MPLS-TP i=
s
faster it will treat the whole virtual link down and then UP with new
properties when the lower layer detection happens.<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>Thanks,<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>Vishwas<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal>On Fri, Feb 26, 2010 at 10:00 AM, Peng He &lt;<a
href=3D"mailto:[email-masked]">[email-masked]</a>&gt; wrote=
:<o:p></o:p></p>

<p class=3DMsoNormal>Hi Vishwas,<br>
<br>
I agree that an Ethernet Link Aggregation group can be view as one<br>
'virtual' interface to the MPLS-TP layer. Just a little confused here<br>
that suppose a 4-link Ethernet AGG group, tnow one member/component<br>
link is down then the traffic will be re-distributed to the remain 3<br>
memner/component links of the AGG based on the AGG distribution<br>
algorithm and the traffic can still &nbsp;go through. At this moment,<br>
should MPLS-TP view the (whole) 'virtual' interface down? or the<br>
MPLS-TP views the 'virtual' interface as well as each individual<br>
member/component link and consider it as a normal link down?<br>
<br>
<br>
Regards,<br>
<span style=3D'color:#888888'>Peng</span><o:p></o:p></p>

<div>

<div>

<p class=3DMsoNormal><br>
On Fri, Feb 26, 2010 at 12:24 PM, Vishwas Manral &lt;<a
href=3D"mailto:[email-masked]">[email-masked]</a>&gt; wrote=
:<br>
&gt; Hi Rui,<br>
&gt;<br>
&gt; I have a slightly different view on this.<br>
&gt;<br>
&gt; If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dict=
ate<br>
&gt; how it is to be done, if it is shown as one interface to the MPLS-TP
layer.<br>
&gt; MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet lin=
k<br>
&gt; level will be visible to MPLS-TP as a link down and not which of the<b=
r>
&gt; component links is down.<br>
&gt;<br>
&gt; Thanks,<br>
&gt; Vishwas<br>
&gt; On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant &lt;<a
href=3D"mailto:[email-masked]">[email-masked]</a>&gt; wrote:<br>
&gt;&gt;<br>
&gt;&gt; Rui Costa wrote:<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; Greetings,<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; Once this draft has some content related to ETH interfaces
(carrying<br>
&gt;&gt;&gt; MPLS-TP packets), I'd like to put the question below.<br>
&gt;&gt;&gt; &nbsp;If I understood correctly, ECMP is out of MPLS-TP. I sup=
pose
the reason<br>
&gt;&gt;&gt; is impredictability of BW distribution among LSPs while
maintaining frame<br>
&gt;&gt;&gt; ordering (distributors' algorithms need to maintain &quot;each
conversation's<br>
&gt;&gt;&gt; frame order&quot; and that implies hard BW distribution
predictability and hence<br>
&gt;&gt;&gt; provisioning).<br>
&gt;&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; .. and the inability for the OAM to fate share with the LSP.<br>
&gt;&gt;&gt;<br>
&gt;&gt;&gt; I imagine Ethernet link aggregation distributor algorithms to =
be
very<br>
&gt;&gt;&gt; alike to MPLS ECMP's. If so, can we say that the similarity
between ECMP<br>
&gt;&gt;&gt; (LSPs) and ETH aggregation (ETH interfaces; 802.3, c43) has th=
e same<br>
&gt;&gt;&gt; implicition for ETH link aggregation and hence it won't be
applicable to<br>
&gt;&gt;&gt; MPLS-TP networks?<br>
&gt;&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; I would think that if the operator wanted the bandwidth more than =
they<br>
&gt;&gt; wanted the fate sharing, they will deploy the technology.<br>
&gt;&gt;<br>
&gt;&gt; - Stewart<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt;<br>
&gt;&gt; _______________________________________________<br>
&gt;&gt; mpls-tp mailing list<br>
&gt;&gt; <a href=3D"mailto:[email-masked]">[email-masked]</a><br>
&gt;&gt; <a href=3D"https://www.ietf.org/mailman/listinfo/mpls-tp" target=
=3D"_blank">https://www.ietf.org/mailman/listinfo/mpls-tp</a><br>
&gt;<br>
&gt;<br>
&gt; _______________________________________________<br>
&gt; mpls-tp mailing list<br>
&gt; <a href=3D"mailto:[email-masked]">[email-masked]</a><br>
&gt; <a href=3D"https://www.ietf.org/mailman/listinfo/mpls-tp" target=3D"_b=
lank">https://www.ietf.org/mailman/listinfo/mpls-tp</a><br>
&gt;<br>
&gt;<o:p></o:p></p>

</div>

</div>

</div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

</div>

</body>

</html>

--_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558EAINOAVREX11pti_--

--===============1058375631==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

--===============1058375631==--

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 05 Mar 2010 01:37:50 -0500

In message <[email-masked]>
Rui Costa writes:

>  
> Thanks for answering,
>  
>  
>  
> I'm not sure i understood the "higher priority LSP". If we are essentially =
> transporting/selling BW, do you mean priorities different for someone askin=
> g for 10G and someone asking for 100Mbps? Or, another way to put it: what i=
> f all customers have the same priority and pay for protection? This isn't t=
> he "Titanic's life boats" principle, is it? Do you mean "extra traffic" (i.=
> e., customers that don't pay for protection)?

See "Setup Priority" and "Holding Priority" in RFC 3209.

Or in the terms you've used "if [...] transporting/selling BW", it is
the customer paying extra to be the highest priority.  It may be the
US DoD, emergency services, or it could be anyone.

> As David wrote, "a dilation factor is required": the planning reasoning 2x1=
> 0G=3D20G is not true.
>  
> We could also run several distribution algorithms, simultaneously to the on=
> e in operation, and switch to the best one each time one of the links is re=
> aching its top. Complex, there is always a no-use case, and each switch wou=
> ld potentiate frame misordering, being a violation to the only distributor =
> requirement in 802.3 c43.

How IP src/dst hash works is getting a bit off topic and has been
explained already, though somewhat briefly.

Curtis

> On Fri, Feb 26, 2010 at 12:07 PM, David Allan I <[email-masked]=
> <mailto:[email-masked]>> wrote:
> The one complication being that for CAC'd traffic, the aggregate capacity o=
> f the LAG in failure scenarios can drop below the level of the bandwidth co=
> mmittments associated with the paths that transit the LAG. In which case yo=
> u want to move or protection switch SOME of the traffic.
> Isn't this similar to the case where a higher priority LSP needs to preempt=
> The other complication is due to flow based assignment, the individual link=
> s in the LAG may not be being uniformly loaded...a dilation factor is requi=
> red. And moving the traffic off the LAG as a response to partial failure re=
> duces the statistical sample size for load spreading :-(....
> That is correct. Thats a known limitation of LAGged interfaces with out wit=
> hout MPLS-TP.
>  
> Thanks,
> Vishwas
>  
> From: [email-masked]<mailto:[email-masked]> [mailto:mpl=
> [email-masked]<mailto:[email-masked]>] On Behalf Of Vishwa=
> s Manral
> If Link aggregation is done in the Ethernet layer, MPLS-TP cannot dictate h=
> ow it is to be done, if it is shown as one interface to the MPLS-TP layer. =
> MPLS-TP OAM works at the MPLS level and so a fault at the Ethernet link lev=
> el will be visible to MPLS-TP as a link down and not which of the component=
>  links is down.
>  
> Thanks,
> Vishwas
> On Fri, Feb 26, 2010 at 4:04 AM, Stewart Bryant <[email-masked]<mailto:=
> [email-masked]>> wrote:
> Rui Costa wrote:
> Greetings,
>  
> Once this draft has some content related to ETH interfaces (carrying MPLS-T=
> P packets), I'd like to put the question below.
>  If I understood correctly, ECMP is out of MPLS-TP. I suppose the reason is=
>  impredictability of BW distribution among LSPs while maintaining frame ord=
> ering (distributors' algorithms need to maintain "each conversation's frame=
>  order" and that implies hard BW distribution predictability and hence prov=
> isioning).
>  
> I imagine Ethernet link aggregation distributor algorithms to be very alike=
>  to MPLS ECMP's. If so, can we say that the similarity between ECMP (LSPs) =
> and ETH aggregation (ETH interfaces; 802.3, c43) has the same implicition f=
> or ETH link aggregation and hence it won't be applicable to MPLS-TP network=
> s?
>  
> I would think that if the operator wanted the bandwidth more than they want=
> ed the fate sharing, they will deploy the technology.
>  
> - Stewart
>  
>  
>  
>  
> --_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558E8INOAVREX11pti_
> xmlns:x=3D"urn:schemas-microsoft-com:office:excel" xmlns:p=3D"urn:schemas-m=
> icrosoft-com:office:powerpoint" xmlns:a=3D"urn:schemas-microsoft-com:office=
> :access" xmlns:dt=3D"uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s=3D"=
> uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs=3D"urn:schemas-microsof=
> t-com:rowset" xmlns:z=3D"#RowsetSchema" xmlns:b=3D"urn:schemas-microsoft-co=
> m:office:publisher" xmlns:ss=3D"urn:schemas-microsoft-com:office:spreadshee=
> t" xmlns:c=3D"urn:schemas-microsoft-com:office:component:spreadsheet" xmlns=
> :odc=3D"urn:schemas-microsoft-com:office:odc" xmlns:oa=3D"urn:schemas-micro=
> soft-com:office:activation" xmlns:html=3D"http://www.w3.org/TR/REC-html40" =
> xmlns:q=3D"http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc=3D"http://m=
> icrosoft.com/officenet/conferencing" xmlns:D=3D"DAV:" xmlns:Repl=3D"http://=
> schemas.microsoft.com/repl/" xmlns:mt=3D"http://schemas.microsoft.com/share=
> point/soap/meetings/" xmlns:x2=3D"http://schemas.microsoft.com/office/excel=
> /2003/xml" xmlns:ppda=3D"http://www.passport.com/NameSpace.xsd" xmlns:ois=
> =3D"http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir=3D"http://=
> schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds=3D"http://www.w3=
> .org/2000/09/xmldsig#" xmlns:dsp=3D"http://schemas.microsoft.com/sharepoint=
> /dsp" xmlns:udc=3D"http://schemas.microsoft.com/data/udc" xmlns:xsd=3D"http=
> ://www.w3.org/2001/XMLSchema" xmlns:sub=3D"http://schemas.microsoft.com/sha=
> repoint/soap/2002/1/alerts/" xmlns:ec=3D"http://www.w3.org/2001/04/xmlenc#"=
>  xmlns:sp=3D"http://schemas.microsoft.com/sharepoint/" xmlns:sps=3D"http://=
> schemas.microsoft.com/sharepoint/soap/" xmlns:xsi=3D"http://www.w3.org/2001=
> /XMLSchema-instance" xmlns:udcs=3D"http://schemas.microsoft.com/data/udc/so=
> ap" xmlns:udcxf=3D"http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udc=
> p2p=3D"http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf=3D"http:/=
> /schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss=3D"http://sche=
> mas.microsoft.com/office/2006/digsig-setup" xmlns:dssi=3D"http://schemas.mi=
> crosoft.com/office/2006/digsig" xmlns:mdssi=3D"http://schemas.openxmlformat=
> s.org/package/2006/digital-signature" xmlns:mver=3D"http://schemas.openxmlf=
> ormats.org/markup-compatibility/2006" xmlns:m=3D"http://schemas.microsoft.c=
> om/office/2004/12/omml" xmlns:mrels=3D"http://schemas.openxmlformats.org/pa=
> ckage/2006/relationships" xmlns:spwp=3D"http://microsoft.com/sharepoint/web=
> partpages" xmlns:ex12t=3D"http://schemas.microsoft.com/exchange/services/20=
> 06/types" xmlns:ex12m=3D"http://schemas.microsoft.com/exchange/services/200=
> 6/messages" xmlns:pptsl=3D"http://schemas.microsoft.com/sharepoint/soap/Sli=
> deLibrary/" xmlns:spsl=3D"http://microsoft.com/webservices/SharePointPortal=
> Server/PublishedLinksService" xmlns:Z=3D"urn:schemas-microsoft-com:" xmlns:=
> st=3D"&#1;" xmlns=3D"http://www.w3.org/TR/REC-html40">
>  
> <head>
> <!--[if !mso]>
> v\:* {behavior:url(#default#VML);}
> o\:* {behavior:url(#default#VML);}
> w\:* {behavior:url(#default#VML);}
> .shape {behavior:url(#default#VML);}
> </style>
> <![endif]-->
> <style>
> <!--
>  /* Font Definitions */
>  @font-face
> 	{font-family:Calibri;
> 	{font-family:Consolas;
> 	panose-1:2 11 6 9 2 2 4 3 2 4;}
>  /* Style Definitions */
> p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
> 	{mso-style-priority:99;
> 	mso-style-link:"Plain Text Char";
> 	margin:0cm;
> 	margin-bottom:.0001pt;
> 	font-size:10.5pt;
> 	font-family:Consolas;}
> span.PlainTextChar
> 	{mso-style-name:"Plain Text Char";
> 	mso-style-priority:99;
> 	mso-style-link:"Plain Text";
> 	font-family:Consolas;}
> span.BalloonTextChar
> span.EmailStyle21
> <p class=3DMsoPlainText><span lang=3DEN-US>Thanks for answering,&nbsp;&nbsp=
> ; <o:p></o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US>I&#8217;m not sure i understood =
> the
> &quot;higher priority LSP&quot;. If we are essentially transporting/selling=
>  BW,
> do you mean priorities different for someone asking for 10G and someone ask=
> ing for
> 100Mbps? Or, another way to put it: what if all customers have the same
> priority and pay for protection? This isn't the &quot;Titanic's life
> boats&quot; principle, is it? Do you mean &quot;extra traffic&quot; (i.e.,
> customers that don&#8217;t pay for protection)?&nbsp;&nbsp; <o:p></o:p></sp=
> an></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US>As David wrote, &quot;a dilation=
>  factor
> is required&quot;: the planning reasoning 2x10G=3D20G is not true. <o:p></o=
> :p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US>We could also run several distri=
> bution
> algorithms, simultaneously to the one in operation, and switch to the best =
> one
> each time one of the links is reaching its top. Complex, there is always a
> no-use case, and each switch would potentiate frame misordering, being a
> violation to the only distributor requirement in 802.3 c43.&nbsp;&nbsp;&nbs=
> p; <o:p></o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US>Regards,&nbsp;&nbsp;&nbsp; <o:p>=
> </o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US>Rui<o:p></o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <p class=3DMsoPlainText><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <b>Sent:</b> sexta-feira, 26 de Fevereiro de 2010 23:04<br>
> <b>To:</b> David Allan I<br>
> <p class=3DMsoNormal><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> <p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><span lang=3DEN-US>Hi D=
> avid,<o:p></o:p></span></p>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>On Fri, Feb 26, 2010 at 12:07 PM, D=
> avid
> Allan I &lt;</span><a href=3D"mailto:[email-masked]"><span
> lang=3DEN-US>[email-masked]</span></a><span lang=3DEN-US>&gt; w=
> rote:<o:p></o:p></span></p>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:10.0pt;font-fami=
> ly:"Arial","sans-serif";
> color:blue'>The one complication being that for CAC'd traffic, the aggregat=
> e
> capacity of the LAG in failure scenarios can drop below the level of the
> bandwidth committments&nbsp;associated with the paths that transit the LAG.=
>  In
> which case you want to move or protection switch SOME of the traffic.</span=
> ><span
> lang=3DEN-US><o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Isn't this similar to the case wher=
> e a
> higher priority LSP needs to preempt the lower piority one because the
> bandwidth on a link is exhausted?<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <blockquote style=3D'border:none;border-left:solid #CCCCCC 1.0pt;padding:0c=
> m 0cm 0cm 6.0pt;
> margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:10.0pt;font-fami=
> ly:"Arial","sans-serif";
> color:blue'>The other complication is due to flow based assignment, the
> individual links in the LAG&nbsp;may not be&nbsp;being uniformly loaded...a
> dilation factor is required. And moving the traffic off the LAG as a respon=
> se
> to partial failure reduces the statistical sample size for load spreading
> :-(....</span><span lang=3DEN-US><o:p></o:p></span></p>
>  
> </div>
>  
> </blockquote>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>That is correct. Thats a known limi=
> tation
> of LAGged interfaces with out without MPLS-TP.<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Thanks,<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Vishwas<o:p></o:p></span></p>
>  
> </div>
>  
> <blockquote style=3D'border:none;border-left:solid #CCCCCC 1.0pt;padding:0c=
> m 0cm 0cm 6.0pt;
> margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:10.0pt;font-fami=
> ly:"Arial","sans-serif";
> color:blue'>I do not have solutions to this, I merely note the problems...<=
> /span><span
> lang=3DEN-US><o:p></o:p></span></p>
>  
> <p class=3DMsoNormal><span style=3D'font-size:10.0pt;font-family:"Arial","s=
> ans-serif";
> color:blue'>cheers</span><o:p></o:p></p>
>  
> <p class=3DMsoNormal><span style=3D'font-size:10.0pt;font-family:"Arial","s=
> ans-serif";
> color:blue'>D</span><o:p></o:p></p>
>  
> <div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><span lan=
> g=3DEN-US>
>  
> <hr size=3D2 width=3D"100%" align=3Dcenter>
>  
> </span></div>
>  
> <p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><b><span lang=3DEN-US
> style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></=
> b><span
> lang=3DEN-US style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> =
> <a
> href=3D"mailto:[email-masked]" target=3D"_blank">mpls-tp-bounces@=
> ietf.org</a>
> [mailto:<a href=3D"mailto:[email-masked]" target=3D"_blank">mpls-=
> [email-masked]</a>]
> <b>On Behalf Of </b>Vishwas Manral<br>
> <b>Sent:</b> Friday, February 26, 2010 12:24 PM<br>
> <b>To:</b> <a href=3D"mailto:[email-masked]" target=3D"_blank">stbryant=
> @cisco.com</a><br>
> <b>Cc:</b> Rui Costa; <a href=3D"mailto:[email-masked]" target=3D"_blank"=
> >[email-masked]</a><br>
> ~ECMP =3D&gt; ~ETH aggregation?</span><span lang=3DEN-US><o:p></o:p></span>=
> </p>
>  
> <div>
>  
> <div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Hi Rui,<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>I have a slightly different view on=
>  this.<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>If Link aggregation is done in the =
> Ethernet
> layer, MPLS-TP cannot dictate how it is to be done, if it is shown as one
> interface to the MPLS-TP layer. MPLS-TP OAM works at the MPLS level and so =
> a
> fault at the Ethernet link level will be visible to MPLS-TP as a link down =
> and
> not which of the component links is down.<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Thanks,<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Vishwas<o:p></o:p></span></p>
>  
> </div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>On Fri, Feb 26, 2010 at 4:04 AM, St=
> ewart
> Bryant &lt;</span><a href=3D"mailto:[email-masked]" target=3D"_blank"><=
> span
> lang=3DEN-US>[email-masked]</span></a><span lang=3DEN-US>&gt; wrote:<o:=
> p></o:p></span></p>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Rui Costa wrote:<o:p></o:p></span><=
> /p>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>Greetings, &nbsp; &nbsp; &nbsp;<br>
> <br>
> Once this draft has some content related to ETH interfaces (carrying MPLS-T=
> P
> packets), I'd like to put the question below. &nbsp; &nbsp; &nbsp;<br>
> &nbsp;If I understood correctly, ECMP is out of MPLS-TP. I suppose the reas=
> on
> ordering (distributors' algorithms need to maintain &quot;each conversation=
> 's
> frame order&quot; and that implies hard BW distribution predictability and
> hence provisioning). &nbsp; <br>
> &nbsp;<o:p></o:p></span></p>
>  
> </div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>.. and the inability for the OAM to=
>  fate
> share with the LSP.<o:p></o:p></span></p>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>I imagine Ethernet link aggregation
> distributor algorithms to be very alike to MPLS ECMP's. If so, can we say t=
> hat
> the similarity between ECMP (LSPs) and ETH aggregation (ETH interfaces; 802=
> .3,
> c43) has the same implicition for ETH link aggregation and hence it won't b=
> e
> applicable to MPLS-TP networks? &nbsp;<br>
> &nbsp;<o:p></o:p></span></p>
>  
> <p class=3DMsoNormal><span lang=3DEN-US>I would think that if the operator =
> wanted
> the bandwidth more than they wanted the fate sharing, they will deploy the
> technology.<br>
> <span style=3D'color:#888888'><br>
> - Stewart</span> <o:p></o:p></span></p>
>  
> <div>
>  
> <div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US><br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> mpls-tp mailing list<br>
> </span><a href=3D"mailto:[email-masked]" target=3D"_blank"><span lang=3DE=
> N-US>[email-masked]</span></a><span
> lang=3DEN-US><br>
> </span><a href=3D"https://www.ietf.org/mailman/listinfo/mpls-tp" target=3D"=
> _blank"><span
> lang=3DEN-US>https://www.ietf.org/mailman/listinfo/mpls-tp</span></a><span
> lang=3DEN-US><o:p></o:p></span></p>
>  
> </div>
>  
> </div>
>  
> </div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> </div>
>  
> </div>
>  
> </div>
>  
> </blockquote>
>  
> </div>
>  
> <p class=3DMsoNormal><span lang=3DEN-US><o:p>&nbsp;</o:p></span></p>
>  
> </div>
>  
> </body>
>  
> </html>
>  
> --_000_52981DB05D3C5247A12D0AEE309F3CC2F73E0558E8INOAVREX11pti_--
>  
> --===============0757647195==
> --===============0757647195==--
>

---

From: <[email-masked]>
Date: Fri, 5 Mar 2010 09:14:47 +0000

None of this is LAG specific the problem space is broadly the same for all
server layer links that employ inverse multiplexing to offer higher
bandwidth than a single link in the server layer is capable of supporting.

I don't think we want to get bogged down in the details but just note that
server layer paths may in reality be composed of multiple links presented to
MPLS-TP as a single server layer path and that special care is required when
designing for such scenarios.

Ben

On 03/03/2010 16:00, "Alexander Vainshtein"
<[email-masked]> wrote:

> Adrian, Rui, and all,
> 
> I think that LAG presents two aspects of behavior that should be addressed (or
> at least noted) if we're going to use it in MPLS-TP.
> 
> 1. Effective BW of LAG (treated as a single link) depends on actual client
> traffic carried across and on the way this traffic is adapted into LAG client
> layer. E.g., if distribution uses hashing on the label stack, the same client
> traffic pattern could result in different effective BW depending on actual
> label allocation for specific flows. This effect exists even when the number
> of "live" links comprising a LAG does not change. And IMHO this behavior is
> (more or less) specific for LAG. (Well, effective BW of an Ethernet link also
> depends on actual traffic due to IPG and Preamble overhead; but the effect is
> not so noticeable, and does not depend on the way client is adapted into LAG).
> This behavior can be found in many other situations; VCAT with LCAS is one
> obvious example, and radio links with adaptive code modulation present another
> example (more difficult since link BW becomes a function of the weather
> conditions). 
>

---

From: David Allan I <[email-masked]>
Date: Fri, 5 Mar 2010 08:13:01 -0500

HI Curtis:

Entropy and OAM is a bit of a flashback....

I agree that in a perfect world that adding it would be trivial and useful....

However for a GAL/GACH encapsulated packet, adding entropy is a huge change exclusively for the OAM system, n'est pas? It would have to go after the MEP ID TLVs for randomization...not exactly a desirable plan..

Hmmm
D

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Curtis Villamizar
Sent: Friday, March 05, 2010 12:51 AM
To: Adrian Farrel
Cc: [email-masked]; Rui Costa
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

"Adrian Farrel" writes:
> technology that provides the link connectivity between to adjacent 
> LSRs. The links provided by the server layer have certain resiliency 
> and recovery properties, as well as properties associated with 
> in-order (or not) packet delivery that are announced to the MPLS-TP 
> network (usually through

The goal is to make OAM work by adding entropy.  If there is no underlying LAG, then the entropy serves no purpose but does no harm.

If there is an underlying LAG then not having entropy allows the OAM traffic to test only one member.  For example, if an MPLS LSP hop
(section) is a LAG, traffic within that MPLS LSP would take different LAG members if the entire label stack was not the same for all traffic or if the LSP carried IP traffic from more than one host pair.  If the OAM traffic had a fixed label stack it would test only one LAG member.

> LAG at a server Ethernet layer is a way of providing a composite link 
> to the MPLS-TP layer. That link has specific properties, but the 
> MPLS-TP layer cannot be expected to take specific measures to operate 
> the link. That is the job of the server layer itself (noting that the 
> adaptation into the LAG is a function of the server layer, just as the operation).

If CL is allowed for MPLS-TP, then OAM must work for CL.  We should not knowingly just specify something that is broken.

> So, the MPLS-TP layer runs OAM across the link. It doesn't know how 
> the link is formed. It is the responsibility of adaptation function to 
> either distribute the MPLS-TP packets across the group members such 
> that link degradation will be noticed by the MPLS-TP layer (this could 
> be noted as a requirement that the MPLS-TP layer puts on the server 
> layer that link degradation should be detectable by any measure of 
> MPLS-TP packets); or the server layer must perform its own OAM to 
> detect link degradation and report it to the MPLS-TP layer at the link end points.

For plain old MPLS, it doesn't know about any LAG either, but the 127.x source addresses add the entropy needed to provide effective OAM.  That is why IETF specified MPLS Ping and and rejected 1711 and why MPLS Ping works and 1711 doesn't work.

> LAG is not the only "complex" way of forming links in the MPLS-TP 
> network from multiple links in the server network, and we really don't 
> want to embark on making MPLS-TP understand each and every server technology.
>  
> Cheers,
> Adrian

However, any method which is capable of supporting an LSP that is greater in capacity than any one component link needs to look below the top label to do so.  I think that is trivailly provable given that the top level is always the same for that LSP.

Therefore no matter what we define CL to be, if this CL supports LSP of greater than a LAG member size, a capability we currently have with Ethernet LAG or ECMP over parallel links (another form of link aggregation that predates Ethernet LAG by over a decade), then OAM needs to provide entropy to insure that any CL/LAG/ECMP along the path is exercised.

Curtis

> >> In message <[email-masked]> Stewart Bryant writes:
> >>
> >>>  I would think that if the operator wanted the bandwidth more than 
> >>> they wanted the fate sharing, they will deploy the technology.
> >>>  - Stewart
> >>>
> >>
> >>

---

From: "Adrian Farrel" <[email-masked]>
Date: Fri, 5 Mar 2010 19:18:51 -0000

Hi Curtis,

I think I have a fundamental disagreement with you about layering.

I believe that a client layer is independent of its server layer, and that a 
server layer is independent of the client layer.

So the OAM in the MPLS-TP layer is testing connectivity in the MPLS-TP 
layer. If the server layer has "interesting" features, these must be tested 
using OAM in the server layer, and faults or degradations detected in the 
server layer will be reported at the MPLS-TP link ends.

For example, when building a composite link from a bundle of OC192s to 
advertise as a single link in the IGP, I suspect that we run BFD over the IP 
link to test for overall connectivity, and we run OAM on each OC192 to check 
its connectivity. Any OC192 failure will result in a degradation being 
reported at the IP link ends.

If we did try to architect MPLS-TP OAM to handle every potential type of 
server technology and aggregation technique (and every possible way of 
hashing data onto component links) I suspect we would end up with 
outrageously complex MPLS-Tp OAM.

Cheers,
Adrian
----- Original Message ----- 
From: "Curtis Villamizar" <[email-masked]>
To: "Adrian Farrel" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; "Rui Costa" 
<[email-masked]>; <[email-masked]>
Sent: Friday, March 05, 2010 5:50 AM
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => 
~ETH aggregation?

>> >>> they

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 05 Mar 2010 14:34:43 -0500

Adrian,

    While I agree %100 with you about layering violations, I am not sure
that Curtis is asking for one. I think what he is asking for a mechanism to
exercise the entropy of the server layer if it exists, as a general means of
interfacing between the client/server layers. It would seem like a
reasonable part of the "interface" between the two layers to be able to
specify this, but only if done in a way independent of the specific server
type, because as you say, that would create a nightmare of combinations for
the client to support.  It might be good to add this as an additional
requirement to the MPLS-TP (general or OAM) requirements.

    --Tom

---

From: David Allan I <[email-masked]>
Date: Fri, 5 Mar 2010 14:46:29 -0500

Such a requirement could only be considered if it did not ADD an additional point of failure...

An entopy mechanism unique to the OAM subsystem would fall under the cateogy of things that added points of failure, hence was actually detremental to overall system reliability...

My 2 cents
D 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Thomas D. Nadeau
Sent: Friday, March 05, 2010 2:35 PM
To: Adrian Farrel; [email-masked]
Cc: [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?

    Adrian,

    While I agree %100 with you about layering violations, I am not sure that Curtis is asking for one. I think what he is asking for a mechanism to exercise the entropy of the server layer if it exists, as a general means of interfacing between the client/server layers. It would seem like a reasonable part of the "interface" between the two layers to be able to specify this, but only if done in a way independent of the specific server type, because as you say, that would create a nightmare of combinations for the client to support.  It might be good to add this as an additional requirement to the MPLS-TP (general or OAM) requirements.

    --Tom

On 3/5/10 2:18 PM, "Adrian Farrel" <[email-masked]> wrote:

> I believe that a client layer is independent of its server layer, and 
> that a server layer is independent of the client layer.
> 
> layer. If the server layer has "interesting" features, these must be 
> tested using OAM in the server layer, and faults or degradations 
> detected in the server layer will be reported at the MPLS-TP link ends.
> 
> advertise as a single link in the IGP, I suspect that we run BFD over 
> the IP link to test for overall connectivity, and we run OAM on each 
> OC192 to check its connectivity. Any OC192 failure will result in a 
> degradation being reported at the IP link ends.
> 
> If we did try to architect MPLS-TP OAM to handle every potential type 
> of server technology and aggregation technique (and every possible way 
> of hashing data onto component links) I suspect we would end up with 
>>> Surely the formation of the server layer and the use of OAM within 
>>> the server layer is entirely an issue for the server layer.
>>> 
>>> links provided by the server layer have certain resiliency and 
>>> recovery properties, as well as properties associated with in-order 
>>> (or not) packet delivery that are announced to the MPLS-TP network 
>>> (usually through
>> LAG members if the entire label stack was not the same for all 
>> traffic or if the LSP carried IP traffic from more than one host 
>> pair.  If the OAM traffic had a fixed label stack it would test only one LAG member.
>> 
>>> LAG at a server Ethernet layer is a way of providing a composite 
>>> link to the MPLS-TP layer. That link has specific properties, but 
>>> the MPLS-TP layer cannot be expected to take specific measures to 
>>> operate the link. That is the job of the server layer itself (noting 
>>> that the adaptation into the LAG is a function of the server layer, 
>>> just as the operation).
>> 
>>> the link is formed. It is the responsibility of adaptation function 
>>> to either distribute the MPLS-TP packets across the group members 
>>> such that link degradation will be noticed by the MPLS-TP layer 
>>> (this could be noted as a requirement that the MPLS-TP layer puts on 
>>> the server layer that link degradation should be detectable by any 
>>> measure of MPLS-TP packets); or the server layer must perform its 
>>> own OAM to detect link degradation and report it to the MPLS-TP 
>>> layer at the link end points.
>> 
>>> network from multiple links in the server network, and we really 
>>> don't want to embark on making MPLS-TP understand each and every server technology.
>>> 
>>> Cheers,
>>> Adrian
>> 
>> the top label to do so.  I think that is trivailly provable given 
>> that the top level is always the same for that LSP.
>> 
>> of greater than a LAG member size, a capability we currently have 
>> with Ethernet LAG or ECMP over parallel links (another form of link 
>> needs to provide entropy to insure that any CL/LAG/ECMP along the 
>> path is exercised.
>> 
>> Curtis
>> 
>> 
>>>>>>  I would think that if the operator wanted the bandwidth more 
>>>>>> than they wanted the fate sharing, they will deploy the 
>>>>>> technology.
>>>>>>  - Stewart
>>>>>> 
>>>>> 
>>>>> 
>>>>> There is no reason not to put some entropy in the extension to 
>>>>> MPLS Ping so that there was fate sharing over LAG.
>>>>> 
>>>>> Curtis
>>>>> 
>>>>> 
>>>> Curtis
>>>>

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 05 Mar 2010 14:53:04 -0500

I am not sure this would be unique to OAM; its directly related to the
underlying data plane (and trouble-shooting it).  In our client/server layer
parlance, its unique to one type of supported server layer, but concievably
could be of others in the future.

    --Tom

On 3/5/10 2:46 PM, "David Allan I" <[email-masked]> wrote:

> An entopy mechanism unique to the OAM subsystem would fall under the cateogy
> of things that added points of failure, hence was actually detremental to
> overall system reliability...
> 
> My 2 cents
> D 
> 
>     While I agree %100 with you about layering violations, I am not sure that
> Curtis is asking for one. I think what he is asking for a mechanism to
> interfacing between the client/server layers. It would seem like a reasonable
> part of the "interface" between the two layers to be able to specify this, but
> only if done in a way independent of the specific server type, because as you
> say, that would create a nightmare of combinations for the client to support.
> It might be good to add this as an additional requirement to the MPLS-TP
> (general or OAM) requirements.
> 
>     --Tom
> 
> 
>

---

From: <[email-masked]>
Date: Fri, 5 Mar 2010 21:38:44 -0000

Adrian....bottom-line is you are technically right...more in-line: 

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Adrian Farrel
> Sent: 05 March 2010 19:19
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [mpls-tp] Comments on 
> draft-fbb-mpls-tp-data-plane-00: ~ECMP =>~ETH aggregation?
> 
> Hi Curtis,
> 

NH=> Join the queue!
> 
> I believe that a client layer is independent of its server 
> layer, and that a server layer is independent of the client layer.

NH=> This is the way it should be Adrian.  Unfortunately this is not the
case with MPLS.  The mere fact that PWs exist shows that MPLS failed to
get layering right.  MPLS-TP really should fix this if it wants to be
taken seriously as a transport network.  Sadly, this does not look like
it is going to happen....hard-nosed commercials and soft-nosed
face-saving/embarassment hold-sway.  
> 
> So the OAM in the MPLS-TP layer is testing connectivity in 
> the MPLS-TP layer. If the server layer has "interesting" 
> features, these must be tested using OAM in the server layer, 
> and faults or degradations detected in the server layer will 
> be reported at the MPLS-TP link ends.

NH=> Totally correct Adrian.  But there is a little more to it than just
this:

1	The principle service a server layer transport network provides
to its clients is *transparency* (of information transported).  This has
several implications, but a key one is that the server layer must not
attempt to understand the information contained in the client
symbols...and it for sure must not change these client symbols.  LAG and
any type inverse muxing in some server layer network must make specific
assumptions not only about the client traffic unit structure but also
about the importance (also see next item) of client traffic
units.....this type of DPI/snooping is a very, very bad idea for a
server layer transport network....indeed, it indicates an immediate
failure of purpose as it obviously violates the key transparency
requirement.

2	In all networking cases there must be a top-of-stack network
technology present that supports the external message/file/stream
applications.  IP is dominant in this role.  Most other technologies,
including Ethernet where LAG came from and also MPLS, currently cannot
fulfil this role.  It is only in this top-of-stack network that one can
say anything meaningful about importance.....noting that:
-	importance has no relationship to the application type....any
message/file/stream application can assume any importance...and this is
why all successful transport networks (to date all such network are
co-cs mode) must make no judgement on this metric, ie all client *bits*
(a bit is the atomic information symbol) are treated equally and
ordering of bits is preserved....this is also why a transport network
must not have any QoS (sic) classes, and indeed the co-cs mode
does-not/cannot have any QoS classes.
-	the above is not simply wrt *within* client instance X but
applies *across* all client instances of the server layer....in other
words it is even more meaningless to try and make importance judgements
*between* applications in different client service instances
-	in a cl-ps network the traffic unit must carry the importance
semantic...indeed the traffic unit must carry full information labels
wrt SA, DA and PID.....one can take no labelling short-cuts in the cl-ps
mode
-	in a co-ps or co-cs network network it is the parent connection
that carries the importance semantic and *not* the child traffic
units...and it is because the parent connection contains implicit
information itself that we can take some labelling short-cuts in the
child traffic units, eg omit SA, omit PID, reduce scope of DA label to
link unique.

> 
> For example, when building a composite link from a bundle of 
> OC192s to advertise as a single link in the IGP, I suspect 
> that we run BFD over the IP link to test for overall 
> connectivity, and we run OAM on each OC192 to check its 
> connectivity. Any OC192 failure will result in a degradation 
> being reported at the IP link ends.

> 
> If we did try to architect MPLS-TP OAM to handle every 
> potential type of server technology and aggregation technique 
> (and every possible way of hashing data onto component links) 
> I suspect we would end up with outrageously complex MPLS-Tp OAM.

NH=> Exactly so, this is why we need client and server independence.
What you also saying (a little differently to how I would) is that we do
not have transparency here.  It's obvious from Curtis's comments he does
not grasp these issues very well....which is why he also thinks adding
arbitrary 'entropy' 127/8 SA labels in *client* IP OAM ping pkts is a
good way to test the ECMP layering violations that usually come with
*server layer* LDP.  The problem here is the server layer merging LDP
and its layering violating ECMP bedfellow.  Adding entropy (what a grand
name for stating we have no idea what we are testing!) is not a great
OAM idea in a transport network as Dave as already pointed out. 

regards, Neil
> Subject: Re: [mpls-tp] Comments on 
> draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation?
> 
> 
> >
> >> technology that provides the link connectivity between to 
> adjacent LSRs.
> >> The
> >> such that link degradation will be noticed by the MPLS-TP 
> layer (this 
> >> could be noted as a requirement that the MPLS-TP layer puts on the 
> >> server layer that link degradation should be detectable by any 
> >> measure of MPLS-TP packets); or the server layer must 
> perform its own 
> >> OAM to detect link degradation and report it to the 
> MPLS-TP layer at 
> >> the link end points.
> >
> >> Subject: Re: [mpls-tp] Comments on 
> draft-fbb-mpls-tp-data-plane-00:

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 07 Mar 2010 14:11:59 -0500

In message <[email-masked]>
> However for a GAL/GACH encapsulated packet, adding entropy is a huge change exclusively for the OAM system, n'est pas? It would have to go after the MEP ID TLVs for randomization...not exactly a de…

Its not a huge change at all.  The receiving side of OAM need only
look for the BOS bit rather than assume the payload is behind the PW
label or GAL.

For PW a method to negociate whether both sides know about this
extension is already specificied in the fat-pw draft.  Therefore the
work is already done for PW and it is just GAL that needs a minor
change wrt GAL BOS mandate and interpretation of BOS.  It would be
preferable if, as already defined for PW, a negociation for use with
GAL preceeded enabling this entropy encaps to be sure the other end
could handle it.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 07 Mar 2010 14:54:29 -0500

In message <2BFD791F1A2A401DB0CBC435E0441E21@your029b8cecfe>
> outrageously complex MPLS-Tp OAM.

The client is not aware of how the service is being delivered but
needs a means to reliably determine if the service is being delivered.
Without the entropy there are situations where the client cannot
detect that the service has a fault which is undetected at the service
layer.  Testing a LAG member does not test all insegments and hash
values mapping through the forwarding hardware to that LAG member.

This is a real world problem and insegment problems (entire insegment
mapping broken in the problems of a decade ago, particularly for one
vendor) is the main reaon that MPLS has OAM capability today (BFD,
MPLS Ping, MPLS traceroute).  Reducing the reliability of that
capability in MPLS-TP would be a step backwards.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 07 Mar 2010 15:23:20 -0500

In message <[email-masked]>
> An entopy mechanism unique to the OAM subsystem would fall under the cateogy of things that added points of failure, hence was actually detremental to overall system reliability...

No additional point of failure is added.  I think fat-pw is already in
use in non MPLS-TP context.

Maybe an example would help.

Consider layer A, B, C.  B is a client of A.  C is a client of B.
Lets not try to assign administrative boundaries as it is irrelevant
at this point.  At layer B it is helpful to be able to verify that the
service being provided at layer A actually works.  There is no way for
layer B to predict or control the label stack added by layer C, but
that label stack creates entropy in the payload that layer B must
provide.  If the OAM does not have any entropy, then layer B cannot
verify that its own LSP or PW is delivering the service promised to
layer C.

If there is no LAG within layer A, then the additional label in the
stack has no effect since layer A looks only at the top of stack for
both ordinary payload and layer B OAM traffic.  If there is a LAG
within layer A, then all members of the LAG are exercised, although
somewhat randomly.  With entropy, layer B and can detect an error
within layer A, but cannot diagnose it.

Using PW with fat-pw (draft-ietf-pwe3-fat-pw-03) the two ends
negociate to determine that both ends can handle an extra label below
the PW label.  If so the stack has a various TE labels, the PW label,
and then the entropy label, then a PW control word that indicates
either client payload or GACH and OAM.  The existing MPLS Ping uses a
127.x destination address to add entropy.  The extension to MPLS-TP
extension to MPLS Ping can use fat-pw with PW but needs an equivalent
capability with GAL.

Putting an entropy label at the BOS below the GAL label provides an
equivalent solution to fat-pw.  Vendors who recognize a need for this
have two options.  Just do it as a configured option and float an
informational draft with no signaling or define signaling for use with
GAL that is equivalent to the fat-pw signaling.

Curtis

---

From: Maarten Vissers <[email-masked]>
Date: Mon, 08 Mar 2010 09:24:48 +0100

Curtis,

MPLS-TP and other packet transport technologies can be run over a compound
link. Such compound link can be provided by multiple section layer transport
paths or by multiple edge-to-edge LSPs. In both cases the nodes at the edge
of such compound link have to make sure that all traffic within a
service/transport path is transported via only one link within the compound
link. This to maintain e.g. the order of packets/frames within the
service/transport path.

To deploy the survivability provided by compound links, ITU-T Q.9/15 has
developed the Adaptive Compound Link SubNetwork Group protection with
Inherent monitoring (ACL-SNCG/I) mechanism. This ACL-SNCG/I protection
controls the distribution of 
1) PWs, service-LSPs and service path layer PST LSPs over a service path
layer component links (supported by edge-to-edge LSPs) within a compound
link,
2) edge-to-edge LSPs and transport path layer PST LSPs over a transport path
layer component links (supported by sections)within a compound link .

The initial work done assumed the presence of non-adaptive media, which
would cause a component link to be completely available or completely
unavailable (as in ethernet LAG). At the moment this work is being extended
to also support adaptive media, which are able to reduce their bandwidth to
a lower level; i.e. creating a partly available state in between the
original two states.

Input to the distribution process is the CIR(EIR) and relative priority of
each service path or transport path that is carried over the compound link
and the available bandwidth of each component link within the compound link.
The distribution process computes the distribution of the service paths, or
transport paths. If the bandwidth of the compound link becomes too small to
support all, then the lowest priority service/transport paths are blocked
and packets/frames of those paths are not longer forwarded over the compound
link.

A service path or transport path's bandwidth must always be smaller then the
bandwidth of the component links.

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Curtis Villamizar
Sent: vrijdag 5 maart 2010 6:51
To: Adrian Farrel
Cc: [email-masked]; Rui Costa
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

> configuration) and the MPLS-TP network can choose which links to use 

The goal is to make OAM work by adding entropy.  If there is no underlying
LAG, then the entropy serves no purpose but does no harm.

If there is an underlying LAG then not having entropy allows the OAM traffic
to test only one member.  For example, if an MPLS LSP hop
(section) is a LAG, traffic within that MPLS LSP would take different LAG
members if the entire label stack was not the same for all traffic or if the
LSP carried IP traffic from more than one host pair.  If the OAM traffic had
a fixed label stack it would test only one LAG member.

> adaptation into the LAG is a function of the server layer, just as the
operation).

If CL is allowed for MPLS-TP, then OAM must work for CL.  We should not
knowingly just specify something that is broken.

> detect link degradation and report it to the MPLS-TP layer at the link end
points.

For plain old MPLS, it doesn't know about any LAG either, but the 127.x
source addresses add the entropy needed to provide effective OAM.  That is
why IETF specified MPLS Ping and and rejected 1711 and why MPLS Ping works
and 1711 doesn't work.

> want to embark on making MPLS-TP understand each and every server
technology.

However, any method which is capable of supporting an LSP that is greater in
capacity than any one component link needs to look below the top label to do
so.  I think that is trivailly provable given that the top level is always
the same for that LSP.

Therefore no matter what we define CL to be, if this CL supports LSP of
greater than a LAG member size, a capability we currently have with Ethernet
LAG or ECMP over parallel links (another form of link aggregation that
predates Ethernet LAG by over a decade), then OAM needs to provide entropy
to insure that any CL/LAG/ECMP along the path is exercised.

Curtis

---

From: David Allan I <[email-masked]>
Date: Mon, 8 Mar 2010 05:42:48 -0500

Hi Curtis:

I agree than in the absence of snooping of ACH TLVs an entropy label is the only other alternative to add entropy specifically to OAM flows.

However if you assume labels are the only input into entropy, you have a problem in that a LAG would then decouple the OAM from the flow it is to fate share with simply because of the existence of the GAL. Adding an entropy label would only muddy the waters further...

Something would think a LAG broke if it broke, but somewhat unlikely the right thing would know...OTOH that would be the status quo anyway...with or without...

Hmmmm
D

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Sunday, March 07, 2010 3:23 PM
To: David Allan I
Cc: Thomas D. Nadeau; Adrian Farrel; [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP => ~ETH aggregation? 

Curtis

---

From: Phil Bedard <[email-masked]>
Date: Mon, 8 Mar 2010 13:22:56 -0500

This seems similar to RFC 4201 which covers signaling for bundled links, as opposed to LAG.   The strongest point being no single LSP/flow can be larger than a single member link.  In the RSVP-TE case you use setup/hold priority and bandwidth reservations to determine which LSPs stay during times of reduced bandwidth.   I think Curtis (or perhaps someone else) mentioned allowing MPLS-TP to ONLY use bundling and not LAG.  

The problem can be solved by just not using LAG or link bundling however this increases the amount of information in the network, in the case of dynamic link information dissemination and LSP path provisioning.  It also does not allow one to oversubscribe a single link, which is one of the main reason people use LAG today.  In our network, we have LSPs >10G and an optical infrastructure only capable of supporting OTU2 transport... We rely on the entropy of data in the higher layer for even distribution.  

I don't think we want to specify the mechanisms for distributing traffic across the underlying lower layer links which make up the LAG.  Like has previously been said, there are many imux techniques for different Layer1 technologies.  Some use distribution techniques like byte interleaving which may not work well with mapping a single flow to a physical component.    Operators are well aware of the risks of using LAG such as giant flows on a single member link, differential delay, etc.   I think it just needs to be noted.    

Phil

---

From: John E Drake <[email-masked]>
Date: Tue, 9 Mar 2010 06:52:03 -0800

Adrian,

I interpreted Curtis to mean that the MPLS layer was managing the bundled link itself.  If the bundled link was managed by a lower layer, that lower layer wouldn't necessarily understand the label stack, including the entropy label.  One could almost say that, by definition, if the layer managing the bundled link understands the label stack, it is part of the MPLS layer.

I think it would be useful to remove the stipulation that for MPLS-TP the GAL must have the BOS bit set.

Thanks,

John

> Of Adrian Farrel
> Sent: Wednesday, March 03, 2010 3:27 AM
> => ~ETH aggregation?
> =>

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 10 Mar 2010 14:41:37 +0000

Curtis

Whilst operation over link bundles is an important issue, it will take 
some time to fully understand the implications and design a solution.

In the interests of completing our committed deliverables to the ITU-T 
in a timely manner, can I propose that we continue to exclude them from 
the current MPLS-TP drafts.  Can I also propose that in the meanwhile, 
you or others that wish to pursue technology to support link bundles, 
publish drafts which update the MPLS-TP specifications and describe the 
necessary requirements and technical solutions to address this problem.

- Stewart

-- 
For corporate legal information go to:

http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 10 Mar 2010 14:44:54 +0000

[email-masked] wrote:
> .... but just note that
We will add text to this effect.

Stewart

---

From: Peng He <[email-masked]>
Date: Thu, 11 Mar 2010 04:37:43 -0500

I agree with Phil on this ' Operators are well aware of the risks of
using LAG such as giant flows on a single member link, differential
delay, etc...I think it just needs to be noted..'..

Regards,
Peng

---

From: "tom.petch" <[email-masked]>
Date: Thu, 11 Mar 2010 20:11:04 +0100

Neil

It is time you wrote
 draft-harrison-mpls-tp-getting-layering-right-00.txt
I could write it myself with what I have learnt from you but you
would still do it much better than I.

It is needed. Whether or not it is adopted by MPLS TP, the IETF needs
it on record.  MPLS is a very successful protocol, yet it could have been
significantly better, and that now shows with efforts to enhance it (shades
of IPv4:-).

The necessary changes may not happen this time around, but unless what
you say is recorded in the traditional archive of the IETF, the RFC series,
then the IETF may make the same mistakes again, with this or another
successful(!) protocol.

Tom Petch

----- Original Message -----
From: <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, March 05, 2010 10:38 PM
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP=>~ETH
aggregation?

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 14 Mar 2010 00:11:58 -0500

In message <[email-masked]>
Maarten Vissers writes:
>  
> Curtis,
>  
> MPLS-TP and other packet transport technologies can be run over a
> compound link. Such compound link can be provided by multiple section
> layer transport paths or by multiple edge-to-edge LSPs. In both cases
> the nodes at the edge of such compound link have to make sure that all
> traffic within a service/transport path is transported via only one
> link within the compound link. This to maintain e.g. the order of
> packets/frames within the service/transport path.
>  
> To deploy the survivability provided by compound links, ITU-T Q.9/15
> has developed the Adaptive Compound Link SubNetwork Group protection
> with Inherent monitoring (ACL-SNCG/I) mechanism. This ACL-SNCG/I
> protection controls the distribution of 1) PWs, service-LSPs and
> service path layer PST LSPs over a service path layer component links
> (supported by edge-to-edge LSPs) within a compound link, 2)
> edge-to-edge LSPs and transport path layer PST LSPs over a transport
> path layer component links (supported by sections)within a compound
> link .
>  
> The initial work done assumed the presence of non-adaptive media,
> which would cause a component link to be completely available or
> completely unavailable (as in ethernet LAG). At the moment this work
> is being extended to also support adaptive media, which are able to
> reduce their bandwidth to a lower level; i.e. creating a partly
> available state in between the original two states.
>  
> Input to the distribution process is the CIR(EIR) and relative
> priority of each service path or transport path that is carried over
> the compound link and the available bandwidth of each component link
> within the compound link.  The distribution process computes the
> distribution of the service paths, or transport paths. If the
> bandwidth of the compound link becomes too small to support all, then
> the lowest priority service/transport paths are blocked and
> packets/frames of those paths are not longer forwarded over the
> compound link.
>  
> A service path or transport path's bandwidth must always be smaller
> then the bandwidth of the component links.
>  
> Regards,
> Maarten

Maarten,

Thank you for reminding me that the ITU is ignoring reality.  That is
nothing new though.

 ->    Input to the distribution process is the CIR(EIR)

The reality is that when LSP hierarchy is used (to reduce the number
of labels and reduce the amount of signaling that needs to occur on a
fault and in doing so improve scaling) or when MPLS is applied to a
small high bandwidth core, the LSPs are signaled with BW values that
are greater than the size of a single LAG/CL member.  This is where
src/dst or label stack based hash has worked very well for about a
decade.  This information is not in the signaling of the outer LSP.

Using src/dst or label stack based has insures that no microflow is
reordered (this is important) but does reorder traffic within the
outer LSP (this doesn't matter).

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 14 Mar 2010 00:21:26 -0500

In message <[email-masked]>
David Allan I writes:
>  
> Hi Curtis:
>  
> I agree than in the absence of snooping of ACH TLVs an entropy label
> is the only other alternative to add entropy specifically to OAM
> flows.

And since existing hardware, just about any core router built in the
last decade, already hashes on the label stack, this works with
existing forwarding hardware where snooping the ACH TLVs would not.

> However if you assume labels are the only input into entropy, you have
> a problem in that a LAG would then decouple the OAM from the flow it
> is to fate share with simply because of the existence of the
> GAL. Adding an entropy label would only muddy the waters further...

Hardware should ingore labels 0-15 in the hash computation.  If
existing hardware does not, then new hardware should.  In the single
label on the label stack case if LAG behavior decoupled the GAL
traffic from payload, an entropy label on the OAM would insure that
the path taken by traffic was tested at least some of the time.

> Something would think a LAG broke if it broke, but somewhat unlikely
> the right thing would know...OTOH that would be the status quo
> anyway...with or without...

Did not parse.

What I think you were getting at is that some intuition says that if a
member is broken the server layer would know about it.  Where that is
not the case is where the member is carrying traffic but has an
insegment misprogramming on one or more ASIC.  The server layer
injects traffic beyond that point and exercises only the outsegment.

---

From: Maarten Vissers <[email-masked]>
Date: Mon, 15 Mar 2010 09:08:31 +0100

Curtis,

> Using src/dst or label stack based has insures that no microflow
> is reordered (this is important) but does reorder traffic within
> the outer LSP (this doesn't matter).

In a transport network with performance monitoring it **does matter** if
packets with the **same CoS** are forwarded over different links.
Performance monitoring will provide incorrect results in such case.

When you put a Section LSP over a LAG/CL, then there is 1 Section LSP and
performance monitoring will not be functioning. 

The better alternative is to put the set of edge-to-edge LSPs over a LAG/CL
and to use an Ethernet Section 'VLAN' per link. Now there will be many
edge-to-edge LSPs that have to traverse the LAG/CL and those can be
distributed over the links on the basis of their edge-to-edge LSP label
value.

When the the LAG/CL is between PE nodes, then you can use LAG or ACL-SNCG/I
for the set of PW, Service-LSP and transport service layer PST-LSP
connections between two PEs. The distribution is on the basis of PW,
service-LSP or PST-LSP label values in this case.

A further alternative is to disable performance monitoring of the
edge-to-edge LSPs. When this is done, then it is possible to do LAG/CL
between two P nodes on the basis of the PW, service-LSP and PST-LSP label
values. The performance monitoring in the transport service layer
connections is not impacted by this approach. 

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: zondag 14 maart 2010 6:12
To: Maarten Vissers
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

Maarten,

The reality is that when LSP hierarchy is used (to reduce the number of
labels and reduce the amount of signaling that needs to occur on a fault and
in doing so improve scaling) or when MPLS is applied to a small high
bandwidth core, the LSPs are signaled with BW values that are greater than
the size of a single LAG/CL member.  This is where src/dst or label stack
based hash has worked very well for about a decade.  This information is not
in the signaling of the outer LSP.

Using src/dst or label stack based has insures that no microflow is
reordered (this is important) but does reorder traffic within the outer LSP
(this doesn't matter).

Curtis

> Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: 
> ~ECMP =>~ETH aggregation?
>  
>  
> > just as the
> > don't want to embark on making MPLS-TP understand each and every 
> > server
> technology.
> >  
> > Cheers,
> > Adrian
>  
> needs to provide entropy to insure that any CL/LAG/ECMP along the path is
exercised.
> > >>> than they wanted the fate sharing, they will deploy the technology.
> > >>>  - Stewart
> > >>>
> > >>
> > >>

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 15 Mar 2010 11:12:08 +0200

Curtis, Maarten and all:
My reading of Maarten's comment is like following:

	Y.1731-style packet loss measurement will not work correctly across LAG.

Please note that this is correct in the "pure Ethernet" situation; MPLS-TP does not add anything new here.
I suspect that this was one of the reasons why IEEE has not included performance monitoring mechanisms in 802.1ag; it would be nice if the people that have participated in these discussions could confirm (or disprove) this.

To the best of my knowledge, the only requirement that can be interpreted as a suggestion to use Y.1731-style packet loss measurements is given in  Section 2.2.1 "Packet Loss Measurement" of the MPLS-TP OAM Requirements draft (http://www.ietf.org/id/draft-ietf-mpls-tp-oam-requirements-06.txt):

<quote>
   It SHOULD be possible to rely on user traffic to perform that
   functionality.
<end quote> 

("That functionality" refers to packet loss ration measurement).

Quoting from RFC 2119:

<quote>
3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
   may exist valid reasons in particular circumstances to ignore a
   particular item, but the full implications must be understood and
   carefully weighed before choosing a different course.
<end quote>

IMHO and FWIW this text ideally applies to usage (or non-usage) of LAG-based Sections in MPLS-TP.

My 2c,
     Sasha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Maarten Vissers
Sent: Monday, March 15, 2010 10:09 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP =>~ETH aggregation?

Curtis,

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: zondag 14 maart 2010 6:12
To: Maarten Vissers
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

Maarten,

Curtis

exercised.

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 18 Mar 2010 23:56:49 -0400

In message <[email-masked]>

Maarten,

Its sort of amusing when someone asserts that something that has been
deployed for over a decade and works extremely well can't work.

It was amusing when ITU tried to create a profile for packet networks
in the mid 1990s and asserted that IP would never be suitable for
business because it didn't have the strict QoS that ITU had declared
manditory.

It was also amusing when ITU folks asserted that voice could never be
run over IP becaus the jitter was too high and it was too unreliable.

So now you are asserting that we can't run "real transport" traffic
because we don't have the equivalent PM.  This is despite the fact
that provider like Level3 using L2VPN, then PW, have already proven it
to be very cost effective and quite satisfactory.

We are going to have to agree to disagree for now.  For those who
value PM above all else, link bundling is the only way to go.  For
those that want to create LSP larger than a LAG member and have a more
dynamic distribution of load, LAG may be a better choice.  You assert
that only the latter is valid.  I assert that *both* are valid choices
and that we should simply acknowledge that and let the market decide.

So I propose that we agree to disagree and make a concerted effort to
support both the LAG behaviour that is deployed and working and the
link bundling mode that some MPLS-TP advocates prefer.

It is fine to say that implementations MUST support link bundling and
MAY support LAG and then put in the OAM entropy needed to support LAG
that won't hurt the link bundling case.  It is even easy enough to
make small enhancements to signaling to identify nodes capapable of
link bundle only or LAG and link bundle and those capable of carving
out single member capacity for link bundle LSP from capacity otherwise
used for LSP over LAG.

Curtis

---

From: Maarten Vissers <[email-masked]>
Date: Fri, 19 Mar 2010 07:35:39 +0100

Curtis,

It will not be amusing for an operator when he deploys LAG as you describe
in which 
- the frames with Priority X of a p2p EVC (monitored with Y.1731 Frame
  Loss Measurement OAM) are distributed over multiple links in the LAG 
- causing a reordering of these frames
- resulting in the detection of lost frames or unexpected additional frames.

The reason this will not be amusing for the operator is that his EVC support
resutls in reporting of Severely Errored Seconds and possibly Unavailable
time by his UNI-N MEP functions and by the customer's UNI-C MEP functions,
thus causing non-compliance with the SLA for this EVC service.

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: vrijdag 19 maart 2010 4:57
To: Maarten Vissers
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

Maarten Vissers writes:
> In a transport network with performance monitoring it **does matter** 
> if packets with the **same CoS** are forwarded over different links.
> When you put a Section LSP over a LAG/CL, then there is 1 Section LSP 
> and performance monitoring will not be functioning.
>  
> The better alternative is to put the set of edge-to-edge LSPs over a 
> LAG/CL and to use an Ethernet Section 'VLAN' per link. Now there will 
> be many edge-to-edge LSPs that have to traverse the LAG/CL and those 
> can be distributed over the links on the basis of their edge-to-edge 
> LSP label value.
>  
> When the the LAG/CL is between PE nodes, then you can use LAG or 
> ACL-SNCG/I for the set of PW, Service-LSP and transport service layer 
> PST-LSP connections between two PEs. The distribution is on the basis 
> of PW, service-LSP or PST-LSP label values in this case.
>  
> between two P nodes on the basis of the PW, service-LSP and PST-LSP 
> label values. The performance monitoring in the transport service 
> layer connections is not impacted by this approach.
>  
> Regards,
> Maarten

Maarten,

It was amusing when ITU tried to create a profile for packet networks in the
mid 1990s and asserted that IP would never be suitable for business because
it didn't have the strict QoS that ITU had declared manditory.

It was also amusing when ITU folks asserted that voice could never be run
over IP becaus the jitter was too high and it was too unreliable.

So now you are asserting that we can't run "real transport" traffic because
we don't have the equivalent PM.  This is despite the fact that provider
like Level3 using L2VPN, then PW, have already proven it to be very cost
effective and quite satisfactory.

We are going to have to agree to disagree for now.  For those who value PM
above all else, link bundling is the only way to go.  For those that want to
create LSP larger than a LAG member and have a more dynamic distribution of
load, LAG may be a better choice.  You assert that only the latter is valid.
I assert that *both* are valid choices and that we should simply acknowledge
that and let the market decide.

So I propose that we agree to disagree and make a concerted effort to
support both the LAG behaviour that is deployed and working and the link
bundling mode that some MPLS-TP advocates prefer.

It is fine to say that implementations MUST support link bundling and MAY
support LAG and then put in the OAM entropy needed to support LAG that won't
hurt the link bundling case.  It is even easy enough to make small
enhancements to signaling to identify nodes capapable of link bundle only or
LAG and link bundle and those capable of carving out single member capacity
for link bundle LSP from capacity otherwise used for LSP over LAG.

Curtis

> > compound link. Such compound link can be provided by multiple 
> > section layer transport paths or by multiple edge-to-edge LSPs. In 
> > both cases the nodes at the edge of such compound link have to make 
> > sure that all traffic within a service/transport path is transported 
> > via only one link within the compound link. This to maintain e.g. 
> > the order of packets/frames within the service/transport path.
> >  
> > service path layer PST LSPs over a service path layer component 
> > links (supported by edge-to-edge LSPs) within a compound link, 2) 
> > bandwidth of the compound link becomes too small to support all, 
> > then the lowest priority service/transport paths are blocked and 
> > > It is a mistake for the MPLS-TP layer to be aware of the 
> > > underlying technology that provides the link connectivity between 
> > > to adjacent LSRs. The links provided by the server layer have 
> > > certain resiliency and recovery properties, as well as properties 
> > > associated with in-order (or not) packet delivery that are 
> > > announced to the MPLS-TP network (usually through
> > > configuration) and the MPLS-TP network can choose which links to 
> > > use accordingly.
> >  
> > (section) is a LAG, traffic within that MPLS LSP would take 
> > different LAG members if the entire label stack was not the same for 
> > all traffic or if the LSP carried IP traffic from more than one host 
> > > operate the link. That is the job of the server layer itself 
> > > (noting that the adaptation into the LAG is a function of the 
> > > server layer, just as the
> > operation).
> >  
> > > So, the MPLS-TP layer runs OAM across the link. It doesn't know 
> > > how the link is formed. It is the responsibility of adaptation 
> > > function to either distribute the MPLS-TP packets across the group 
> > > members such that link degradation will be noticed by the MPLS-TP 
> > > layer (this could be noted as a requirement that the MPLS-TP layer 
> > > puts on the server layer that link degradation should be 
> > > detectable by any measure of MPLS-TP packets); or the server layer 
> > > must perform its own OAM to detect link degradation and report it 
> > > to the MPLS-TP layer at the link end
> > points.
> >  
> > path is
technology.

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 21 Mar 2010 00:02:58 -0400

In message <[email-masked]>
> - resulting in the detection of lost frames or unexpected additional frames.

Y.1731 is Ethernet OAM and would be inside a PW.  No MAC to MAC flow
will be reordered as long as the PW T-PE doesn't do something that
would cause reordering.

The single Ethernet PW is not a challenge.  Todays LAG of a few
hundred Gb/s, likely to be a Tb/s by the time MPLS-TP is deployed, is
where LAG is needed.

MPLS Ping is the current MPLS OAM and as long as we don't break what works
today in "enhancing" it to be MPLS-TP, then it will continue to work fine.

Of course, if MPLS-TP is only applicable to low bandwidth used, then
there is no issue.

> thus causing non-compliance with the SLA for this EVC service.

You seen to be assuming that Y.1731 is MPLS-TP OAM.  That is not
expected to be the case.  The requirements for CC/CV, etc, will be
met, but not with Y.1731.  When IETF rejected Y.1711 as an MPLS OAM
and adopted MPLS Ping, it was with good reason.  We should not break
MPLS Ping in adapting to a MPLS-TP profile.

Our goal should be to let MPLS-TP support link bundle PW carrying
Ethernet with no reordering at all, let MPLS-TP support PW over LAS
distributing traffic on a MAC pair (or other) basis over LAG using
flow label (fat-pw), and let the large LSP over LAG the that exist
today use MPLS-TP if they want to.  Either that or MPLS-TP will only
be capable of handlubg the low speed traffic in the core and the vast
majority of the traffic will continue to use plain MPLS.

Curtis

---

From: Maarten Vissers <[email-masked]>
Date: Mon, 22 Mar 2010 21:18:26 +0100

Curtis,

If there is a guarantee that Y.1731 OAM within EVC signals is not corrupted
by a LAG within the MPLS-TP network, then it would be good to state that in
the MPLS-TP document describing LAG as a component in a MPLS-TP network.
That will take away one concern I have with LAG deployment. I understadn
that a LAG in MPLS-TP network will thus never look behind a PW label for its
hashing.

Having cleared this concern, I would like to make you aware of another
related concern, now being part of MPLS-TP OAM definition.

I was just reading section 5.5 of the MPLS-TP OAM framework document and
found the following text:

----------------------
5.5. Packet Loss Measurement  

   Packet Loss Measurement (LM) is one of the capabilities supported by 
   the MPLS-TP Performance Monitoring (PM) function in order to 
   facilitate reporting of QoS information for a transport path as 
   required in section 2.2.11 of [12]. LM is used to exchange counter 
   values for the number of ingress and egress packets transmitted and 
   received by the transport path monitored by a pair of MEPs.   

   Proactive LM is performed by periodically sending LM OAM packets from 
   a MEP to a peer MEP and by receiving LM OAM packets from the peer MEP 
   (if a bidirectional transport path) during the life time of the 
   transport path. Each MEP performs measurements of its transmitted and 
   received packets. These measurements are then transactionally 
   correlated with the peer MEP in the ME to derive the impact of packet 
   loss on a number of performance metrics for the ME in the MEG. The LM 
   transactions are issued such that the OAM packets will experience the 
   same queuing discipline as the measured traffic while transiting 
   between the MEPs in the ME. 

   For a MEP, near-end packet loss refers to packet loss associated with 
   incoming data packets (from the far-end MEP) while far-end packet 
   loss refers to packet loss associated with egress data packets 
   (towards the far-end MEP).  

5.5.1. Configuration considerations 

   In order to support proactive LM, the transmission rate and PHB 
   associated with the LM OAM packets originating from a MEP need be 
   configured as part of the LM provisioning procedures. LM OAM packets 
   should be transmitted with the same PHB class that the LM is intended 
   to measure. If that PHB is not an ordered aggregate where the 
   ordering constraint is all packets with the PHB being delivered in 
   order, LM can produce inconsistent results. 
-------------------

The last sentence in section 5.5.1 refers to a required ordering. 

When a LAG is deployed within a Section layer transport path (i.e. Section
MEP functions are outside LAG), then the LM procedure will most likely
produce inconsistent results. Consistent results would be obtained only if
the Section MEP functions are located inside the LAG, i.e. on each link in
the LAG. 

When a LAG is deployed within a Transport Path layer transport path (e.g.
edge-to-edge LSP), then the edge-to-edge LSP LM procedure may produce
inconsistent results if the edge-to-edge LSP packets of the PHB and the
associated LM OAM packets are transported over more then one link in the
LAG. Can a LAG be operated such that all packets within an edge-to-edge LSP
with the same PHB will be carried over a single link in the LAG?

REgards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: zondag 21 maart 2010 5:03
To: Maarten Vissers
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

Maarten Vissers writes:
>  
> Curtis,
>  
> It will not be amusing for an operator when he deploys LAG as you 
> describe in which

Y.1731 is Ethernet OAM and would be inside a PW.  No MAC to MAC flow will be
reordered as long as the PW T-PE doesn't do something that would cause
reordering.

The single Ethernet PW is not a challenge.  Todays LAG of a few hundred
Gb/s, likely to be a Tb/s by the time MPLS-TP is deployed, is where LAG is
needed.

Of course, if MPLS-TP is only applicable to low bandwidth used, then there
is no issue.

> The reason this will not be amusing for the operator is that his EVC 
> support resutls in reporting of Severely Errored Seconds and possibly 
> Unavailable time by his UNI-N MEP functions and by the customer's 
> UNI-C MEP functions, thus causing non-compliance with the SLA for this EVC
service.
> Maarten

You seen to be assuming that Y.1731 is MPLS-TP OAM.  That is not expected to
be the case.  The requirements for CC/CV, etc, will be met, but not with
Y.1731.  When IETF rejected Y.1711 as an MPLS OAM and adopted MPLS Ping, it
was with good reason.  We should not break MPLS Ping in adapting to a
MPLS-TP profile.

Our goal should be to let MPLS-TP support link bundle PW carrying Ethernet
with no reordering at all, let MPLS-TP support PW over LAS distributing
traffic on a MAC pair (or other) basis over LAG using flow label (fat-pw),
and let the large LSP over LAG the that exist today use MPLS-TP if they want
to.  Either that or MPLS-TP will only be capable of handlubg the low speed
traffic in the core and the vast majority of the traffic will continue to
use plain MPLS.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 22 Mar 2010 17:59:21 -0400

In message <[email-masked]>
> that a LAG in MPLS-TP network will thus never look behind a PW label for its

That is very close to correct.  The hashing will never look beyond the
BOS if the payload is not identified as IPv4 or IPv6.  That is what
the PW code word is for.

OTOH, there could be a label beyond the PW label known as the flow
label and described in the fat-pw draft.

> found the following text:

Given the choice of meeting this requirement and keeping the existing
LAG functionality, some providers, maybe most of them operating a core
network, may choose to keep the LAG functionality.

Mandating that they lose this functionality may only hinder acceptance
of MPLS-TP or result in verndors, upon insistance from their
customers, provide an option to disregard this part of the spec.  If
so, the spec as written becomes a hinderance to new entrants in the
market who are unaware of the practical need to do this.

Curtis

---

From: Maarten Vissers <[email-masked]>
Date: Tue, 23 Mar 2010 07:48:00 +0100

Curtis,

Can you address this 'MPLS-TP LM OAM usage over a LAG' issue with the
authors of the MPLS-TP OAM Framework and MPLS Transport Profile Data Plane
Architecture drafts. I think that the data plane draft should include a LAG
section.

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: maandag 22 maart 2010 22:59
To: Maarten Vissers
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Comments on draft-fbb-mpls-tp-data-plane-00: ~ECMP
=>~ETH aggregation?

Maarten Vissers writes:
>  
> Curtis,
>  
> If there is a guarantee that Y.1731 OAM within EVC signals is not 
> corrupted by a LAG within the MPLS-TP network, then it would be good 
> to state that in the MPLS-TP document describing LAG as a component in a
MPLS-TP network.
> That will take away one concern I have with LAG deployment. I 
> understadn that a LAG in MPLS-TP network will thus never look behind a 
> PW label for its hashing.

That is very close to correct.  The hashing will never look beyond the BOS
if the payload is not identified as IPv4 or IPv6.  That is what the PW code
word is for.

OTOH, there could be a label beyond the PW label known as the flow label and
described in the fat-pw draft.

> I was just reading section 5.5 of the MPLS-TP OAM framework document 
> and found the following text:
>  
> ----------------------
> When a LAG is deployed within a Section layer transport path (i.e. 
> Section MEP functions are outside LAG), then the LM procedure will 
> most likely produce inconsistent results. Consistent results would be 
> obtained only if the Section MEP functions are located inside the LAG, 
> i.e. on each link in the LAG.
>  
> inconsistent results if the edge-to-edge LSP packets of the PHB and 
> the associated LM OAM packets are transported over more then one link 
> in the LAG. Can a LAG be operated such that all packets within an 
> edge-to-edge LSP with the same PHB will be carried over a single link in
the LAG?
> Maarten

Given the choice of meeting this requirement and keeping the existing LAG
functionality, some providers, maybe most of them operating a core network,
may choose to keep the LAG functionality.

Mandating that they lose this functionality may only hinder acceptance of
MPLS-TP or result in verndors, upon insistance from their customers, provide
an option to disregard this part of the spec.  If so, the spec as written
becomes a hinderance to new entrants in the market who are unaware of the
practical need to do this.

Curtis

> MPLS Ping is the current MPLS OAM and as long as we don't break what 
> works today in "enhancing" it to be MPLS-TP, then it will continue to work
fine.
> > support resutls in reporting of Severely Errored Seconds and 
> > possibly Unavailable time by his UNI-N MEP functions and by the 
> > customer's UNI-C MEP functions, thus causing non-compliance with the 
> > SLA for this EVC
> service.
> >  
> > Regards,
> > Maarten
>

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 24 Mar 2010 13:44:03 -0400

In message <[email-masked]>

Maarten,

This is a question of whether some things are SHOULD or MUST in the
eye of the service provider deploying equipment.  Ideally that would
match the documents, but reality has often deviated from standards if
SDO are too rigid.  In the view of the TP framework document what we
have been discussing could be viewed as the second case in the usage
(not looking at the document at the moment, but from memory) where
MPLS-TP coexists with MPLS.  If the only different between the
"conformant" MPLS-TP and MPLS is the former has LM and the latter can
support huge LSP using CL or LAG, the latter is likely to carry the
vast majority of traffic and the former may not even exist.

If on the other hand huge LSP using CL or LAG is recognized as a
requirement by MPLS-TP, then the only difference is that such a
deployment could be called an MPLS-TP compliant deployment rather than
be called an MPLS deployment with most capabilities of MPLS-TP
enabled.  It matters very little to the outcome, but its always nice
when framework documents more accurately reflect the outcome.

So yes, I will take your advice and send specific comments to the
framework authors to encourage then to consider this.  Minor mention
in the TP framework wouldn't hurt if it would not interfere with
advancement.  The data plane framework should have a mention of this.
The OAM framework should also have a mention if some types of OAM are
affected, which LM is.

On a relatd topic, there is also a question of accuracty of LM on
existing hardware which will slow path OAM but fast path forwarding.
I was planning to send a paragraph to be considered for inclusion in
the OAM framework.

Curtis

ps - I tried to respond yesterday but gave up trying to use the
wireless at IETF.