Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

From: <[email-masked]>
Date: Tue, 3 Jun 2014 12:17:04 +0000

Hi,

IMHO, before defining the capability, I would be more interrested on seeing some progress on a solution for EL over SPRING. A draft exists with multiple options, maybe we should wait for one option to have consensus before worrying about ELC encoding (quite easy part of the EL for SPRING).

Stephane

-----Message d'origine-----
De : Isis-wg [mailto:[email-masked]] De la part de Xuxiaohu
Envoyé : samedi 31 mai 2014 10:05
À : [email-masked]
Cc : [email-masked]; [email-masked]
Objet : [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

Hi WG co-chairs,

This draft (http://tools.ietf.org/html/draft-xu-isis-mpls-elc-00) describes how to advertise the MPLS Entropy Label Capability (ELC) using IS-IS in SPRING networks. Since (http://tools.ietf.org/html/draft-ietf-isis-segment-routing-extensions-00) has been adopted as a WG draft, as co-authors of draft-xu-isis-mpls-elc-00, we hope you could consider the WG adoption for this draft as well.

Best regards,
Xiaohu (on behalf of all-authors)  

_______________________________________________
Isis-wg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/isis-wg

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 4 Jun 2014 02:16:15 +0000

Hi Stephane,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, June 03, 2014 8:17 PM
> To: Xuxiaohu; [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: RE: Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi,
> 
> IMHO, before defining the capability, I would be more interrested on seeing
> some progress on a solution for EL over SPRING. A draft exists with multiple
> options, maybe we should wait for one option to have consensus before
> worrying about ELC encoding (quite easy part of the EL for SPRING).

IMHO, the following rationale for the EL concept has never been changed no matter the EL mechanism is used for a single LSP or stacked LSPs: An ingress LSR cannot insert ELs for packets going into a given LSP unless the egress LSR of that LSP has indicated that it can process ELs on that LSP. SPRING is just an approach of constructing explicitly routed tunnels using stacked LSPs (see http://tools.ietf.org/html/draft-gredler-spring-mpls-06). Therefore, no matter option 1 (i.e., inserting a single EL at the bottom of the LSP stack), option 2(i.e., inserting an EL at every LSP of the LSP stack), option 3 (i.e., re-uses the EL of the terminated LSP for the next inner LSP) or option 4 (i.e., inserting ELs at readable label stack depths) is used, the ELC advertisement is necessary. In other word, the ELC advertisement mechanism is orthogonal to the EL inserting options.

Best regards,
Xiaohu

> De : Isis-wg [mailto:[email-masked]] De la part de Xuxiaohu Envoyé :
> samedi 31 mai 2014 10:05 À : [email-masked] Cc :
> [email-masked]; [email-masked] Objet : [Isis-wg] Request
> for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi WG co-chairs,
> 
> This draft (http://tools.ietf.org/html/draft-xu-isis-mpls-elc-00) describes how to
> advertise the MPLS Entropy Label Capability (ELC) using IS-IS in SPRING
> networks. Since
> (http://tools.ietf.org/html/draft-ietf-isis-segment-routing-extensions-00) has
> been adopted as a WG draft, as co-authors of draft-xu-isis-mpls-elc-00, we hope
> you could consider the WG adoption for this draft as well.
> 
> Best regards,
> _____________________________________________________________
> ____________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou
> copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le
> signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration, Orange decline toute responsabilite
> si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed, used
> or copied without authorisation.

---

From: <[email-masked]>
Date: Wed, 4 Jun 2014 06:41:45 +0000

Hi,

I completely agree this is orthogonal, but the solution for adding EL is stacked tunnels is unfortunately not as easy as for a single LSP.
I just like to be sure that we would have a consensus on a solution for inserting the ELC/EL before worrying on encoding the capability. What would happen if we cannot have a consensus on the solution for the label insertion ? It just a question of delaying a bit ... I don't think this is a big issue ... and may be you can push at your side also to have the solution progressing for the label insertion ...

Stephane

-----Message d'origine-----
De : Xuxiaohu [mailto:[email-masked]] 
Envoyé : mercredi 4 juin 2014 04:16
À : LITKOWSKI Stephane SCE/IBNF; [email-masked]
Cc : [email-masked]; [email-masked]
Objet : RE: Request for WG adoption of draft-xu-isis-mpls-elc-00

Hi Stephane,

> IMHO, before defining the capability, I would be more interrested on 
> seeing some progress on a solution for EL over SPRING. A draft exists 
> with multiple options, maybe we should wait for one option to have 
> consensus before worrying about ELC encoding (quite easy part of the EL for SPRING).

Best regards,
Xiaohu

> [email-masked]; [email-masked] Objet : 
> [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi WG co-chairs,
> 
> This draft (http://tools.ietf.org/html/draft-xu-isis-mpls-elc-00) 
> describes how to advertise the MPLS Entropy Label Capability (ELC) 
> using IS-IS in SPRING networks. Since
> (http://tools.ietf.org/html/draft-ietf-isis-segment-routing-extensions
> -00) has been adopted as a WG draft, as co-authors of 
> draft-xu-isis-mpls-elc-00, we hope you could consider the WG adoption for this draft as well.
> 
> Best regards,
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi 
> que les pieces jointes. Les messages electroniques etant susceptibles 
> d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not 
> be distributed, used or copied without authorisation.

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 4 Jun 2014 08:21:40 +0000

Hi Stephane,

> Sent: Wednesday, June 04, 2014 2:42 PM
> I completely agree this is orthogonal, but the solution for adding EL is stacked
> tunnels is unfortunately not as easy as for a single LSP.
> I just like to be sure that we would have a consensus on a solution for inserting
> the ELC/EL before worrying on encoding the capability. What would happen if
> we cannot have a consensus on the solution for the label insertion ? It just a

Since the ELC advertisement is orthogonal to the EL insertion solution, it doesn't matter whether the consensus on the best EL insertion solution can be reached or not. Furthermore, IMHO, since each option has its pros and cons, the final result may be "make your own choice according to your network capabilities and your applications". However, no matter whichever EL insertion option is enabled on the ingress LSR in practice, the ELC advertisement is a necessity. Don't you think so?

Best regards,
Xiaohu

> question of delaying a bit ... I don't think this is a big issue ... and may be you can
> push at your side also to have the solution progressing for the label insertion ...
> 
> Stephane
> 
> 
> De : Xuxiaohu [mailto:[email-masked]] Envoyé : mercredi 4 juin 2014
> 04:16 À : LITKOWSKI Stephane SCE/IBNF; [email-masked] Cc :
> [email-masked]; [email-masked] Objet : RE: Request for
> WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi Stephane,
> 
> IMHO, the following rationale for the EL concept has never been changed no
> matter the EL mechanism is used for a single LSP or stacked LSPs: An ingress LSR
> cannot insert ELs for packets going into a given LSP unless the egress LSR of that
> LSP has indicated that it can process ELs on that LSP. SPRING is just an approach
> of constructing explicitly routed tunnels using stacked LSPs (see
> http://tools.ietf.org/html/draft-gredler-spring-mpls-06). Therefore, no matter
> option 1 (i.e., inserting a single EL at the bottom of the LSP stack), option 2(i.e.,
> inserting an EL at every LSP of the LSP stack), option 3 (i.e., re-uses the EL of the
> terminated LSP for the next inner LSP) or option 4 (i.e., inserting ELs at readable
> label stack depths) is used, the ELC advertisement is necessary. In other word,
> the ELC advertisement mechanism is orthogonal to the EL inserting options.
> 
> Best regards,
> Xiaohu
> 
> > Stephane
> >
> Envoyé :

---

From: <[email-masked]>
Date: Wed, 4 Jun 2014 11:34:18 +0000

Worst scenario (that I would not push for :) ) : What would you do with your capability if the conclusion is that EL insertion would not be supported for stacked tunnels because too complex ? 

What's the problem for you to delay a bit your doc ? As it will never been implemented until there is an EL insertion solution ...

-----Message d'origine-----
Envoyé : mercredi 4 juin 2014 10:22

Hi Stephane,

> I completely agree this is orthogonal, but the solution for adding EL 
> is stacked tunnels is unfortunately not as easy as for a single LSP.
> I just like to be sure that we would have a consensus on a solution 
> for inserting the ELC/EL before worrying on encoding the capability. 
> What would happen if we cannot have a consensus on the solution for 
> the label insertion ? It just a

Best regards,
Xiaohu

> question of delaying a bit ... I don't think this is a big issue ... 
> and may be you can push at your side also to have the solution progressing for the label insertion ...
> 
> Stephane
> 
> 
> [email-masked]; [email-masked] Objet : RE: 
> Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi Stephane,
> 
> > seeing some progress on a solution for EL over SPRING. A draft 
> > exists with multiple options, maybe we should wait for one option to 
> > have consensus before worrying about ELC encoding (quite easy part 
> > of the EL for
> SPRING).
> 
> IMHO, the following rationale for the EL concept has never been 
> changed no matter the EL mechanism is used for a single LSP or stacked 
> LSPs: An ingress LSR cannot insert ELs for packets going into a given 
> LSP unless the egress LSR of that LSP has indicated that it can 
> process ELs on that LSP. SPRING is just an approach of constructing 
> explicitly routed tunnels using stacked LSPs (see 
> http://tools.ietf.org/html/draft-gredler-spring-mpls-06). Therefore, 
> no matter option 1 (i.e., inserting a single EL at the bottom of the 
> LSP stack), option 2(i.e., inserting an EL at every LSP of the LSP 
> stack), option 3 (i.e., re-uses the EL of the terminated LSP for the 
> next inner LSP) or option 4 (i.e., inserting ELs at readable label stack depths) is used, the ELC advertisement is necessary. In other word, the ELC advertisement mechanism is orthogonal to the EL inserting options.
> 
> Best regards,
> Xiaohu
> 
> > Stephane
> >
> > draft-xu-isis-mpls-elc-00, we hope you could consider the WG 
> > adoption for
> this draft as well.
> >
> > Best regards,
> > confidentielles ou privilegiees et ne doivent donc pas etre 
> > diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> > ce message par erreur, veuillez le signaler a l'expediteur et le 
> > detruire ainsi que les pieces jointes. Les messages electroniques 
> > etant susceptibles d'alteration, Orange decline toute responsabilite 
> > If you have received this email in error, please notify the sender 
> > and delete this message and its attachments.
> > As emails may be altered, Orange is not liable for messages that 
> > have been modified, changed or falsified.
> > Thank you.
> 
>

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 5 Jun 2014 01:22:57 +0000

> Sent: Wednesday, June 04, 2014 7:34 PM
> Worst scenario (that I would not push for :) ) : What would you do with your
> capability if the conclusion is that EL insertion would not be supported for
> stacked tunnels because too complex ?

I have to disagree with your assumption. At least for option 1 (i.e., inserting a single EL at the bottom of the LSP stack), do you believe it's too complex? In addition, remember that one of the use cases of SPRING is IGP-based MPLS Tunneling (see section 2 of draft-filsfils-spring-segment-routing-use-cases-00).

> What's the problem for you to delay a bit your doc ? As it will never been
> implemented until there is an EL insertion solution ...

It's no problem for me to delay a bit as long as that delay is meaningful.

Best regard,
Xiaohu

> 10:22 À : LITKOWSKI Stephane SCE/IBNF; [email-masked] Cc :
> Since the ELC advertisement is orthogonal to the EL insertion solution, it doesn't
> matter whether the consensus on the best EL insertion solution can be reached
> or not. Furthermore, IMHO, since each option has its pros and cons, the final
> result may be "make your own choice according to your network capabilities
> and your applications". However, no matter whichever EL insertion option is
> enabled on the ingress LSR in practice, the ELC advertisement is a necessity.
> Don't you think so?
> 
> Best regards,
> Xiaohu
> 
> > next inner LSP) or option 4 (i.e., inserting ELs at readable label stack depths) is
> used, the ELC advertisement is necessary. In other word, the ELC advertisement
> mechanism is orthogonal to the EL inserting options.
> >
> > Best regards,
> > Xiaohu
> >
> > > Stephane
> > >

---

From: Xuxiaohu <[email-masked]>
Date: Thu, 5 Jun 2014 08:46:41 +0000

Hi Clarence,

> From: Clarence Filsfils [mailto:[email-masked]]
> Sent: Thursday, June 05, 2014 4:09 PM
> To: Xuxiaohu
> Cc: [email-masked]; [email-masked];
> [email-masked]; [email-masked]
> Subject: Re: Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Xiaohu,
> 
> >> -----Original Message----- From: [email-masked]
> >> [mailto:[email-masked]] Sent: Wednesday, June 04,
> >> 2014 7:34 PM To: Xuxiaohu; [email-masked] Cc:
> >> [email-masked]; [email-masked] Subject:
> >> RE: Request for WG adoption of draft-xu-isis-mpls-elc-00
> >>
> >> Worst scenario (that I would not push for :) ) : What would you do
> >> with your capability if the conclusion is that EL insertion would not
> >> be supported for stacked tunnels because too complex ?
> >
> > I have to disagree with your assumption. At least for option 1 (i.e.,
> > inserting a single EL at the bottom of the LSP stack), do you believe
> > it's too complex? In addition, remember that one of the use cases of
> > SPRING is IGP-based MPLS Tunneling (see section 2 of
> > draft-filsfils-spring-segment-routing-use-cases-00).
> 
> What is the logical relationship?

Stephane had assumed the EL insertion for stacked tunnels (i.e., SPRING in this context) would be too complex. However, in the use case illustrated in section 2, the EL insertion for SPRING is no difference from that for today's LDP or RSVP-TE LSPs. 

Best regards,
Xiaohu  

> Cheers,
> Clarence

---

From: <[email-masked]>
Date: Thu, 5 Jun 2014 09:25:33 +0000

I think some HW may be completely unable to look at EL if it's too deep in the stack (5,6 or 7 labels after top ...) as mentioned in Kireeti's draft ...
The subject is not as easy as you sounds to think ...

-----Original Message-----
From: Xuxiaohu [mailto:[email-masked]] 
Sent: Thursday, June 05, 2014 10:47
To: Clarence Filsfils
Cc: LITKOWSKI Stephane SCE/IBNF; [email-masked]; [email-masked]; [email-masked]
Subject: RE: Request for WG adoption of draft-xu-isis-mpls-elc-00

Hi Clarence,

> >> with your capability if the conclusion is that EL insertion would 
> >> not be supported for stacked tunnels because too complex ?
> >
> > I have to disagree with your assumption. At least for option 1 
> > (i.e., inserting a single EL at the bottom of the LSP stack), do you 
> > believe it's too complex? In addition, remember that one of the use 
> > cases of SPRING is IGP-based MPLS Tunneling (see section 2 of 

Best regards,
Xiaohu

---

From: Hannes Gredler <[email-masked]>
Date: Thu, 5 Jun 2014 11:46:39 +0200

<IS-IS WG chair hat off>

i'm discomfortable with the *granularity* of the cap-advertisement:

rather than saying:

"i can crawl as deep as you like on any of my interfaces"

  i'd like to announce:

"i can crawl for EL <N> up to labels deep on interface XYZ"

/hannes

---

From: Clarence Filsfils <[email-masked]>
Date: Thu, 05 Jun 2014 10:08:40 +0200

Xiaohu,

Cheers,
Clarence

---

From: Clarence Filsfils <[email-masked]>
Date: Thu, 05 Jun 2014 13:54:18 +0200

Xiaohu,

> Stephane had assumed the EL insertion for stacked tunnels (i.e.,
> SPRING in this context) would be too complex. However, in the use
> case illustrated in section 2, the EL insertion for SPRING is no
> difference from that for today's LDP or RSVP-TE LSPs.

In that example, it is especially optional. If the transported packet is 
IP, then the hash can be done on the 7-tuple field easily accessible 
behind 1 or 2 labels. If the transported traffic is on a "fat PW", then 
the "entropy" label can be in the PW context.

The point Stephane makes is that the use-cases for EL should be better 
documented first.

Cheers,
Clarence

> Best regards, Xiaohu
>> Cheers, Clarence
> .

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 5 Jun 2014 14:25:12 +0200

Hi Xu,

Actually let me express an opinion that while for other MPLS
signalling protocols the concept of entropy label as describe in RFC
6790 may be useful I would rather question if the same applies to
segment routing architecture.

Fundamental difference here is that other MPLS signalling protocols
bind labels to FECs. SR on the other hand binds SIDs (which one of the
special case of can be label) to nodes or links. That means that
number of such bindings will naturally be orders of magnitude less in
the SR concept.

That means that perhaps one could really analyze if the tuple ELI + EL
(one or many) is that much needed as opposed to advertise wider range
of SIDs and simply use those in flat layer for loadbalancing reasons ?
Just mapping different flows on ingress to different label from said
range.

Also that not only can accomplish egress node loadbalancing, but also
all via nodes will have no problem with such input to a hash function
regardless if they are SR capable or not ?

Maybe rather then copying all fixes of MPLS original architecture to
SR its better to adjust SR architecture to at least not repeat the
same mistakes we have already made in the past ?

Comments ?

Cheers,
R.

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Thu, 5 Jun 2014 13:15:43 +0000

Hi Robert, Xu,

> That means that perhaps one could really analyze if the tuple ELI + EL (one or
> many) is that much needed as opposed to advertise wider range of SIDs and
> simply use those in flat layer for loadbalancing reasons ?
> Just mapping different flows on ingress to different label from said range.

That's an interesting idea, but I'd like to highlight one down side. "wider range of SIDs" will likely have to be fairly large/wide to have good probability of good ECMP distribution throughout the network, and that will likely result in a significant increase in forwarding table in every network nodes. Obvious side effects include increased SRGB, increased memory usage, debugging difficulty, increased number in in-band-OAM to test all forwarding entries.

Considering above, perhaps fitting in EL to MPLS based SR is still a reasonable option to discuss. It's not a trivial topic, but I think this is an important topic that has lacked attention.

Xu, I think draft-xu-isis-mpls-elc should have a reference to draft-kini-mpls-spring-entropy-label. And I agree with Stephane, there's minimal benefit in progressing draft-xu-isis-mpls-elc before draft-kini-mpls-spring-entropy-label.

Thanks.

-Nobo

> From: Isis-wg [mailto:[email-masked]] On Behalf Of Robert Raszuk
> Sent: Thursday, June 05, 2014 8:25 AM
> To: Xuxiaohu
> Cc: [email-masked]; [email-masked]; isis-
> [email-masked]
> Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi Xu,
> 
> Actually let me express an opinion that while for other MPLS signalling
> protocols the concept of entropy label as describe in RFC
> 6790 may be useful I would rather question if the same applies to segment
> routing architecture.
> 
> Fundamental difference here is that other MPLS signalling protocols bind
> labels to FECs. SR on the other hand binds SIDs (which one of the special
> case of can be label) to nodes or links. That means that number of such
> bindings will naturally be orders of magnitude less in the SR concept.
> 
> That means that perhaps one could really analyze if the tuple ELI + EL (one or
> many) is that much needed as opposed to advertise wider range of SIDs and
> simply use those in flat layer for loadbalancing reasons ?
> Just mapping different flows on ingress to different label from said range.
> 
> Also that not only can accomplish egress node loadbalancing, but also all via
> nodes will have no problem with such input to a hash function regardless if
> they are SR capable or not ?
> 
> Maybe rather then copying all fixes of MPLS original architecture to SR its
> better to adjust SR architecture to at least not repeat the same mistakes we
> have already made in the past ?
> 
> Comments ?
> 
> Cheers,
> R.
> 
> 
> 
> > This draft (http://tools.ietf.org/html/draft-xu-isis-mpls-elc-00) describes
> how to advertise the MPLS Entropy Label Capability (ELC) using IS-IS in
> SPRING networks. Since (http://tools.ietf.org/html/draft-ietf-isis-segment-
> routing-extensions-00) has been adopted as a WG draft, as co-authors of

---

From: <[email-masked]>
Date: Thu, 5 Jun 2014 13:25:08 +0000

Robert,

I would tend to disagree with you.
With your model, loadbalancing becomes "stateful" so you need to create new forwarding states on your nodes to achieve the granularity you want and you will never be able to the same level of granularity that you have today with hop by hop hashing approach. I agree that hop by hop approach may also have some drawbacks like polarization.
IMHO, loadbalancing must be kept "stateless" as much as possible.

But nothing says that one model must kill the other (ingress lb vs hop by hop lb) and both may be used in the same network for different reason.

Stephane

-----Original Message-----
From: Isis-wg [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, June 05, 2014 14:25
To: Xuxiaohu
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

Hi Xu,

Actually let me express an opinion that while for other MPLS signalling protocols the concept of entropy label as describe in RFC
6790 may be useful I would rather question if the same applies to segment routing architecture.

Fundamental difference here is that other MPLS signalling protocols bind labels to FECs. SR on the other hand binds SIDs (which one of the special case of can be label) to nodes or links. That means that number of such bindings will naturally be orders of magnitude less in the SR concept.

That means that perhaps one could really analyze if the tuple ELI + EL (one or many) is that much needed as opposed to advertise wider range of SIDs and simply use those in flat layer for loadbalancing reasons ?
Just mapping different flows on ingress to different label from said range.

Also that not only can accomplish egress node loadbalancing, but also all via nodes will have no problem with such input to a hash function regardless if they are SR capable or not ?

Maybe rather then copying all fixes of MPLS original architecture to SR its better to adjust SR architecture to at least not repeat the same mistakes we have already made in the past ?

Comments ?

Cheers,
R.

---

From: <[email-masked]>
Date: Thu, 5 Jun 2014 13:34:04 +0000

Hi Hannes,

I agree with you but if draft-kini-mpls-spring-entropy-label evolve to the option where entropy label is always behind top label, there is no need to encode such granularity.
That's why I think it's too early to progress the capability document.

Thoughts ?

Stephane

-----Original Message-----
From: Hannes Gredler [mailto:[email-masked]] 
Sent: Thursday, June 05, 2014 11:47
To: Xuxiaohu
Cc: [email-masked]; [email-masked]; [email-masked] list; LITKOWSKI Stephane SCE/IBNF; <[email-masked]>
Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

<IS-IS WG chair hat off>

rather than saying:

  i'd like to announce:

/hannes

On Jun 3, 2014, at 2:17 PM, <[email-masked]> <[email-masked]> wrote:

> Envoyé : samedi 31 mai 2014 10:05 À : [email-masked] Cc : 
> ______________________________________________________________________
> ___________________________________________________
> 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 5 Jun 2014 15:43:05 +0200

Hi Nobo,

Well the range size would be as big as most ECMP paths you have. Say
32 or 64 as example.

As to the forwarding table size in the core it is rather
implementation dependent .. you really do not need to do exact match
for all labels in a range (unlike with atomic 20 bit wide MPLS label)
it is still much less then you have entries on the ingress and egress
.. as those are in millions.

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 5 Jun 2014 15:45:32 +0200

I really do not see anything stateful in the network with the model described.

Cheers,
R.

---

From: <[email-masked]>
Date: Thu, 5 Jun 2014 14:02:35 +0000

As far as I understand your proposal, you need to create forwarding entries "per flow" in all nodes, that's creating more states ...
Moreover, SIDs are bound to links and prefixes (not really node), so for Prefix SID, it's quite similar to a FEC , except that you can bind multiple SIDs to the same prefix. But each Prefix SID you are adding, adds a new forwarding state in the whole IGP domain.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, June 05, 2014 15:46
To: LITKOWSKI Stephane SCE/IBNF
Cc: Xuxiaohu; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 5 Jun 2014 16:07:39 +0200

> Moreover, SIDs are bound to links and prefixes (not really node), so for Prefix SID, it's quite similar to a FEC , except that you can bind multiple SIDs to the same prefix. But each Prefix SID you…

No not at all :)

SIDs can be aggregated in forwarding just fine. Think v6 address for
simplicity .. or MPLS labels if your data plane supports such
aggregation ..

Then what is in the packet's header  is only input for today's hash function.

If there is any logic (and that does not need to be stateful) it is
only on ingress to impose packets with the proper IP/MPLS header.
That's it.

Last sure you have prefix SIDs but those prefixes just describe node
say loopbacks as example. Frankly I do not understand why this forced
indirection is really needed - but this is different topic.

Best,
R.

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Thu, 5 Jun 2014 14:42:17 +0000

Hi Robert,

Please see in-line.

> Sent: Thursday, June 05, 2014 9:43 AM
> To: Nobo Akiya (nobo)

Number of ECMP paths doesn't necessary equal to number of hash buckets in implementations. In addition, SID advertising node would need to be aware of upper bound ECMP paths (and hash buckets) from all [potential] ingresses to itself in order to advertise the right number of SIDs ... or always advertise _max_ number of SIDs for ECMP.

> As to the forwarding table size in the core it is rather implementation
> dependent .. you really do not need to do exact match for all labels in a
> range (unlike with atomic 20 bit wide MPLS label) it is still much less then
> you have entries on the ingress and egress .. as those are in millions.

Still, in an SR network consisting of 100 nodes ... each network node used to have 100 node SID forwarding entries (assuming one SID advertised per node) but now each network node will have 3200 or 6400 or 12800 node SID forwarding entries. That's a huge increase. Sure the forwarding table can do some aggregation but number of "resources" reserved, consumed, advertised and OAM requires to support them are still enlarged significantly.

I still see tackling EL to be appealing :)

Thanks!

-Nobo

> >> many) is that much needed as opposed to advertise wider range of SIDs
> >> and simply use those in flat layer for loadbalancing reasons ?
> > That's an interesting idea, but I'd like to highlight one down side. "wider
> range of SIDs" will likely have to be fairly large/wide to have good
> probability of good ECMP distribution throughout the network, and that will
> likely result in a significant increase in forwarding table in every network
> nodes. Obvious side effects include increased SRGB, increased memory
> usage, debugging difficulty, increased number in in-band-OAM to test all
> forwarding entries.
> >
> > Considering above, perhaps fitting in EL to MPLS based SR is still a
> reasonable option to discuss. It's not a trivial topic, but I think this is an
> important topic that has lacked attention.
> >
> > Xu, I think draft-xu-isis-mpls-elc should have a reference to draft-kini-
> mpls-spring-entropy-label. And I agree with Stephane, there's minimal
> benefit in progressing draft-xu-isis-mpls-elc before draft-kini-mpls-spring-
> entropy-label.
> >
> > Thanks.
> >
> > -Nobo
> >
> >> Cc: [email-masked];
> >> [email-masked]; isis- [email-masked]
> >> bind labels to FECs. SR on the other hand binds SIDs (which one of
> >> the special case of can be label) to nodes or links. That means that
> >> number of such bindings will naturally be orders of magnitude less in the
> SR concept.
> >>
> >> many) is that much needed as opposed to advertise wider range of SIDs
> >> and simply use those in flat layer for loadbalancing reasons ?
> >> > describes
> >> how to advertise the MPLS Entropy Label Capability (ELC) using IS-IS
> >> in SPRING networks. Since
> >> (http://tools.ietf.org/html/draft-ietf-isis-segment-
> >> routing-extensions-00) has been adopted as a WG draft, as co-authors
> >> of draft-xu-isis-mpls-elc-00, we hope you could consider the WG
> >> adoption for this draft as well.
> >> >
> >> > Best regards,

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 5 Jun 2014 16:49:39 +0200

> always advertise _max_ number of SIDs for ECMP.

I meant that. No point to go lower at all.

> Still, in an SR network consisting of 100 nodes ... each network node used to have 100 node SID forwarding entries (assuming one SID advertised per node) but now each network node will have 3200 or 6400 or 12800 node SID forwarding entries. That's a huge increase.

Assume no aggregation it looks huge, but it's tiny if you compare it
with state required on ingress as example where all of your overlay
services reside.

> Sure the forwarding table can do some aggregation but number of "resources" reserved, consumed, advertised and OAM requires to support them are still enlarged significantly.

I do not agree. Assume v6 SR where I advertise instead of single SID
say a block of /96. It is still one prefix for all what forwarding
needs to care about.  What would be different for that prefix vs /128
prefix from the point of view of  "resources" reserved, consumed,
advertised and OAM ?

> I still see tackling EL to be appealing :)

Is this still more appealing then say mpls over UDP draft ;-) ?
http://tools.ietf.org/html/draft-ietf-mpls-in-udp-05

Cheers,
R.

---

From: "Bragg, Nigel" <[email-masked]>
Date: Thu, 5 Jun 2014 11:42:39 -0400

I agree with you, Robert, for fundamental reasons.   When you stand back there are two ways of mitigating network congestion :
-	spread the offered load over the widest range of routes available,  or
-	place load onto routes specifically selected to minimize the chance of conflict for resources.
ECMP mechanizes one of these, SR the other;  they are bookends.

So, launch with entropy immediately under the top (i.e.  egress node) label -- goes ECMP route;
Anything else is parsed as an SR label stack ?
-	far too simple  :-), but might be a start.
Nigel

-----Original Message-----
From: Isis-wg [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: 05 June 2014 15:50
To: Nobo Akiya (nobo)
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

> Still, in an SR network consisting of 100 nodes ... each network node used to have 100 node SID forwarding entries (assuming one SID advertised per node) but now each network node will have 3200 or…

Assume no aggregation it looks huge, but it's tiny if you compare it with state required on ingress as example where all of your overlay services reside.

> Sure the forwarding table can do some aggregation but number of "resources" reserved, consumed, advertised and OAM requires to support them are still enlarged significantly.

I do not agree. Assume v6 SR where I advertise instead of single SID say a block of /96. It is still one prefix for all what forwarding needs to care about.  What would be different for that prefix vs /128 prefix from the point of view of  "resources" reserved, consumed, advertised and OAM ?

Cheers,
R.

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Thu, 5 Jun 2014 22:44:40 +0000

Hi Robert,

> I do not agree. Assume v6 SR where I advertise instead of single SID say a
> block of /96. It is still one prefix for all what forwarding needs to care about.
> What would be different for that prefix vs /128 prefix from the point of view
> of  "resources" reserved, consumed, advertised and OAM ?

In MPLS, there's no standards in load balancing. Meaning an implementation may load balance only on the top label, only on the bottom label or set of labels which may or may not be the entire label stack. To take this into account, every SID advertised will need to be advertised as a "range", even service SIDs. And what you are suggesting is that the "range" needs to be 7 or 8 bits out of 20 bits. Yes even SID advertisements can be optimized to take the "range" into account, but still a very big link/service label space & SRGB is required for this.

In addition, AFAIK, many existing implementations need specific label entry in the forwarding table, i.e. masked label entry isn't supported. So in these implementations, every advertised SIDs will exist. That will mean it is very beneficial for OAM to test every one of these SIDs. For example, if we were to use BFD to do continuity check, then we now need exploding number of BFD sessions to test every SID.

I think what you are proposing is interesting, but I do see potential some consequences that should be discussed/evaluated.

> http://tools.ietf.org/html/draft-ietf-mpls-in-udp-05

I prefer not to get into this topic for now :)

Thanks!

-Nobo

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 6 Jun 2014 00:56:56 +0200

Hi Nobo,

Inline ...

> In MPLS, there's no standards in load balancing. Meaning an
> implementation may load balance only on the top label, only on the
> bottom label or set of labels which may or may not be the entire
> label stack.

Practically I do not know any implementation which would ignore top
label as an input for hash, but you are correct that there is no
standard and that such implementation is possible in theory.

> To take this into account, every SID advertised will need to be advertised
> as a "range", even service SIDs.

Nope. I do not think this needs to be extended to service SIDs at all.
After all services require transport and transport SIDs is all we care
about for IP/MPLS loadbalancing.

> And what you are suggesting is that the "range" needs to be 7 or 8
> bits out of 20 bits.

5 bits gives option for 32 ECMP paths, 6 - 64, 7-128 that should do it :)

> Yes even SID advertisements can be optimized to take the "range" into
> account, but still a very big link/service label space & SRGB is required
> for this.

I think for v6 forwarding this is clearly beneficial. Hence such
optimization in SR architecture is welcome.

If MPLS is going to benefit from it remains to be seen or agreed up
on. I was just trying to share quick thought which just came up when
reading this thread.

> I prefer not to get into this topic for now :)

:-)

Cheers,
R.

---

From: "Nobo Akiya (nobo)" <[email-masked]>
Date: Fri, 6 Jun 2014 00:12:06 +0000

Hi Robert,

Please see in-line.

> > bottom label or set of labels which may or may not be the entire label
> > stack.
> 
> Practically I do not know any implementation which would ignore top label
> as an input for hash, but you are correct that there is no standard and that
> such implementation is possible in theory.
> 
> > To take this into account, every SID advertised will need to be
> > advertised as a "range", even service SIDs.
> 
> After all services require transport and transport SIDs is all we care about for
> IP/MPLS loadbalancing.

If (and only if) there is an implementation that load balances only on the bottom label, then service SID is what gets used for load balancing.

> 5 bits gives option for 32 ECMP paths, 6 - 64, 7-128 that should do it :)

That (7 bits) will realistically be sufficient, although this will be break if someone has >128 hash buckets.

> > account, but still a very big link/service label space & SRGB is
> > required for this.
> 
> I think for v6 forwarding this is clearly beneficial. Hence such optimization in
> SR architecture is welcome.

For v6, this sounds good.

> 
> If MPLS is going to benefit from it remains to be seen or agreed up on. I was
> just trying to share quick thought which just came up when reading this
> thread.

Yes and it's great that this topic is finally getting much-needed discussions ... although we need a lot more :)

One other thing that I'd like to point out is that EL is the first thing that has come close to standardizing the load balancing on MPLS data plane, and it is setting up the "infrastructure" for improved OAM techniques (ex: CC per ECMP path). Whether we go with EL or something else on MPLS based SR, I think it'll be very beneficial to really think about the OAM angle.

Thanks for interesting discussion!

-Nobo

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 6 Jun 2014 02:28:56 +0000

Hi Hannes,

It seems that you are not talking about the particular ELC concept as defined in RFC6790 (i..e, the capability of recognizing the ELI and popping that ELI and the following EL). Instead, it seems that you are talking about something else related to the capability of accessing the maximum label stack depth.

As for the granularity of the ELC advertisement (i.e., per-node basis or per-interface-board basis), it has been discussed before. Although the ELC is interface-board related, since it's almost impossible to determine at which interface a packet destined for one of the router's addresses would arrive (e.g., a MPLS packet with the top label being a node segment ID of a given LSR), the feasible way of advertising the ELC is: the egress LSR SHOULD not advertise its ELC unless all of its interface-boards have the ELC. By the way, this is exactly the same way that can be supported by RFC6790, IMHO.

Best regards,
Xiaohu 

> From: Isis-wg [mailto:[email-masked]] On Behalf Of Hannes Gredler
> Sent: Thursday, June 05, 2014 5:47 PM
> To: Xuxiaohu
> Cc: [email-masked]; <[email-masked]>;
> [email-masked]; [email-masked] list
> > This draft (http://tools.ietf.org/html/draft-xu-isis-mpls-elc-00) describes how
> to advertise the MPLS Entropy Label Capability (ELC) using IS-IS in SPRING
> networks. Since
> _________
> > par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces
> jointes. Les messages electroniques etant susceptibles d'alteration, Orange
> decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> >
> > privileged information that may be protected by law; they should not be
> distributed, used or copied without authorisation.

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 6 Jun 2014 02:37:05 +0000

> Sent: Thursday, June 05, 2014 9:34 PM
> To: Hannes Gredler; Xuxiaohu
> Cc: [email-masked]; [email-masked];
> [email-masked] list; <[email-masked]>
> Subject: RE: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00
> 
> Hi Hannes,
> 
> I agree with you but if draft-kini-mpls-spring-entropy-label evolve to the option
> where entropy label is always behind top label, there is no need to encode such
> granularity.

Whether or not delay the adoption of the ELC advertisement doc, I have to point out the fact that the ELC advertisement DOESN'T impact the EL insertion solution, and vice versa. If you disagree, please give a concrete example.

Best regards,
Xiaohu

> Cc: [email-masked]; [email-masked];
> [email-masked] list; LITKOWSKI Stephane SCE/IBNF; <[email-masked]>

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 6 Jun 2014 03:24:15 +0000

Hi Robert,

> Sent: Thursday, June 05, 2014 8:25 PM
> To: Xuxiaohu
> [email-masked]
> Actually let me express an opinion that while for other MPLS signalling protocols
> the concept of entropy label as describe in RFC

I'm a bit confused with the above argument. The EL usage (i.e., the data-plane processing of ELs) is completely independent from the signaling protocols for ELs. In other words, assume operators choose to manually configure the ELC of egress LSRs on a given ingress LSR, the EL trick can be used without any problem. The signaling protocol just provides a way to automatically discovery the ELC of egress LSRs. This situation has no difference from RFC5512. Without RFC5512, do you believe that the MPLS-in-GRE or MPLS-in-IP tunnels couldn't be used between PE pairs? Since he MPLS-based SPRING or SR architecture has no change to the MPLS data-plane, why can't we apply the EL concept directly to SPRING or SR architecture? 

> Fundamental difference here is that other MPLS signalling protocols bind labels
> to FECs. SR on the other hand binds SIDs (which one of the special case of can be
> label) to nodes or links. That means that number of such bindings will naturally
> be orders of magnitude less in the SR concept.

In this thread, we are just talking about the MPLS-based SRPING or SR case. In that case, I don't believe the above argument (i.e., that number of such bindings will naturally
be orders of magnitude less in the SR concept) is true. Compare LDP with the IGP-based label distribution protocol, why do you believe the latter means orders of magnitude less of bindings?

> Just mapping different flows on ingress to different label from said range.

It seems that the above approach you mentioned has been denied by RFC6790 (See section 2 of RFC6790).

> nodes will have no problem with such input to a hash function regardless if they
> are SR capable or not ?
> 
> Maybe rather then copying all fixes of MPLS original architecture to SR its better
> to adjust SR architecture to at least not repeat the same mistakes we have
> already made in the past ?

What' the same mistakes in your mind? The entropy label concept as defined in RFC6790? Or allocate a single label rather than a label range for a given FEC?

Best regards,
Xiaohu

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 6 Jun 2014 03:52:37 +0000

Stephane ,

Oh, I finally realized the reason why we have different opinions on whether the ELC advertisement is orthogonal to the EL insertion solution. That's because we have different understanding of the concept of the "ELC". My understanding of the "ELC" is in accordance with the particular ELC concept as defined in RFC6790 (i.e., the capability of recognizing the ELI and popping ELI and EL, in addition, this is the capability of the EGRESS LSR). Your understanding of the "ELC" looks like a mix of the particular ELC concept as defined in RFC6790 and something else (e.g., the capability of accessing the max label stack deepth)

Best regards,
Xiaohu

> From: Xuxiaohu
> Sent: Friday, June 06, 2014 10:37 AM
> To: [email-masked]'; Hannes Gredler
> > I agree with you but if draft-kini-mpls-spring-entropy-label evolve to
> > the option where entropy label is always behind top label, there is no
> > need to encode such granularity.
> Whether or not delay the adoption of the ELC advertisement doc, I have to point
> out the fact that the ELC advertisement DOESN'T impact the EL insertion solution,
> and vice versa. If you disagree, please give a concrete example.
> 
> Best regards,
> Xiaohu
> 
> > Thoughts ?
> >
> > Stephane
> >
> > > seeing
> > some progress on a solution for EL over SPRING. A draft exists with
> > multiple options, maybe we should wait for one option to have
> > > De : Isis-wg [mailto:[email-masked]] De la part de
> > > Xuxiaohu Envoyé : samedi 31 mai 2014 10:05 À : [email-masked]
> Cc :
> > > describes how
> > to advertise the MPLS Entropy Label Capability (ELC) using IS-IS in
> > SPRING networks. Since
> > > detruire ainsi que les pieces
> > jointes. Les messages electroniques etant susceptibles d'alteration,
> > > be

---

From: Hannes Gredler <[email-masked]>
Date: Fri, 6 Jun 2014 09:14:31 +0200

valid point - i am not terribly happy with the EL behind the top label as it may require
new data plane capabilities (you need to retain the EL label while the
SR stack gets POPed off). - not sure if all existing MPLS HW does
support that.

/hannes

---

From: <[email-masked]>
Date: Fri, 6 Jun 2014 07:38:04 +0000

You are right, I think EL processing in SPRING is mainly an implementor discussion about what is possible to do, what's the impact on legacy HWs ...

Some HW may also never be able to look at EL if it's 7 or 8 labels after the top one ... and FAT PW solution (FAT label at bottom of stack) may not work anymore. 

-----Original Message-----
From: Hannes Gredler [mailto:[email-masked]] 
Sent: Friday, June 06, 2014 09:15
To: LITKOWSKI Stephane SCE/IBNF
Cc: Xuxiaohu; [email-masked]; [email-masked]; [email-masked] list; <[email-masked]>
Subject: Re: [Isis-wg] Request for WG adoption of draft-xu-isis-mpls-elc-00

valid point - i am not terribly happy with the EL behind the top label as it may require new data plane capabilities (you need to retain the EL label while the SR stack gets POPed off). - not sure if all existing MPLS HW does support that.

/hannes 

On Jun 5, 2014, at 3:34 PM, <[email-masked]> wrote:

>> _____________________________________________________________________
>> _ ___________________________________________________
>>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 6 Jun 2014 09:44:54 +0200

Xu,

> Since he MPLS-based SPRING or SR architecture has no change to the
> MPLS data-plane, why can't we apply the EL concept directly to SPRING
> or SR architecture?

You sure can. That was not my point. My point was that we should
analyze and also consider alternatives other then EL approach.

> be orders of magnitude less in the SR concept) is true. Compare LDP with
> the IGP-based label distribution protocol, why do you believe the latter
> means orders of magnitude less of bindings?

Because as it has been already proven in other threads number of SR
nodes in a given domain may be much less then number of IGP nodes. (I
remove the adj SIDs requirement for now). In the latter and in the LDP
based FECs you rather must support signalling of ranges of FECs end to
end for it to work. With SR you can just do it for 10 nodes in 500
node IGP domain to get pretty good results. That's key difference.

> What' the same mistakes in your mind? The entropy label concept as defined in RFC6790? Or allocate a single label rather than a label range for a given FEC?

Mistake that you need a new level of stack to support load balancing.
If MPLS would provide few additional bits in the header day one
Kireeti would not need to invent entropy labels at all. :)

With SID range we can accomplish the same without need to touch MPLS
data plane so it is fully backwords compatible too.

Cheers,
R.

---

From: Xuxiaohu <[email-masked]>
Date: Fri, 6 Jun 2014 09:01:47 +0000

> Sent: Friday, June 06, 2014 3:45 PM
> To: Xuxiaohu
> You sure can. That was not my point. My point was that we should analyze and
> also consider alternatives other then EL approach.
> 
> 
> > be orders of magnitude less in the SR concept) is true. Compare LDP
> > with the IGP-based label distribution protocol, why do you believe the
> > latter means orders of magnitude less of bindings?
> 
> 
> Because as it has been already proven in other threads number of SR nodes in a
> given domain may be much less then number of IGP nodes. (I remove the adj
> SIDs requirement for now). In the latter and in the LDP based FECs you rather
> must support signalling of ranges of FECs end to end for it to work. With SR you
> can just do it for 10 nodes in 500 node IGP domain to get pretty good results.
> That's key difference.

Oho, in the LDP case, you can use the local label allocation filtering to selectively allocate local labels for a subset of the prefixes learned from the IGP. IMHO, there is no significant difference between LDP and IGP as the label distribution protocol, especially from the mpls forwarding table scalability perspective.

> > What' the same mistakes in your mind? The entropy label concept as defined
> in RFC6790? Or allocate a single label rather than a label range for a given FEC?
> 
> If MPLS would provide few additional bits in the header day one Kireeti would
> not need to invent entropy labels at all. :)

>From the MPLS label stack perspective, containing few additional bits at every MPLS header of the label stack may not be ideal as well:)

> With SID range we can accomplish the same without need to touch MPLS data
> plane so it is fully backwords compatible too.

The above argument is not true unless the MPLS forwarding is based on the longest-match algorithm:) Otherwise, the MPLS forwarding table scalability issue will emerge while the load-balancing issue is solved.

Best regards,
Xiaohu