Subject: [mpls] Concerns about ISD

From: Tianran Zhou <[email-masked]>
Date: Thu, 7 Apr 2022 11:39:11 +0000

I followed the discussion in the mailing list for a while,  and read all the proposed solutions.
As I see some proposals mentioned about ISD, I do not think we really need it.
The only reason I see is the lower parsing depth.
However, the complexity we add to the implementation and the backward compatibility issue is a disaster.

Let's see some concerns:
1. ISD breaks the MPLS stack. It's very different from existing implementations. Different label processions are required for the whole stack.
2. ISD has constrained space and format. For example, the s bit is reserved to not use. This will fragment the data space. Hence introduce a lot of complexities on the parsing, processing, encoding, hence damage the flexibility and extensibility.
3. Pop the entire ISD within a lsp/sr-path is a very different procession to existing implementation.
4. ISD itself will increase the depth of the label stack. Several ISDs encoded in the label stack will severely impact the payload efficiency.
5.  If ISD carries any field that can change en route, it will impact the ECMP hash.
6. ISD cannot replace PSD. If we can achieve with only PSD with one piece of code, I can find no reason to implement both ISD and PSD.

In all, my design philosophy is simplicity. However, ISD breaks the elegant MPLS label stack.
I would strongly object to ISD.

Best,
Tianran

---

From: Tony Li <[email-masked]>
Date: Thu, 7 Apr 2022 07:14:51 -0700

Hi Tianran,

Thank you for your opinion.

> 1. ISD breaks the MPLS stack. It's very different from existing implementations. Different label processions are required for the whole stack.

The precedent, of course, is the Entropy label, which is effectively ISD.

> 2. ISD has constrained space and format. For example, the s bit is reserved to not use. This will fragment the data space. Hence introduce a lot of complexities on the parsing, processing, encoding…

Working around the s bit is certainly annoying and time consuming, but is not unduly complex. If your implementation is motivated to concatenate around the s bit, then two mask operations, a shift, and an or operation will give you contiguous bits.

> 3. Pop the entire ISD within a lsp/sr-path is a very different procession to existing implementation.

Yes, the code will have to change. Adding functions requires new code.

> 5.  If ISD carries any field that can change en route, it will impact the ECMP hash.

That’s implementation specific. An implementation is not required to use the entire stack in doing an ECMP hash.  You could just hash on embedded entropy labels.

> 6. ISD cannot replace PSD. If we can achieve with only PSD with one piece of code, I can find no reason to implement both ISD and PSD.

As you note, the advantage of ISD is that it lowers the parsing depth. Pushing everything to PSD causes a major performance hit if relevant data is past the readable label stack depth.

> I would strongly object to ISD.

Thank you for commenting,
Tony

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 8 Apr 2022 06:35:24 +0000

Hi Tony,
Thanks for your reply and good to know you thoughts.
However, what I see the ISD is a specific optimization for specific hardware.
I do not think we should standardize such a solution.

Please see my reply in line for detail.

Best,
Tianran.

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Thursday, April 7, 2022 10:15 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

Let's see some concerns:

ZTR> I do not think this is a good example. Firstly, entropy label is still an MPLS label, but ISD is not, with very different process. Secondly, RFC6790 indicates the in stack process of entropy label is optional. That said, this is only for specific hardware.

Working around the s bit is certainly annoying and time consuming, but is not unduly complex. If your implementation is motivated to concatenate around the s bit, then two mask operations, a shift, and an or operation will give you contiguous bits.
ZTR> Of cause, anyway you can implement. But not elegant.

That’s implementation specific. An implementation is not required to use the entire stack in doing an ECMP hash.  You could just hash on embedded entropy labels.
ZTR> But there are different implementations to do hash. You do not know the device in the network, if they can change code or not. This will course backward compatibility issue.

As you note, the advantage of ISD is that it lowers the parsing depth. Pushing everything to PSD causes a major performance hit if relevant data is past the readable label stack depth.
ZTR> As you note, anyway you can implement, even the parsing buffer is low, you can loop or cache twice for PSD. But let’s see further on the ISD use cases. For MPLS/LSP, the normal case is two or three labels. There is no big difference on in stack or post stack. For MPLS-SR, the path is programmable. And PSD could be ignored by node not support. On the other hand, if the device is resource constraint, with limited parsing buffer or so, I do not think it should even consider the ISD for programmability. Especially, ISD will introduce complexity on the constraint device.

---

From: Tony Li <[email-masked]>
Date: Fri, 8 Apr 2022 08:20:29 -0700

Hi Tianran,

> I do not think we should standardize such a solution.

The readable label stack depth has long been a concern. Nothing about MNA is specific to any given piece of hardware.

> ZTR> I do not think this is a good example. Firstly, entropy label is still an MPLS label, but ISD is not, with very different process. Secondly, RFC6790 indicates the in stack process of entropy l…

To me, the entropy label is 20 random bits that one system adds at the start of the LSP. While it occupies a label field in an LSE, it is in no way a label. Specifically, it is not an index that a system uses for a lookup in its LFIB.  In other words, it’s opaque data, use solely for systems that need to load balance. ISD is certainly more opaque data. The functionality is, of course, different.  Using MNA is, of course, optional. 

> ZTR> Of cause, anyway you can implement. But not elegant.

Welcome to the IETF. We do real engineering. It’s not always elegant. :)

> ZTR> But there are different implementations to do hash. You do not know the device in the network, if they can change code or not. This will course backward compatibility issue.

Devices that cannot upgrade their code will not be able to support any form of MNA. Is this somehow worth debating?

> ZTR> As you note, anyway you can implement, even the parsing buffer is low, you can loop or cache twice for PSD. But let’s see further on the ISD use cases. For MPLS/LSP, the normal case is two or…

Looping or recycling through a forwarding engine can cause a major performance hit (50% or more). Folks would like to avoid that.

With SR, you can have much deeper stack depths. That can cause a big difference in access times.

I don’t follow the rest of your argument. You seem to be implying that other vendors should simply give up. Somehow, I think that unlikely. :)

I expect that devices that can support MNA will try to support MNA. It is to our collective advantage (there goes my altruistic streak again) to get MNA deployed and to help network operators get as much value from MNA as possible. Asking an operator to discard even a tiny portion of their hardware is unlikely to result in any kind of progress, only hindering MNA.

Regards,
Tony

---

From: Tianran Zhou <[email-masked]>
Date: Mon, 11 Apr 2022 04:26:38 +0000

Hi Tony,

Please see in line.

Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Friday, April 8, 2022 11:20 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

ZTR> I did not see an RFC specifies the number of readable label stack (e.g. 4? 6?). Hardware varies and can keep evolving. The protocol should work on one direction, so that the hardware can keep optimizing on that direction. I did not ignore the requirement for on-path action.

ZTR> Entropy label could be opaque data. Because, we do not actually parse the label field. However, ISD is not. It’s certainly not “more opaque data”, devices need to parse and process the label field.  And I think you mixed the requirement for on-path action and ISD.
The on-path action is required, and of course optional, people can choose whether to add or not. However, ISD is one implementation, all we discussed are whether we need such a special implementation, while there is PSD.

ZTR> That sounds like any code could be IETF standard. I see the real engineering is the parser cache keeps growing. I see the real engineering is consistent/standard MPLS label operation is hardened in AISC for better performance.

ZTR> My point is about backward compatibility. PSD gets better backward compatibility than ISD.

ZTR> So you agree ISD is designed for some legacy hardware. I argue that ISD will introduce more code and complexity, while PSD can do on-path action. And it will not reward people to implement PSD, hence hinder the on-path action. I argue that ISD will introduce backward compatibility issue, while PSD can support the incremental deployment. Hence PSD is helpful for operators to deploy on-path action in their network for new features. Vendors can do private ISD implementations for some legacy devices. However, I do not think the community should waste energy on ISD standardization.

Regards,
Tony

---

From: Tony Li <[email-masked]>
Date: Sun, 10 Apr 2022 22:57:56 -0700

Hi Tianran,

> ZTR> I did not see an RFC specifies the number of readable label stack (e.g. 4? 6?). Hardware varies and can keep evolving. The protocol should work on one direction, so that the hardware can keep…

Please see RFC 9088 and 9089.

While hardware continues to evolve, as we’ve discussed previously, we really want to try to maximize the value of the hardware that’s already in the field. That’s in  the operator’s best interests and thus in ours as well.

> The on-path action is required, and of course optional, people can choose whether to add or not. However, ISD is one implementation, all we discussed are whether we need such a special implementati…

I’m sorry, I’m not following your point here.

> ZTR> That sounds like any code could be IETF standard. I see the real engineering is the parser cache keeps growing. I see the real engineering is consistent/standard MPLS label operation is harden…

I see hard coding an ASIC as a poor choice. But I’ve only been saying that since 1996. :)  All of the silicon that I’ve had a hand in has been microcoded for exactly this reason, with very little penalty.

> ZTR> My point is about backward compatibility. PSD gets better backward compatibility than ISD.

I strongly disagree.  Putting everything in PSD makes it impossible for devices with limited readable label stack depth to participate effectively.

> ZTR> So you agree ISD is designed for some legacy hardware. I argue that ISD will introduce more code and complexity, while PSD can do on-path action. And it will not reward people to implement PSD…

I disagree and I think it’s time that we just agree to disagree.

Tony

---

From: Tianran Zhou <[email-masked]>
Date: Mon, 11 Apr 2022 07:35:39 +0000

Hi Tony,

ZTR> I quickly go over this two RFCs. I did not see any limits on the number of readable labels. On the contrast, it allows devices with different capabilities. By using the flooding and notification mechanism, the controller or so knows the capability of each device, hence easy to program the stack on the head node.

ZTR> I disagree. New chips are combination of AICS part and microcode part. Microcode is for changes and flexibility. But ASIC is for performance.

For the other issues, you still cannot address my concerns, although you disagree.

Best,
Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Monday, April 11, 2022 1:58 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

Tony

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 11 Apr 2022 09:18:26 +0000

Hi Tony and Tianran,

Please see some observations about this discussion inline with [Jie]:

From: mpls [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Monday, April 11, 2022 1:58 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

[Jie] IMO one of the difference between entropy label and ISD is whether the format of the label field, the TTL field and TC field are redefined or not. This will impact the parsing and processing implementation of network nodes. And concatenation of multiple fields in the same or different LSEs would be a very different behavior from the existing MPLS label processing.

[Jie] For backward compatibility, one thing to consider is the risk of ISD being exposed at the top of label stack on the legacy nodes, which may be misunderstood as forwarding labels.

[Jie] This shows that further hardware analysis is really needed. We’ve had some discussion about the hardware-friendly design principles, and the comparison between the optional encodings in terms of processing complexity and overhead.

I hope at least we can agree that MIAD or MNA requires new functionalities, then it would be helpful to understand what can and cannot be done with legacy hardware, what would be the requirements to new hardware, and hardware people’s feedback about these requirements.

Best regards,
Jie

Tony

---

From: Tony Li <[email-masked]>
Date: Mon, 11 Apr 2022 08:34:36 -0700

Hi Tianran,

> ZTR> I quickly go over this two RFCs. I did not see any limits on the number of readable labels. On the contrast, it allows devices with different capabilities. By using the flooding and notificati…

Exactly.  This exists because there is a need to understand the readable label stack depth (RLSD) and operate within it.  RLSD is important for interoperability and pushing everything to PSD would hinder interoperability. People did things this way for a reason. It makes sense for us to continue to do so.

> ZTR> I disagree. New chips are combination of AICS part and microcode part. Microcode is for changes and flexibility. But ASIC is for performance.

Ok, clearly we operate with different silicon technology.

In a related matter, it does occur to me that if you dislike ISD, you are free to take any sub-stacks that you want to inject and push them to the bottom of the label stack. This would effectively push all of your ISD just above PSD, effectively doing the same thing as doing everything in PSD.  Of course, this would minimize your interoperability.  Other devices would be able to use MNA through your device, but you would not be able to send MNA through them. If this is what you want, we won’t stop you.

Regards,
Tony

---

From: Tony Li <[email-masked]>
Date: Mon, 11 Apr 2022 08:42:45 -0700

Hi Jie,

> [Jie] IMO one of the difference between entropy label and ISD is whether the format of the label field, the TTL field and TC field are redefined or not. This will impact the parsing and processing…

There is no doubt that the processing is different.  This is to be expected, we’re adding functionality.

> [Jie] For backward compatibility, one thing to consider is the risk of ISD being exposed at the top of label stack on the legacy nodes, which may be misunderstood as forwarding labels.

I believe that we’ve been pretty clear: we’re using signaling to indicate which nodes support MNA. It’s up to the head end to do path selection that is MNA capable (and more specifically, action capable, if necessary).

>  [Jie] This shows that further hardware analysis is really needed. We’ve had some discussion about the hardware-friendly design principles, and the comparison between the optional encodings in term…

What is it that you’re hoping to learn?  We’ve been through a few rounds of hardware analysis and had dissimilar results.

> I hope at least we can agree that MIAD or MNA requires new functionalities, then it would be helpful to understand what can and cannot be done with legacy hardware, what would be the requirements t…

I think we agree that it requires new functions. What can and cannot be done with legacy hardware varies with the specific hardware and no one is willing to disclose what their hardware can and cannot do. There are no requirements for new hardware as it is presumably more capable than existing hardware. The feedback from the various hardware analysis has been inconsistent.

Regards,
Tony

---

From: Tianran Zhou <[email-masked]>
Date: Tue, 12 Apr 2022 02:07:55 +0000

Hi Tony,

Please see in line.

Best,
Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Monday, April 11, 2022 11:35 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

ZTR2> PSD can work with RLSD, I cannot see how it will hinder the interoperability.

ZTR2> Of course I would not put ISD just above PSD. As PSD can provide all the functionalities, I do not see the reason why we need ISD anymore. Entropy label is a simple one, community chose to implement the in stack operation <optionally> as in RFC 6790. For ISD, I would not expect it to split the community effort. You can implement any fascinating ISD design privately, we won’t stop you. But I do not think we need any IETF standard on ISD.

Regards,
Tony

---

From: Greg Mirsky <[email-masked]>
Date: Mon, 11 Apr 2022 19:47:15 -0700

Tianran, Tony, and Jie,
thank you for a lively and helpful discussion. I believe it is good to lay
our assumptions on the table, check what is real and what is not so much.
To me, both ISD and PSD are essential elements of MNA. I expect that there
will be lots of considerations, discussions and arguments around where a
particular MNA will have its associated data - ISD or PSD. And, as the case
with ELI/EL, there will come necessary extensions to routing protocols and
corresponding YANG data models. In my view, the new MPLS data plane must be
accompanied with extensions in the control and management plane.
To summarize my position, it seems that constrains of available today HW
must be considered but should not prevent us from defining a more powerful,
flexible new MPLS architecture. As our common experience has demonstrated
again and again - HW advances faster than we can standardize protocols. I
view architectural optional capabilities as an opportunity to realize a new
functionality, for example on-path telemetry like IOAM, but only when when
that does not affect the main goal of the network - efficiently forward
data flows. that seems as an obvious, and I believe that the MNA framework
and requirements documents express it clearly. We only need to follow them.

Regards,
Greg

On Mon, Apr 11, 2022 at 7:08 PM Tianran Zhou <zhoutianran=
[email-masked]> wrote:

> Hi Tony,
>
>
>
> Please see in line.
>
>
>
> Best,
>
> Tianran
>
>
>
> *From:* Tony Li [mailto:[email-masked]] *On Behalf Of *Tony Li
> *Sent:* Monday, April 11, 2022 11:35 PM
> *To:* Tianran Zhou <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* Re: [mpls] Concerns about ISD
>
>
>
>
>
> Hi Tianran,
>
>
>
>
>
> While hardware continues to evolve, as we’ve discussed previously, we
> really want to try to maximize the value of the hardware that’s already in
> the field. That’s in  the operator’s best interests and thus in ours as
> well.
>
>
>
> ZTR> I quickly go over this two RFCs. I did not see any limits on the
> number of readable labels. On the contrast, it allows devices with
> different capabilities. By using the flooding and notification mechanism,
> the controller or so knows the capability of each device, hence easy to
> program the stack on the head node.
>
>
>
>
>
> Exactly.  This exists because there is a need to understand the readable
> label stack depth (RLSD) and operate within it.  RLSD is important for
> interoperability and pushing everything to PSD would hinder
> interoperability. People did things this way for a reason. It makes sense
> for us to continue to do so.
>
>
>
> I see hard coding an ASIC as a poor choice. But I’ve only been saying that
> since 1996. :)  All of the silicon that I’ve had a hand in has been
> microcoded for exactly this reason, with very little penalty.
>
>
>
> ZTR> I disagree. New chips are combination of AICS part and microcode
> part. Microcode is for changes and flexibility. But ASIC is for performance.
>
>
>
> In a related matter, it does occur to me that if you dislike ISD, you are
> free to take any sub-stacks that you want to inject and push them to the
> bottom of the label stack. This would effectively push all of your ISD just
> above PSD, effectively doing the same thing as doing everything in PSD.  Of
> course, this would minimize your interoperability.  Other devices would be
> able to use MNA through your device, but you would not be able to send MNA
> through them. If this is what you want, we won’t stop you.
>
>
>
> ZTR2> Of course I would not put ISD just above PSD. As PSD can provide all
> the functionalities, I do not see the reason why we need ISD anymore.
> Entropy label is a simple one, community chose to implement the in stack
> operation <optionally> as in RFC 6790. For ISD, I would not expect it to
> split the community effort. You can implement any fascinating ISD design
> privately, we won’t stop you. But I do not think we need any IETF standard
> on ISD.
>
>
>
> Regards,
>
> Tony
>
>
>
>
> _______________________________________________
> mpls mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mpls
>

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 12 Apr 2022 03:19:45 +0000

Hi Tony,

It is good we agree that new functionalities are required. Then it is important to compare how much the MPLS architecture will be changed by different proposals, and their implications to hardware.

In the Open DT, we are still missing the hardware analysis which support the changes introduced by ISD. Can they be provided in the following DT meetings?

It is true that hardware capability varies, without concrete analysis it is hard to say one proposal could work with legacy hardware or not.

Best regards,
Jie

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Monday, April 11, 2022 11:43 PM
To: Dongjie (Jimmy) <[email-masked]>
Cc: Tianran Zhou <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Jie,

Regards,
Tony

---

From: Tony Li <[email-masked]>
Date: Mon, 11 Apr 2022 22:55:57 -0700

Greg,

Thank you. Well said.

Tony

> On Apr 11, 2022, at 7:47 PM, Greg Mirsky <[email-masked]> wrote:
> 
> thank you for a lively and helpful discussion. I believe it is good to lay our assumptions on the table, check what is real and what is not so much.
> To me, both ISD and PSD are essential elements of MNA. I expect that there will be lots of considerations, discussions and arguments around where a particular MNA will have its associated data - ISD or PSD. And, as the case with ELI/EL, there will come necessary extensions to routing protocols and corresponding YANG data models. In my view, the new MPLS data plane must be accompanied with extensions in the control and management plane. 
> To summarize my position, it seems that constrains of available today HW must be considered but should not prevent us from defining a more powerful, flexible new MPLS architecture. As our common experience has demonstrated again and again - HW advances faster than we can standardize protocols. I view architectural optional capabilities as an opportunity to realize a new functionality, for example on-path telemetry like IOAM, but only when when that does not affect the main goal of the network - efficiently forward data flows. that seems as an obvious, and I believe that the MNA framework and requirements documents express it clearly. We only need to follow them.

---

From: Tony Li <[email-masked]>
Date: Mon, 11 Apr 2022 23:00:01 -0700

Hi Jie,

> It is true that hardware capability varies, without concrete analysis it is hard to say one proposal could work with legacy hardware or not.

We have had two presentations of hardware analysis. 

One on 3/10/2022 and another on 1/27/2022.  What more are you expecting?

Tony

---

From: Tony Li <[email-masked]>
Date: Mon, 11 Apr 2022 23:02:45 -0700

Hi Tianran,

> ZTR2> PSD can work with RLSD, I cannot see how it will hinder the interoperability.

Pushing data to PSD and beyond the RLSD will cause systems to be excluded from the path or take a significant performance hit. 

Tony

---

From: Tianran Zhou <[email-masked]>
Date: Tue, 12 Apr 2022 06:35:42 +0000

Hi Tony,

>>Pushing data to PSD and beyond the RLSD will cause systems to be excluded from the path or take a significant performance hit.

If I know the RLSD of each node, there are many ways to prevent some nodes from pushing data to PSD.
I would like to know how ISD could survive if the ISD exceed the RLSD?

Best,
Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Tuesday, April 12, 2022 2:03 PM
To: Tianran Zhou <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

Tony

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 12 Apr 2022 07:50:57 +0000

Hi Tony,

Sent: Tuesday, April 12, 2022 2:00 PM

Hi Jie,

[Jie] The analysis we had on 1/27/2022 and 3/3/2022 in the DT showed the overhead and complexity of ISD, and questioned whether ISD is needed or not. The analysis slides are attached in the DT meeting notes.

We are still missing equivalent hardware analysis from the proponents of draft-kompella-mpls-mspl4fa. Without that, it is hard to tell whether the proposal is implementable with existing or new hardware, and what would be the cost.

Best regards,
Jie

Tony

---

From: John E Drake <[email-masked]>
Date: Tue, 12 Apr 2022 12:45:25 +0000

Hi,

Comments inline.

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]> On Behalf Of Tianran Zhou
Sent: Tuesday, April 12, 2022 2:36 AM
To: Tony Li <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

[External Email. Be cautious of content]

Hi Tony,

[JD]  Given that you *only* want to use PSD, what is the above sentence proposing?

[JD]  The correct term is Network Actions  Sub-stack (NAS).  To answer your question, a transit node will miss the NAS regardless of whether it contains in-stack data.  This is the point that Tony has been making for the past few email iterations

Best,
Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Tuesday, April 12, 2022 2:03 PM
To: Tianran Zhou <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

Tony

---

From: Tony Li <[email-masked]>
Date: Tue, 12 Apr 2022 07:13:33 -0700

Hi Jie,

> We are still missing equivalent hardware analysis from the proponents of draft-kompella-mpls-mspl4fa. Without that, it is hard to tell whether the proposal is implementable with existing or new har…

That was the presentation on 3/10.

As I recall, the result was that it was easily implementable, but the recommendation was to replace continuation bits with length fields.

T

---

From: Haoyu Song <[email-masked]>
Date: Tue, 12 Apr 2022 15:45:58 +0000

Hi Tony,

As I recall, Kireeti only gave a few words claiming it’s easily implementable, while the other vendors (HW and Cisco) confirmed my analysis with their analysis documented in PPTs (specifically, the bitmap encoded ISD increases both parser size and parsing latency).
I believe detailed analysis based on the solid assumption (e.g., parse-able header depth) is needed to understand the tradeoff. After all, for both old and new hardware, we prefer simpler design for higher performance.

Best regards,
Haoyu

From: mpls <[email-masked]> On Behalf Of Tony Li
Sent: Tuesday, April 12, 2022 7:14 AM
To: Dongjie (Jimmy) <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Jie,

T

---

From: Tony Li <[email-masked]>
Date: Tue, 12 Apr 2022 08:51:10 -0700

Hi Haoyu,

> As I recall, Kireeti only gave a few words claiming it’s easily implementable, while the other vendors (HW and Cisco) confirmed my analysis with their analysis documented in PPTs (specifically, the…

That was Kireeti’s summary of the discussions that he had with hardware folks. Bitmap encoding is not germane to the ISD discussion at hand.

Tony

---

From: Haoyu Song <[email-masked]>
Date: Tue, 12 Apr 2022 15:55:26 +0000

Well, could we ask for a presentation then? Thanks!

Haoyu

From: Tony Li <[email-masked]> On Behalf Of Tony Li
Sent: Tuesday, April 12, 2022 8:51 AM
To: Haoyu Song <[email-masked]>
Cc: Dongjie (Jimmy) <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

Tony

---

From: Tony Li <[email-masked]>
Date: Tue, 12 Apr 2022 09:08:57 -0700

> Well, could we ask for a presentation then? Thanks!

You’re welcome to ask Kireeti for a reprise. However, I think it would be much more effective if the chairs simply posted the recording of the meeting.

T

---

From: Haoyu Song <[email-masked]>
Date: Tue, 12 Apr 2022 17:41:37 +0000

Hi Greg,

I agree that hardware does advance faster than we can standardize protocols, but there is still a distinction between good design and bad design. While tomorrow's hardware advance may make bad design 10x better than today's good design, it's quite possible that it can also make good design 20x better. Thorough evaluation and then sticking to good design are mandatory to our engineers.

Regards,
Haoyu
From: mpls <[email-masked]> On Behalf Of Greg Mirsky
Sent: Monday, April 11, 2022 7:47 PM
To: Tianran Zhou <zhoutianran=[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Tianran, Tony, and Jie,

Regards,
Greg

On Mon, Apr 11, 2022 at 7:08 PM Tianran Zhou <zhoutianran=[email-masked]<mailto:[email-masked]>> wrote:
Hi Tony,

Please see in line.

Best,
Tianran

From: Tony Li [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Tony Li

Hi Tianran,

Please see RFC 9088 and 9089.
While hardware continues to evolve, as we've discussed previously, we really want to try to maximize the value of the hardware that's already in the field. That's in  the operator's best interests and thus in ours as well.

ZTR2> PSD can work with RLSD, I cannot see how it will hinder the interoperability.
I see hard coding an ASIC as a poor choice. But I've only been saying that since 1996. :)  All of the silicon that I've had a hand in has been microcoded for exactly this reason, with very little penalty.

In a related matter, it does occur to me that if you dislike ISD, you are free to take any sub-stacks that you want to inject and push them to the bottom of the label stack. This would effectively push all of your ISD just above PSD, effectively doing the same thing as doing everything in PSD.  Of course, this would minimize your interoperability.  Other devices would be able to use MNA through your device, but you would not be able to send MNA through them. If this is what you want, we won't stop you.

ZTR2> Of course I would not put ISD just above PSD. As PSD can provide all the functionalities, I do not see the reason why we need ISD anymore. Entropy label is a simple one, community chose to implement the in stack operation <optionally> as in RFC 6790. For ISD, I would not expect it to split the community effort. You can implement any fascinating ISD design privately, we won't stop you. But I do not think we need any IETF standard on ISD.

Regards,
Tony

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls&data=04%7C01%7Chaoyu.song%40futurewei.com%7C63859641e33c401a5ec408da1c2ed8e9%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637853284800900578%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=M3PJMkwtXo1TArtRKtewxvwMGuEK%2F2IkvdkAA4AUzKE%3D&reserved=0>

---

From: Haoyu Song <[email-masked]>
Date: Tue, 12 Apr 2022 17:47:07 +0000

I think the recording is already on the wiki. I guess I’m expecting detailed comparison and analysis rather than reiterating the conclusion. Thanks!

Haoyu

Sent: Tuesday, April 12, 2022 9:09 AM

T

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 13 Apr 2022 01:17:00 +0000

Hi,

Comments in line.

Tianran

Juniper Business Use Only
From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Tianran Zhou
Sent: Tuesday, April 12, 2022 2:36 AM
To: Tony Li <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Tony,

ZTR> If you followed the past few emails, Tony misled people that PSD will hinder the interoperability. I just show you how PSD will not. According to your description below on the ISD process, it's the same as PSD somehow. I did not see the ISD advantage.

ZTR> Creating new terms does not help the community. I do not care the fancy term you created. Could you please point me out how many times Tony has clarified the ISD process? And where they are? I may have missed.
In my brain, Tony did not address any of my concerns on ISD, but only kept ignoring the fact I presented. It's not a good way for technique discussion.

Best,
Tianran

Hi Tianran,

Tony

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 12 Apr 2022 19:09:36 -0700

Hi Tianran,
I have got several questions based on the discussion we had of the use
cases and hope you could share your opinion:

   - Do you see a benefit of supporting EL using NMA? If EL is supported by
   NMA, should it be in ISD or PSD?
   - If NMA supports GISS, where do you see it - ISD or PSD?
   - If NMA to support IOAM in MPLS be only Preallocated Trace Option and
   Direct Export types, would you consider carrying the IOAM Header in ISD?

Regards,
Greg

On Tue, Apr 12, 2022 at 6:17 PM Tianran Zhou <zhoutianran=
[email-masked]> wrote:

> *From:* mpls <[email-masked]> *On Behalf Of *Tianran Zhou
> *Sent:* Tuesday, April 12, 2022 2:36 AM
> *To:* Tony Li <[email-masked]>
> *Cc:* [email-masked]
> *[External Email. Be cautious of content]*
>
>
>
> Hi Tony,
>
>
>
> >>Pushing data to PSD and beyond the RLSD will cause systems to be
> excluded from the path or take a significant performance hit.
>
>
>
> *[JD]  Given that you *only* want to use PSD, what is the above sentence
> proposing? *
>
>
>
> *ZTR> If you followed the past few emails, Tony misled people that PSD
> will hinder the interoperability. I just show you how PSD will not.
> According to your description below on the ISD process, it’s the same as
> PSD somehow. I did not see the ISD advantage.*
>
>
>
> *[JD]  The correct term is Network Actions  Sub-stack (NAS).  To answer
> your question, a transit node will miss the NAS regardless of whether it
> contains in-stack data.  This is the point that Tony has been making for
> the past few email iterations*
>
>
>
> *ZTR> Creating new terms does not help the community. I do not care the
> fancy term you created. Could you please point me out how many times Tony
> has clarified the ISD process? And where they are? I may have missed.*
>
> *In my brain, Tony did not address any of my concerns on ISD, but only
> kept ignoring the fact I presented. It’s not a good way for technique
> discussion.*
>
>
>
> Best,
> Tianran
>
>
>
> *From:* Tony Li [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Tony Li
> *Sent:* Tuesday, April 12, 2022 2:03 PM
> Pushing data to PSD and beyond the RLSD will cause systems to be excluded
> from the path or take a significant performance hit.
>
>
>
> Tony
>
>

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 13 Apr 2022 07:16:22 +0000

Hi Greg,

Please see my thoughts in line.

Cheers,
Tianran

From: Greg Mirsky [mailto:[email-masked]]
Sent: Wednesday, April 13, 2022 10:10 AM
To: Tianran Zhou <[email-masked]>
Cc: John E Drake <[email-masked]>; Tony Li <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,
I have got several questions based on the discussion we had of the use cases and hope you could share your opinion:

  *   Do you see a benefit of supporting EL using NMA? If EL is supported by NMA, should it be in ISD or PSD?
ZTR> I do not see the “benefit”.  I think EL is a mature thing, already implemented everywhere. There is no need to duplicate.

  *   If NMA supports GISS, where do you see it - ISD or PSD?
ZTR> I will use PSD to support slicing, as this one: https://datatracker.ietf.org/doc/html/draft-li-mpls-enhanced-vpn-vtn-id-02.

  *   If NMA to support IOAM in MPLS be only Preallocated Trace Option and Direct Export types, would you consider carrying the IOAM Header in ISD?
ZTR> I am not going to use ISD for IOAM. At first look, IOAM data format is already specified.  I do not see how to encode into ISD, with inconsecutive space(S bit in between).
Regards,
Greg

On Tue, Apr 12, 2022 at 6:17 PM Tianran Zhou <zhoutianran=[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Comments in line.

Tianran

Hi Tony,

ZTR> If you followed the past few emails, Tony misled people that PSD will hinder the interoperability. I just show you how PSD will not. According to your description below on the ISD process, it’s the same as PSD somehow. I did not see the ISD advantage.

ZTR> Creating new terms does not help the community. I do not care the fancy term you created. Could you please point me out how many times Tony has clarified the ISD process? And where they are? I may have missed.
In my brain, Tony did not address any of my concerns on ISD, but only kept ignoring the fact I presented. It’s not a good way for technique discussion.

Best,
Tianran

Hi Tianran,

Tony

---

From: Kireeti Kompella <[email-masked]>
Date: Wed, 13 Apr 2022 11:29:27 +0200

Hi Haoyu,

See inline. 

> On Apr 12, 2022, at 17:46, Haoyu Song <[email-masked]> wrote:
> 
> 
> Hi Tony,
>  
> As I recall, Kireeti only gave a few words claiming

“claiming” :)

Yeah, no microcode. Sorry. 

> it’s easily implementable, while the other vendors (HW and Cisco) confirmed my analysis with their analysis documented in PPTs (specifically, the bitmap encoded ISD increases both parser size and parsing latency).

There are three things to keep in mind:

1) there wasn’t a detailed (public) analysis of how to implement MPLS when it was first proposed: where to put the BoS, TC, how many bits for the label, etc., based on existing hardware. That’s not the IETF approach. While we appreciate your detailed analysis, it is just one data point. 

2) Hardware adapts. Parsing a three label stack was hard once. We are setting standards, not for today, but for the next decade, if not longer. Analysis based solely on today’s hardware is the wrong approach. Analysis based on hardware 10 years from now needs a crystal ball (mine is kaput). So, we use architectural principles as best we can. 

3) I’m not saying no PSD, I’m saying do both. Each has its strongpoints. You’re saying, no ISD — that’s a very narrow viewpoint. (We do both today: ELI and CW.)

At this point, I’m done with this topic. It’s an exciting subject and lots of good debate. Chairs, how do we move forward?

Kireeti

---

From: John E Drake <[email-masked]>
Date: Wed, 13 Apr 2022 13:27:54 +0000

Comments inline below, identified with [JD2].  I think it's time to move on from this discussion, as we seem to have completed it.

Yours Irrespectively,

John

Juniper Business Use Only
From: Tianran Zhou <zhoutianran=[email-masked]>
Sent: Tuesday, April 12, 2022 9:17 PM
To: John E Drake <[email-masked]>; Tony Li <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Hi,

Comments in line.

Tianran

Hi Tony,

ZTR> If you followed the past few emails, Tony misled people that PSD will hinder the interoperability.

[JD2]  I understood his emails to say that many implementations cannot find the bottom of the label stack, as documented in RFC 8662.  Therefore, those implementations
would not be able to process ancillary post-stack data.  This is just fact

I just show you how PSD will not.

[JD2]  You did nothing of the kind.  You asserted, above, that the problem would be solved by not imposing ancillary post-stack data;  this is self-defeating

According to your description below on the ISD process, it's the same as PSD somehow. I did not see the ISD advantage.

[JD2]  I have no idea what the above statement is trying to say

ZTR> Creating new terms does not help the community. I do not care the fancy term you created.

[JD2]  The NAS, as described in the framework draft, consists of a network actions indicator, a specified set of network actions, and any ancillary in-stack data required by those network actions.  (If the specified network actions required ancillary post-stack data, it will also be present.)  The implications of the issue described in RFC 8662 are that multiple copies of an NAS may be required in an MPLS label stack and that some network actions may require ancillary in-stack data, which will be present in each copy of an NAS

Could you please point me out how many times Tony has clarified the ISD process? And where they are? I may have missed.
In my brain, Tony did not address any of my concerns on ISD, but only kept ignoring the fact I presented. It's not a good way for technique discussion.

[JD2] What I read is that you don't like in-stack data and an assertion, contrary to RFC 8662, that either all nodes can handle ancillary post-stack data or that if they can't, ancillary post-stack data won't be included

Best,
Tianran

Hi Tianran,

Tony

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 13 Apr 2022 07:43:33 -0700

Hi Tianran,
thank you for sharing your thoughts. Of course, there's no way to slice the
bread and whether to use only PSD comes with its limitations like burning a
value for the first nibble after the BoS. And it needs to interwork with PW
CW and ACH/G-ACH. I see an ISD-based solution as more flexible and easier
to apply to MPLS PWs.

Regards,
Greg

On Wed, Apr 13, 2022 at 12:16 AM Tianran Zhou <[email-masked]>
wrote:

> *From:* Greg Mirsky [mailto:[email-masked]]
> *Sent:* Wednesday, April 13, 2022 10:10 AM
> *Cc:* John E Drake <[email-masked]>; Tony Li <[email-masked]>;
> [email-masked]
>    - Do you see a benefit of supporting EL using NMA? If EL is supported
>    by NMA, should it be in ISD or PSD?
>
> ZTR> I do not see the “benefit”.  I think EL is a mature thing, already
> implemented everywhere. There is no need to duplicate.
>
> ZTR> I will use PSD to support slicing, as this one:
> https://datatracker.ietf.org/doc/html/draft-li-mpls-enhanced-vpn-vtn-id-02
> .
>
> ZTR> I am not going to use ISD for IOAM. At first look, IOAM data format
> is already specified.  I do not see how to encode into ISD, with inconsecutive
> space(S bit in between).
>
> Regards,
>
> Greg
>
>
>

---

From: Haoyu Song <[email-masked]>
Date: Wed, 13 Apr 2022 16:37:37 +0000

Hi Kireeti,

As I said, hardware advance would benefit good design more than bad design. Also, Tony insists that the design should also be applied for today’s hardware. Therefore, we do need careful evaluation before making potentially costly decisions.

I have done my homework and formed my opinion. I presented it in details publicly and welcome feedback. If I'm proven wrong, I would be willing to adjust my stance. What I cannot accept is either one claims there is secret sauce that can't be revealed or simply claim the future hardware would solve all the problems. I believe in evidence but not speculation.

Best regards,

Haoyu

From: Kireeti Kompella <[email-masked]>
Sent: Wednesday, April 13, 2022 2:29 AM
To: Haoyu Song <[email-masked]>
Cc: Tony Li <[email-masked]>; Dongjie (Jimmy) <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,
See inline.

On Apr 12, 2022, at 17:46, Haoyu Song <[email-masked]<mailto:[email-masked]>> wrote:

Hi Tony,

“claiming” :)

Kireeti

Best regards,
Haoyu

From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Tony Li
Sent: Tuesday, April 12, 2022 7:14 AM
To: Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Jie,

T

---

From: John E Drake <[email-masked]>
Date: Wed, 13 Apr 2022 18:31:07 +0000

Hi,

Comments inline below.

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]> On Behalf Of Haoyu Song
Sent: Wednesday, April 13, 2022 12:38 PM
To: Kireeti Kompella <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Kireeti,

As I said, hardware advance would benefit good design more than bad design.

[JD]  And that is for the working group to decide

Also, Tony insists that the design should also be applied for today’s hardware. Therefore, we do need careful evaluation before making potentially costly decisions.

I have done my homework and formed my opinion. I presented it in details publicly and welcome feedback.

[JD]  The problem is that you have presented how a single hardware design works and then asserted that all hardware designs work as you have described.  Members of the design team, myself included, don’t believe this

If I'm proven wrong, I would be willing to adjust my stance. What I cannot accept is either one claims there is secret sauce that can't be revealed or simply claim the future hardware would solve all the problems. I believe in evidence but not speculation.

[JD]  It’s not in your remit to evaluate these claims or to make a decision of which design to adopt.  What the process stipulates is that hardware designers will be asked whether a given design is easy or hard to implement but they are not required to explain their reasoning, the assumption being that they are answering in good faith

Best regards,

Haoyu

From: Kireeti Kompella <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, April 13, 2022 2:29 AM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>
Cc: Tony Li <[email-masked]<mailto:[email-masked]>>; Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,
See inline.

Hi Tony,

“claiming” :)

Kireeti

Best regards,
Haoyu

Hi Jie,

T

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/mpls__;!!NEt6yMaO-gk!TjXrzMCdH7mFLrbzEc4_NQ7dDL4LxwayKEfcbrhq3T7I0-I5uRVPZDO2fH0fHng$>

---

From: Haoyu Song <[email-masked]>
Date: Wed, 13 Apr 2022 19:47:11 +0000

Inline, thanks.

From: John E Drake <[email-masked]>
Sent: Wednesday, April 13, 2022 11:31 AM
To: Haoyu Song <[email-masked]>; Kireeti Kompella <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Hi,

Comments inline below.

Yours Irrespectively,

John

Juniper Business Use Only
From: mpls <[email-masked]<mailto:[email-masked]>> On Behalf Of Haoyu Song
Sent: Wednesday, April 13, 2022 12:38 PM
To: Kireeti Kompella <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Kireeti,

[HS] Don’t you believe this claim? I don’t see why the WG can decide this. If you mean the WG should evaluate and compare different designs and decide which is better, then yes.

[HS] The parsing logic I described is general, regardless of hardware, software, or whether or not an explicit parser is implemented. It describes the common logic to understand what information is encoded. Please be specific what you don’t believe, I’ll try my best to explain further.

[HS] It’s not about easy or hard to implement, it’s about the tradeoff between performance, cost, flexibility, and extensibility. Unlike the control plane protocols, the design will run in high performance device data plane, so every clock tick counts. In this regard, “Easy” or “Hard” can be and should be evaluated based on a set of solid qualitative and quantitative criteria when we compare different schemes.  We should choose the best design to the best of our knowledge.

Best regards,

Haoyu

Hi Haoyu,
See inline.

Hi Tony,

“claiming” :)

Kireeti

Best regards,
Haoyu

Hi Jie,

T

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fmpls__%3B!!NEt6yMaO-gk!TjXrzMCdH7mFLrbzEc4_NQ7dDL4LxwayKEfcbrhq3T7I0-I5uRVPZDO2fH0fHng%24&data=05%7C01%7Chaoyu.song%40futurewei.com%7Ca882208f295949acdb9908da1d7bcc44%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637854714814416125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=t6piKQSyz%2FOC5G6KaCu39MMqLdmVFvLqxfc2Zf3aTpg%3D&reserved=0>

---

From: John E Drake <[email-masked]>
Date: Wed, 13 Apr 2022 20:09:24 +0000

Hi,

Comments inline below, indicated by [JD2}.

Yours Irrespectively,

John

Juniper Business Use Only
From: Haoyu Song <[email-masked]>
Sent: Wednesday, April 13, 2022 3:47 PM
To: John E Drake <[email-masked]>; Kireeti Kompella <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Inline, thanks.

From: John E Drake <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, April 13, 2022 11:31 AM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>; Kireeti Kompella <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: [mpls] Concerns about ISD

Hi,

Comments inline below.

Yours Irrespectively,

John

Hi Kireeti,

[JD2]  Ultimately, the WG will reach rough consensus on a design, based upon whatever criteria the individual WG members decide is important to them, in consultation with their
hardware experts.  I don’t believe that is possible to demonstrate that there is a single best design for all hardware designs/use cases

[JD2]  Kireeti has checked with hardware experts at both Juniper and Broadcom, and has found that those experts don’t necessarily agree with your conclusions.  As an aside, while not a hardware expert, I understand that parsing in not necessarily the most important aspect to consider

[JD2]  I don’t think this is true at all.  I don’t believe there is one best design for every piece of hardware that will implement MNA.  Rather, I think that we will reach some sort of compromise which is acceptable to a majority, as determined by the chairs.

Best regards,

Haoyu

Hi Haoyu,
See inline.

Hi Tony,

“claiming” :)

Kireeti

Best regards,
Haoyu

Hi Jie,

T

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls<https://urldefense.com/v3/__https:/nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fmpls__*3B!!NEt6yMaO-gk!TjXrzMCdH7mFLrbzEc4_NQ7dDL4LxwayKEfcbrhq3T7I0-I5uRVPZDO2fH0fHng*24&data=05*7C01*7Chaoyu.song*40futurewei.com*7Ca882208f295949acdb9908da1d7bcc44*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637854714814416125*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C&sdata=t6piKQSyz*2FOC5G6KaCu39MMqLdmVFvLqxfc2Zf3aTpg*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!SOyQ3gj9bkv5YOx_8jQ-_mk-MN3iD6-kGLEh0OVS196pctj8z0hwiUwoiT-kMgA$>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 13 Apr 2022 22:32:43 +0200

Hi Haoyu,

> *[HS] It’s not about easy or hard to implement, it’s about the tradeoff
> between performance, cost, flexibility, and extensibility. Unlike the
> control plane protocols, the design will run in high performance device
> data plane, so every clock tick counts. In this regard, “Easy” or “Hard”
> can be and should be evaluated based on a set of solid qualitative and
> quantitative criteria when we compare different schemes.  We should choose
> the best design to the best of our knowledge.  *
>

I believe this is very well said indeed.

What is apparently being worked on here for months is in my view merely a
CPR for MPLS.

Instead IMO (and that is why I brought up the control plane assisted model)
MPLS could way exceed capabilities of SR NP without any excessive load on
packets or major surgery to MPLS architecture by encoding reference to
forwarding behaviour itself instead of just a pile atomic "actions"
(irrespective if those are stuffed into ISD or PSD).

So yes - no draft no proposal - I get this. I will write one. When I get to
it.

Kind regards,
Robert.

---

From: Haoyu Song <[email-masked]>
Date: Wed, 13 Apr 2022 23:37:30 +0000

Hi John,

I’m a hardware guy and I’d like to discuss with your hardware experts on the design tradeoffs. You keep saying that you don’t believe my analysis so I really want to learn what’s wrong with it or which part you don’t agree with.
You can show them my slides to begin with. Thanks!

Best regards,
Haoyu
From: John E Drake <[email-masked]>
Sent: Wednesday, April 13, 2022 1:09 PM
To: Haoyu Song <[email-masked]>; Kireeti Kompella <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Hi,

Yours Irrespectively,

John

Juniper Business Use Only
From: Haoyu Song <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, April 13, 2022 3:47 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>; Kireeti Kompella <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: [mpls] Concerns about ISD

Inline, thanks.

Hi,

Comments inline below.

Yours Irrespectively,

John

Hi Kireeti,

Best regards,

Haoyu

Hi Haoyu,
See inline.

Hi Tony,

“claiming” :)

Kireeti

Best regards,
Haoyu

Hi Jie,

T

_______________________________________________
mpls mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mpls<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fnam11.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fmpls__*3B!!NEt6yMaO-gk!TjXrzMCdH7mFLrbzEc4_NQ7dDL4LxwayKEfcbrhq3T7I0-I5uRVPZDO2fH0fHng*24%26data%3D05*7C01*7Chaoyu.song*40futurewei.com*7Ca882208f295949acdb9908da1d7bcc44*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637854714814416125*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C%26sdata%3Dt6piKQSyz*2FOC5G6KaCu39MMqLdmVFvLqxfc2Zf3aTpg*3D%26reserved%3D0__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!NEt6yMaO-gk!SOyQ3gj9bkv5YOx_8jQ-_mk-MN3iD6-kGLEh0OVS196pctj8z0hwiUwoiT-kMgA%24&data=05%7C01%7Chaoyu.song%40futurewei.com%7Cda9d36909a9a4a9ec2c108da1d898371%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637854773728966684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HQwcWaELit%2BFdy3jCFMwpnOX3tHret9Pp4WNaRlWRHE%3D&reserved=0>

---

From: Tianran Zhou <[email-masked]>
Date: Thu, 14 Apr 2022 01:50:45 +0000

Hi Greg,

Your statement is floating in the area.
I am confused. You asked me some questions on specific use cases.  I replied with my point with reason. But here you jumped to some other things, like a philosophy thing. What’s your thoughts on my reply on each of you question? If you disagree, what’s your proposal and how does it work better?
If you want to discuss PW, I would like to know firstly the use cases. Signaling or data traffic? How ISD is more flexible?

Best,
Tianran

Sent: Wednesday, April 13, 2022 10:44 PM

Hi Tianran,
thank you for sharing your thoughts. Of course, there's no way to slice the bread and whether to use only PSD comes with its limitations like burning a value for the first nibble after the BoS. And it needs to interwork with PW CW and ACH/G-ACH. I see an ISD-based solution as more flexible and easier to apply to MPLS PWs.

Regards,
Greg

On Wed, Apr 13, 2022 at 12:16 AM Tianran Zhou <[email-masked]<mailto:[email-masked]>> wrote:
Hi Greg,

Cheers,
Tianran

From: Greg Mirsky [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Wednesday, April 13, 2022 10:10 AM
To: Tianran Zhou <[email-masked]<mailto:[email-masked]>>
Cc: John E Drake <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Tianran,

Comments in line.

Tianran

Hi Tony,

Best,
Tianran

Hi Tianran,

Tony

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 13 Apr 2022 19:06:17 -0700

Hi Tianran,
I appreciate you've shared your point of view. I have offered mine. Simply
put it "It depends". And yes, I believe that we're still discussing the
MPLS data plane and the impact MNA may have. As for the philosophical
aspect, I believe that we'll find a balance between new capabilities and
the pain/cost inflicted through an open technical discussion.

Regards,
Greg

On Wed, Apr 13, 2022 at 6:50 PM Tianran Zhou <[email-masked]> wrote:

> I am confused. You asked me some questions on specific use cases.  I
> replied with my point with reason. But here you jumped to some other
> things, like a philosophy thing. What’s your thoughts on my reply on each
> of you question? If you disagree, what’s your proposal and how does it work
> better?
>
> If you want to discuss PW, I would like to know firstly the use cases.
> Signaling or data traffic? How ISD is more flexible?
>
>
>
> Best,
>
> Tianran
>
>
>
> *Sent:* Wednesday, April 13, 2022 10:44 PM
> thank you for sharing your thoughts. Of course, there's no way to slice
> the bread and whether to use only PSD comes with its limitations like
> burning a value for the first nibble after the BoS. And it needs to
> interwork with PW CW and ACH/G-ACH. I see an ISD-based solution as more
> flexible and easier to apply to MPLS PWs.
>
>
>
> Regards,
>
> Greg
>
>
>

---

From: Tony Li <[email-masked]>
Date: Wed, 13 Apr 2022 20:28:50 -0700

Hi Haoyu,

> On Apr 13, 2022, at 4:37 PM, Haoyu Song <[email-masked]> wrote:
> 

Well, I’m no hardware guy, but here’s the issues that I have with your presentation:

- You start off the an example of the Trident architecture. All well and good, but that’s one and only one data point. It’s always dangerous to extrapolate a solution from a single data point.  

- Other architectures simply provide the start of the packet to the network processor and map in some number of bytes. If you’re very lucky, some are mapped into registers already for you. The rest you have to issue reads for.  In the cases that I’ve dealt with, the cost of the computation is trivial, the time is spent doing reads. The cost of parsing is irrelevant: you just need to minimize the reads. You actually agree with this point on slide 12. ISD allows us to minimize the number of reads.

- You mention header chains. AFAIK, no one is proposing header chains.

- Similarly, you object to TLVs. No one is proposing TLVs. On this, we agree, this would be a poor design choice.

- Then you make this point:

	• If no PSD, and ISD is at ToS, and no need to scan other labels and payload, 
	  then there’s some gain on parsing speed, but negligible compared to the 
	  cost for actual header/data processing cost

I disagree with this completely.  If there is no PSD and only ISD, then we will only need to read to the end of ISD.  If the data were in PSD instead of being in ISD, then not only do we have to deal with the MNA label, but we have to parse down to the bottom of the label stack.  That’s a linear search through the stack, reading  every label.  This is where PSD is very, very, very painful.  BTW, this is true if ISD isn’t at the TOS either.  Any place other than the bottom of stack and ISD is better than PSD.

Regards,
Tony

---

From: Tianran Zhou <[email-masked]>
Date: Thu, 14 Apr 2022 06:10:52 +0000

Hi Robert,

I followed your discussions recently in the mailing list. I find your idea is very interesting.
I do not see your solution. But I think it might share the similar idea of our proposal on IOAM marking based direct export.
https://datatracker.ietf.org/doc/draft-song-ippm-postcard-based-telemetry/

If so, it could be very useful.

Best,
Tianran

From: mpls [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Thursday, April 14, 2022 4:33 AM
To: Haoyu Song <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

What is apparently being worked on here for months is in my view merely a CPR for MPLS.

Instead IMO (and that is why I brought up the control plane assisted model) MPLS could way exceed capabilities of SR NP without any excessive load on packets or major surgery to MPLS architecture by encoding reference to forwarding behaviour itself instead of just a pile atomic "actions" (irrespective if those are stuffed into ISD or PSD).

So yes - no draft no proposal - I get this. I will write one. When I get to it.

Kind regards,
Robert.

---

From: Tianran Zhou <[email-masked]>
Date: Thu, 14 Apr 2022 09:31:32 +0000

Hi Greg,

My evaluation is based on existing drafts.
I did not see the balance point from:
https://datatracker.ietf.org/doc/draft-kompella-mpls-mspl4fa/
which kept pushing the WG for adoption.

Best,
Tianran
Sent: Thursday, April 14, 2022 10:06 AM

Hi Tianran,
I appreciate you've shared your point of view. I have offered mine. Simply put it "It depends". And yes, I believe that we're still discussing the MPLS data plane and the impact MNA may have. As for the philosophical aspect, I believe that we'll find a balance between new capabilities and the pain/cost inflicted through an open technical discussion.

Regards,
Greg

On Wed, Apr 13, 2022 at 6:50 PM Tianran Zhou <[email-masked]<mailto:[email-masked]>> wrote:
Hi Greg,

Best,
Tianran

Hi Tianran,

Regards,
Greg

Cheers,
Tianran

Hi Tianran,

Comments in line.

Tianran

Hi Tony,

Best,
Tianran

Hi Tianran,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Thu, 14 Apr 2022 16:12:57 +0000

Hi Robert,

I’m glad we share the same view. I do believe we should try to keep the complexity in control plane as much as possible and meanwhile make the data plane as simple as possible. But I also realize that  this can be done only to certain extent until we can’t make it simpler. There are some use cases (e.g., slice ID, IOAM trace) which requires the user packets to carry some extra metadata. Chaining them after the MPLS label stack is the least intrusive and simplest way I can envision. In addition, keeping the data plane performance in mind, we should set a strict barrier on admitting use cases to this framework, and sometimes we may have to seek alternative methods. Look forward to your proposal.

Best regards,
Haoyu

From: Robert Raszuk <[email-masked]>
Sent: Wednesday, April 13, 2022 1:33 PM
To: Haoyu Song <[email-masked]>
Cc: John E Drake <[email-masked]>; Kireeti Kompella <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

Kind regards,
Robert.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2022 18:33:58 +0200

Hi Haoyu,

> There are some use cases (e.g., slice ID, IOAM trace) which requires the
user packets to carry some extra metadata.

I really do not think that examples you provide would require metadata in
the data plane.

Take - slide ID - slices in my book are build end to end and there is no
need for each node to make an independent decision. Flows are mapped to
slice in ingress and travel through that slice to the egress. There are
number of ways to construct such slices today. That said if there is some
advantage for a set of flows to be recognized and routed hop by hop
forwarding behaviour identifier will be sufficient to make such hop by hop
recognition and correct hop by hop forwarding.

Likewise IOAM trace postcard can also include any amount of information
propagated to those networks elements which are to iOAM act on packets and
such information can be again fetched by data plane from control plane (or
pushed by control plane to data plane) based on the forwarding behaviour
id. Not to mention that the reply itself can contain the entire mpls label
stack and even bytes beyond that.

I am yet to see a case where the proposed forwarding behaviour ID (FBI)
fails to map to control plane distributed actions, metadata etc .. which in
turn will require to stuff such data to each and every packet. Note we are
not talking lab here ... we are talking about forwarding efficiency of TB/s
trunks full of data.  As you said in those cases every bit counts. In fact
I hope most of those would be optical lambdas switched without going back
to electrical at transit hops) in the not so distant future.

Regards,
R.

On Thu, 14 Apr 2022 at 18:13, Haoyu Song <[email-masked]> wrote:

> Hi Robert,
>
>
>
> I’m glad we share the same view. I do believe we should try to keep the
> complexity in control plane as much as possible and meanwhile make the data
> plane as simple as possible. But I also realize that  this can be done only
> to certain extent until we can’t make it simpler. There are some use cases
> (e.g., slice ID, IOAM trace) which requires the user packets to carry some
> extra metadata. Chaining them after the MPLS label stack is the least
> intrusive and simplest way I can envision. In addition, keeping the data
> plane performance in mind, we should set a strict barrier on admitting use
> cases to this framework, and sometimes we may have to seek alternative
> methods. Look forward to your proposal.
>
>
>
> Best regards,
>
> Haoyu
>
>
>
> *From:* Robert Raszuk <[email-masked]>
> *Sent:* Wednesday, April 13, 2022 1:33 PM
> *To:* Haoyu Song <[email-masked]>
> *Cc:* John E Drake <[email-masked]>; Kireeti Kompella <
> [email-masked]>; [email-masked]
> Instead IMO (and that is why I brought up the control plane assisted
> model) MPLS could way exceed capabilities of SR NP without any excessive
> load on packets or major surgery to MPLS architecture by encoding
> reference to forwarding behaviour itself instead of just a pile atomic
> "actions" (irrespective if those are stuffed into ISD or PSD).
>
>
>

---

From: Haoyu Song <[email-masked]>
Date: Thu, 14 Apr 2022 18:01:57 +0000

Hi Robert,

What you said is true. It’s almost always possible to find control plane alternatives for the same problem.
But then should we discard the data plane-based solution altogether? Here’s some of my thoughts.

Some in-network header and metadata has been standardize or is in the process of standardization and there are also many ongoing works proposing to apply them in different types of networks, which means hardware will need to support these functions if these standards are actually to be adopted. If so, it would be nice to have MPLS-based solution accordingly for these functions (by then the only new work in data plane would be the new header parsing)
Further, this is even required if MPLS only covers a section of the network path while the other parts of the network running different protocols apply the new data plane functions, and the applications require E2E coverage.  In this case, MPLS needs to participate in using the similar data plane header and metadata.

My second point is that I find it’s hard for a control plane solution and its counterpart in data plane to maintain exactly the same semantics. I’m not familiar with Slice ID so I won’t comment on it. For IOAM, there are multiple possible options available, each with its own pros and cons. It’s better to let operators choose which option best suits their needs. For another example, MPLS SFC may need to pass some metadata between different SFs. This can only be done on a per-packet basis. I don’t see how control plane can do it.

I’m familiar with the capability of the state-of-the-art forwarding chips. Some tasks are challenging but some are doable or even preferred to be done in data plane. We need to evaluate the tradeoffs carefully case by case.

Best regards,
Haoyu

From: Robert Raszuk <[email-masked]>
Sent: Thursday, April 14, 2022 9:34 AM
To: Haoyu Song <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

> There are some use cases (e.g., slice ID, IOAM trace) which requires the user packets to carry some extra metadata.

I really do not think that examples you provide would require metadata in the data plane.

Take - slide ID - slices in my book are build end to end and there is no need for each node to make an independent decision. Flows are mapped to slice in ingress and travel through that slice to the egress. There are number of ways to construct such slices today. That said if there is some advantage for a set of flows to be recognized and routed hop by hop  forwarding behaviour identifier will be sufficient to make such hop by hop recognition and correct hop by hop forwarding.

Likewise IOAM trace postcard can also include any amount of information propagated to those networks elements which are to iOAM act on packets and such information can be again fetched by data plane from control plane (or pushed by control plane to data plane) based on the forwarding behaviour id. Not to mention that the reply itself can contain the entire mpls label stack and even bytes beyond that.

I am yet to see a case where the proposed forwarding behaviour ID (FBI) fails to map to control plane distributed actions, metadata etc .. which in turn will require to stuff such data to each and every packet. Note we are not talking lab here ... we are talking about forwarding efficiency of TB/s trunks full of data.  As you said in those cases every bit counts. In fact I hope most of those would be optical lambdas switched without going back to electrical at transit hops) in the not so distant future.

Regards,
R.

On Thu, 14 Apr 2022 at 18:13, Haoyu Song <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert,

Best regards,
Haoyu

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, April 13, 2022 1:33 PM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>
Cc: John E Drake <[email-masked]<mailto:[email-masked]>>; Kireeti Kompella <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

Kind regards,
Robert.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2022 20:43:13 +0200

Hi Haoyu,

> Here’s some of my thoughts.

Well I am not sure what is your definition of discard above. In all cases
the line rate processing will better to happen in data plane. But dry
actions how to process given packet do not need to be part of every single
one of them. Control plane should provide that information.

And for 100% clarity control plane can be configuration, controller or
dynamic pub-sub protocol driven.

> Some in-network header and metadata has been standardize or is in the
> process of standardization and there are also many ongoing works proposing
> to apply them in different types of networks, which means hardware will
> need to support these functions if these standards are actually to be
> adopted. If so, it would be nice to have MPLS-based solution accordingly
> for these functions (by then the only new work in data plane would be the
> new header parsing)
>

Well from what I have learned in the past we are never done with
extensibility. So what is defined in one draft/rfc today will likely be
extended tomorrow. If we take approach of explicit signalling in the data
plane networks will have a hard time to keep up with it. That is why
indirection helps.

And unlike Kireeti's crystal ball which is temporarily kaput - I never had
one - so I would not even attempt to predict the future. I would try to
prepare for it the best way we can.

> Further, this is even required if MPLS only covers a section of the
> network path while the other parts of the network running different
> protocols apply the new data plane functions, and the applications require
> E2E coverage.  In this case, MPLS needs to participate in using the similar
> data plane header and metadata.
>

Well let's not forget that MPLS runs on IP networks. There is no pure MPLS
only networks. So if there is need for end to end solution and the "other
part " of the network is not MPLS capable I would rather not even try to
run it with MPLS to start with.

>  My second point is that I find it’s hard for a control plane solution and
> its counterpart in data plane to maintain exactly the same semantics. I’m
> not familiar with Slice ID so I won’t comment on it. For IOAM, there are
> multiple possible options available, each with its own pros and cons. It’s
> better to let operators choose which option best suits their needs. For
> another example, MPLS SFC may need to pass some metadata between different
> SFs. This can only be done on a per-packet basis. I don’t see how control
> plane can do it.
>

Ok - but I do not see that MPLS transport is specifically capable of
carrying such opaque data as part of MPLS stack. Those should be carried
outside of it if needed. And by outside I mean behind lable stack or in the
front of the lable stack (in the front means by additional encap between SF
nodes).

Best,
Robert

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 15 Apr 2022 02:02:10 +0000

It's still far from complete. A lot of concerns about ISD have not been addressed.
You occasionally ignore my reply. Let's firstly see my comments on your reply.

T.

From: John E Drake [mailto:jdrake=[email-masked]]
Sent: Wednesday, April 13, 2022 9:28 PM
To: Tianran Zhou <[email-masked]>; Tony Li <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Yours Irrespectively,

John

Juniper Business Use Only
From: Tianran Zhou <zhoutianran=[email-masked]<mailto:zhoutianran=[email-masked]>>
Sent: Tuesday, April 12, 2022 9:17 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: [mpls] Concerns about ISD

Hi,

Comments in line.

Tianran

Hi Tony,

ZTR2> So you point out a new RFC8662, which is not Tony pointed. You kept taking EL for example. However, EL is a very simple one. It's still a label and we can use very simple way to implement.
But the ISD in https://datatracker.ietf.org/doc/draft-kompella-mpls-mspl4fa/ is not that case.
You simply assume all ISD could be processed. If we look into it deeply, there are a lot of questions regarding to ISD.
What's the MSD capability you considered(5 or 6 or so)? What's the use case from the use case document you want to use ISD?
How many data you are going to put into the ISD without exceeding the MSD? What if the ISD exceed the MSD?
For an end to end path as indicated in the SR stack, if you do not put the data into PSD, you need to repeat put ISD in the stack multi times. How many times you would expect?
In this case at least PSD is necessary so that the end node know what to do.
If at least two ISDs put in the stack, this will occupy the already constrained MSD. This will also at least impact the egress node.
With so many issues, together with those I raised in the my first mail in this thread, I can only conclude the ISD is a fancy design but really useless, even harmful.

ZTR2> You simply assume the ISD could be processed. But you do not well consider the scenario and the whole solution. And you cannot guarantee the ISD will not exceed the MSD.
As you mentioned, if ISD exceed the MSD, you also asserted, below, that the problem would be solved by not imposing ISD.

ZTR2> I know a set of new terms are introduced in the new framework draft when Tony joined. What's the difference to the terms already in MAID? What's the value/benefit to introduce and use the new terms? Does these all really discussed? Are there any consensus in the minute? I do not actually care about any terms only if I can understand. But do not consider creating new terms is the contribution to the community. They cannot really solve problem.

[JD2] What I read is that you don't like in-stack data and an assertion, contrary to RFC 8662, that either all nodes can handle ancillary post-stack data or that if they can't, ancillary post-stack data won't be included
ZTR2> You said "Tony has been making for the past few email iterations". But you still not answer my question. Please make any statement based on the fact.
And let me clarify. I support PSD, and hope PSD would be included. I just oppose to ISD.

Best,
Tianran

Hi Tianran,

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 15 Apr 2022 13:24:32 +0000

Hi,

You've made it clear that you don't like in-stack ancillary data, despite the many attempts to explain to you why it's necessary in
some cases.

I did a check of current proposed network actions  and found the following:  resource partition, E2E Loss Measurement, E2E OAM,
HBH OAM, entropy, no reroute, flow ID, 5G slice ID, traffic accounting, bounded delay, and L2 Frame Reassembly.  Of those, I think
resource partition, entropy, no-reroute, flow ID, traffic accounting, and bounded delay are candidates for having in-stack ancillary
data.  Detnet may also have some use for it.

The questions you ask in your email, below, are to be answered by any MNA solution draft.  A draft defining a given network action
will need to specify, among other things, whether that network action requires ancillary data, what is that ancillary data, and if it is
in-stack or post-stack.

Yours Irrespectively,

John

Sent: Thursday, April 14, 2022 10:02 PM

T.

To: Tianran Zhou <[email-masked]<mailto:[email-masked]>>; Tony Li <[email-masked]<mailto:[email-masked]>>

Yours Irrespectively,

John

Hi,

Comments in line.

Tianran

Hi Tony,

But the ISD in https://datatracker.ietf.org/doc/draft-kompella-mpls-mspl4fa/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-kompella-mpls-mspl4fa/__;!!NEt6yMaO-gk!VvyxGcx5k7uU0lELz7nl2Q5YfQhBAbZhdLOM4dHqfWUFDq6bor50CE9BKmM1l2Q$> is not that case.

Best,
Tianran

Hi Tianran,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 17:30:47 +0000

Hi John,

You said ISD is "necessary" but my opinion is quite opposite. I think it is unnecessary and even harmful. For all the use cases you mentioned, I don't see why they can't be realized in PSD if they ever need any data. And as Robert pointed out, not all these use cases are suitable for data plane implementations. They need to be examined one by one with detailed requirement and cost analysis.

ISD is a significant departure from the current MPLS architecture and mechanisms. It's basically a redesign of MPLS. It encodes new headers in "MPLS labels". It unnecessarily limits new header's size and format in order to fit the new header under the constraints of MPLS label format. It worsens both the parsing cost and performance by multiple times. Given so many negative effects, I see no point to support ISD.

The extension to MPLS should take the least intrusive path and conform to the established methods. If there are hierarchical tunnels in the MPLS label stack, it can be partitioned to multiple stacks (i.e., a stack of stack) and each stack can have its own PSD following it immediately. Without needing to invent new label handling method, this approach can mitigate the parsing depth concern.

Best regards,
Haoyu

From: mpls <[email-masked]> On Behalf Of John E Drake
Sent: Friday, April 15, 2022 6:25 AM
To: Tianran Zhou <zhoutianran=[email-masked]>; Tony Li <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi,

Yours Irrespectively,

John

T.

Yours Irrespectively,

John

Hi,

Comments in line.

Tianran

Hi Tony,

But the ISD in https://datatracker.ietf.org/doc/draft-kompella-mpls-mspl4fa/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-kompella-mpls-mspl4fa%2F__%3B!!NEt6yMaO-gk!VvyxGcx5k7uU0lELz7nl2Q5YfQhBAbZhdLOM4dHqfWUFDq6bor50CE9BKmM1l2Q%24&data=05%7C01%7Chaoyu.song%40futurewei.com%7C692d6fe4a9c943f3ff9908da1ee3571f%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856259046895391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR4645I%2BQsZmKLYU36GvCq0w5INacaGCx0iIAw4ngZ0%3D&reserved=0> is not that case.

Best,
Tianran

Hi Tianran,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 17:42:15 +0000

Hi Robert,

Ok - but I do not see that MPLS transport is specifically capable of carrying such opaque data as part of MPLS stack. Those should be carried outside of it if needed. And by outside I mean behind lable stack or in the front of the lable stack (in the front means by additional encap between SF nodes).

Totally agree. And this is exactly reflected in our extension header draft https://datatracker.ietf.org/doc/draft-song-mpls-extension-header/ in which the data are chained behind the stack.  I believe this is the least intrusive and the simplest method to support the extensions. Look forward to your suggestions and comments on it.

Best regards,
Haoyu

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Apr 2022 10:44:07 -0700

Hi Haoyu,

> You said ISD is “necessary” but my opinion is quite opposite. I think it is unnecessary and even harmful. For all the use cases you mentioned, I don’t see why they can’t be realized in PSD if they ever need any data. And as Robert pointed out, not all these use cases are suitable for data plane implementations. They need to be examined one by one with detailed requirement and cost analysis.

Well, you’re welcome to your opinion.  Not everyone shares it. In fact, some of us think the exact opposite.  Putting data in PSD unnecessarily is a major performance hit.

Continually repeating yourself is not in the least bit convincing and does not move the discussion forward.

> ISD is a significant departure from the current MPLS architecture and mechanisms. It’s basically a redesign of MPLS. It encodes new headers in “MPLS labels”. It unnecessarily limits new header’s size and format in order to fit the new header under the constraints of MPLS label format. It worsens both the parsing cost and performance by multiple times. Given so many negative effects, I see no point to support ISD.

As we’ve noted repeatedly, yes, we are adding functionality to MPLS. New functionality requires change.  

> The extension to MPLS should take the least intrusive path and conform to the established methods. If there are hierarchical tunnels in the MPLS label stack, it can be partitioned to multiple stack…

I know of no existing case where there is PSD in a hierarchical label stack, so what you’re proposing here does not conform to ‘established methods’ and it’s effectively recreating ISD.

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 15 Apr 2022 17:47:34 +0000

Tony has succinctly captured what should be blindly obvious to the informed reader.

Sent from my iPhone

On Apr 15, 2022, at 1:44 PM, Tony Li <[email-masked]> wrote:

Hi Haoyu,

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 15 Apr 2022 17:54:27 +0000

Oopsie, ‘blindingly’

Sent from my iPhone

On Apr 15, 2022, at 1:48 PM, John E Drake <jdrake=[email-masked]> wrote:

Sent from my iPhone

Hi Haoyu,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 17:55:00 +0000

Hi Tony,

Please take a look at our EH draft if you haven’t yet. The mechanism is there and the essence is that it doesn’t tamper the MPLS label stack and label semantics. That’s why I say PSD is least intrusive.

Best regards,
Haoyu

From: Tony Li <[email-masked]> On Behalf Of Tony Li
Sent: Friday, April 15, 2022 10:44 AM
To: Haoyu Song <[email-masked]>
Cc: John E Drake <jdrake=[email-masked]>; Tianran Zhou <zhoutianran=[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

Tony

---

From: John E Drake <[email-masked]>
Date: Fri, 15 Apr 2022 17:59:43 +0000

Post-stack data between successive MPLS label stacks is not intrusive?  The performance impacts are truly impressive.

Sent from my iPhone

On Apr 15, 2022, at 1:55 PM, Haoyu Song <[email-masked]> wrote:

Hi Tony,

Best regards,
Haoyu

Hi Haoyu,

Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Apr 2022 11:08:42 -0700

Hi Haoyu,

> Please take a look at our EH draft if you haven’t yet. The mechanism is there and the essence is that it doesn’t tamper the MPLS label stack and label semantics. That’s why I say PSD is least intru…

Thank you, I was unaware of it and have now looked at it.  If I understand it correctly, the EHI is the trigger to look for extension headers, all of which are in PSD.  Again, the parsing performance is a linear read to find the bottom of stack.

While it may be minimally intrusive to the label stack semantics, that doesn’t seem like a good trade-off for the performance penalty.

On a separate point, I have deep concerns about the encoding efficiency of the extension headers.  We can talk about this if and when it becomes relevant.

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 18:14:10 +0000

“Post-stack data between successive MPLS label stacks is not intrusive?  The performance impacts are truly impressive.”
Hi John,

  1.  I mean it’s less intrusive than ISD
  2.  I have analyzed the performance and cost and it’s documented. I expect you or others can provide other inputs. (Yes, it’s indeed impressive based on the evaluation we have so far)
Best regards,
Haoyu

---

From: John E Drake <[email-masked]>
Date: Fri, 15 Apr 2022 18:16:20 +0000

The last time this was discussed, Haoyu indicated that it was a requirement that the EHI must be the only SPL in the label stack, which seems a bit restrictive.

Sent from my iPhone

On Apr 15, 2022, at 2:08 PM, Tony Li <[email-masked]> wrote:

Hi Haoyu,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 18:21:02 +0000

Hi John,

Please read our EHI  draft. https://datatracker.ietf.org/doc/html/draft-song-mpls-eh-indicator-04
While we prefer to use a SPL to indicate EH among all the options , we never said “only”.

Best regards,
Haoyu

From: John E Drake <[email-masked]>
Sent: Friday, April 15, 2022 11:16 AM
To: Tony Li <[email-masked]>
Cc: Haoyu Song <[email-masked]>; Tianran Zhou <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

On Apr 15, 2022, at 2:08 PM, Tony Li <[email-masked]<mailto:[email-masked]>> wrote:

Hi Haoyu,

Tony

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 15 Apr 2022 20:22:55 +0200

Hi Haoyu,

Thank you for the link to your and other industry co-authors' draft.

I must say that EHI can be really optional (ie. not needed). Reason being
is that the label value itself delivering the packet to a given node can
indicate the presence of some metadata in it and instructing the data plane
to look deeper in the stack to fetch it.

But looking at the proposed EH encoding as well as looking at MNA two major
questions pop up:

* Is such a fundamental change to MPLS architecture really necessary ?

* What are the clear technical benefits to go through development and
deployment of all of this (data plane action stacks with associated
metadata) instead of focusing effort on using SRv6 where it is all there
already ?

Wouldn't we all agree that for data plane network programming there is
already solution not only defined but also deployed ?

Cheers,
Robert

On Fri, 15 Apr 2022 at 19:42, Haoyu Song <[email-masked]> wrote:

> Hi Robert,
>
>
>
>
>
> *Ok - but I do not see that MPLS transport is specifically capable of
> nodes). *
> Totally agree. And this is exactly reflected in our extension header draft
> https://datatracker.ietf.org/doc/draft-song-mpls-extension-header/ in
> which the data are chained behind the stack.  I believe this is the least
> intrusive and the simplest method to support the extensions. Look forward
> to your suggestions and comments on it.
>
>
>
> Best regards,
>
> Haoyu
>

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 18:25:51 +0000

Thanks, Tony,

If you continue to read the indicator draft, you’ll find it may be possible to reach the EHs faster.
Even we don’t use that mechanism, the parsing latency is indeed linear to the label stack depth. But if you check the cost of parsing ISD, this can still be a better choice. Moreover, the stack of stack can help to reduce the label stack depth to reach the EHs.

For the EH encoding efficiency, I appreciate your input. Currently it follows the IPv6 EH type (i.e., using NH + LENGTH to delineate every EH). There could certainly be room for improvement.

Best regards,
Haoyu

Sent: Friday, April 15, 2022 11:09 AM

Hi Haoyu,

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 18:34:57 +0000

Hi Robert,

For the EHI, our draft https://datatracker.ietf.org/doc/html/draft-song-mpls-eh-indicator-04  lists an option of using control plane to configure FEC labels as indicator. Not sure if this is what you mean. If you think this is the best option, please discuss it further in the DT.

The other big questions are better left to the community to decide.

Best regards
Haoyu

From: Robert Raszuk <[email-masked]>
Sent: Friday, April 15, 2022 11:23 AM
To: Haoyu Song <[email-masked]>
Cc: [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

I must say that EHI can be really optional (ie. not needed). Reason being is that the label value itself delivering the packet to a given node can indicate the presence of some metadata in it and instructing the data plane to look deeper in the stack to fetch it.

But looking at the proposed EH encoding as well as looking at MNA two major questions pop up:

* What are the clear technical benefits to go through development and deployment of all of this (data plane action stacks with associated metadata) instead of focusing effort on using SRv6 where it is all there already ?

Wouldn't we all agree that for data plane network programming there is already solution not only defined but also deployed ?

Cheers,
Robert

On Fri, 15 Apr 2022 at 19:42, Haoyu Song <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert,

Totally agree. And this is exactly reflected in our extension header draft https://datatracker.ietf.org/doc/draft-song-mpls-extension-header/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-song-mpls-extension-header%2F&data=05%7C01%7Chaoyu.song%40futurewei.com%7C1a36ae5b562343beaa1c08da1f0ccb9a%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856437064103356%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0aJSKP136lOajuG3Rn6fi%2B70T4dta%2BFgtleqZIJdRng%3D&reserved=0> in which the data are chained behind the stack.  I believe this is the least intrusive and the simplest method to support the extensions. Look forward to your suggestions and comments on it.

Best regards,
Haoyu

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Apr 2022 11:56:09 -0700

Hi Haoyu,

> If you continue to read the indicator draft, you’ll find it may be possible to reach the EHs faster.

I’m sorry, I’m not sure which document you’re referring to.

Yes, I realize that you could store a length of the label stack in the EHI LSE.  That would allow you to index directly to PSD rather than linear parsing for the bottom of stack.  However, that’s not the established MPLS mechanism. :-)  Moreover, it causes a redundancy and inconsistency problem: both the length and the bottom of stack must been coordinated.  What happens if they are inconsistent? Is an implementation supposed to cross-check them? If so, we’re back to linear search for the S bit. And the redundancy seems wasteful of bits.

> Even we don’t use that mechanism, the parsing latency is indeed linear to the label stack depth.

Thank you.  

> But if you check the cost of parsing ISD, this can still be a better choice.

Please explain that.  As previously discussed, parsing the full stack to find the bottom of the stack requires more read/load operations. That would seem to mandate poorer performance.

> Moreover, the stack of stack can help to reduce the label stack depth to reach the EHs.

I’m not following that.  It seems to me that if there is a hierarchical stack, then for an implementation to support all of the network actions, it has to parse all of the hierarchy plus all of the extension headers.

> For the EH encoding efficiency, I appreciate your input. Currently it follows the IPv6 EH type (i.e., using NH + LENGTH to delineate every EH). There could certainly be room for improvement.

Ok.  If I understand your proposal correctly, there are 4 octets of overhead (HEH) at the start. Then, for each network action, there would seem to be at least 3 octets of overhead, plus any associated data, plus alignment.

Let’s suppose that we want to encode NFFRR, entropy, and GISS in one packet.  By my math, the cost is:

EHI: 4 octets
HEH: 4 octets
NFFRR: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 28 octets

Do I have that right?

Tony

---

From: Haoyu Song <[email-masked]>
Date: Fri, 15 Apr 2022 19:29:57 +0000

Hi Tony, please see my inline response. Thanks.

Haoyu

Sent: Friday, April 15, 2022 11:56 AM

Hi Haoyu,

[HS] Sorry, I forget to include the link. Here it is https://datatracker.ietf.org/doc/html/draft-song-mpls-eh-indicator-04.   Yes, this is some special processing limited only to the SPL. We include such possibility in the document for community discussion. It doesn’t mean we fully support it.

Thank you.

[HS] Assuming the label stack depth is L,  it needs L steps to reach the EH.  The question here is: what’s the reasonable value of L?   The total parsing cost and latency also need to include the parsing of the AD items. If we include this into consideration, we would see PSD would still win overall. Please see pg. 7 of the slides for the analysis https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1
Moreover, in another draft https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt we describe a way to avoid unnecessary scan of the label stack with the help from the control plane.

[HS] Yes you are right. Then this optimization works for the case where label stack sections can be spliced (e.g., in normal processing, one never needs to scan the entire label stack). For the hierarchical case, such arrangement may help to scope the EHs, but it doesn’t help the reachability.

EHI: 4 octets
HEH: 4 octets
NFFRR: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 28 octets

Do I have that right?

[HS] It’s right if you think 8 octets are needed for Entropy and GISS (BTW, I wonder why we want to redefine entropy. We have already have a standard for it and we don’t need to do anything more about it ).

Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Apr 2022 13:04:16 -0700

Hi Haoyu,

> [HS] Assuming the label stack depth is L,  it needs L steps to reach the EH.  The question here is: what’s the reasonable value of L?   The total parsing cost and latency also need to include the parsing of the AD items. If we include this into consideration, we would see PSD would still win overall. Please see pg. 7 of the slides for the analysis https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1 <https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1>

You’re assuming a TCAM and your only metric here seems to be the storage cost.  Many folks don’t do parsing using a TCAM and the performance cost is in the reads, not in the storage.

> Moreover, in another draft https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt <https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt> we describe a way to avoid unnecessary scan of the label stack with the help from the control plane.

If you’re going to involve the control plane, then it would seem like Robert’s proposal (or what I understand of it), would imply that you push EVERYTHING into the control plane.  Why bother with data plane encodings at all?

> [HS] It’s right if you think 8 octets are needed for Entropy and GISS

If I understand your proposal, and we want >8 bits for both of these fields, then each would require 3 octets of overhead. Storing the EL/GIS value might take 2 or 3 octets.  Alignment would take you to 8.

If I look at the same thing with the FAI draft, I get:

FAI: 4 octets
NFFRR: included in the above, so 0 octets
Entropy: 4 octets (30 bits, plus overhead)
GISS: 4 octets (30 bits, plus overhead)

Total: 12 octets

> (BTW, I wonder why we want to redefine entropy. We have already have a standard for it and we don’t need to do anything more about it ).

It’s true that we have a standard for it. It’s fairly common, so then the interesting question is what happens when it is present in the label stack along with MNA?  If we do not incorporate EL into MNA, then an implementation has to deal with both MNA and EL independently.  What order are they in? How do they interact? If we use the standard ELI/EL encoding, that’s 8 octets just for entropy.  OTOH, if we incorporate EL into MNA, then when using both, we can collapse the encoding.  For example, in the example above, entropy is 4 octets, resulting in a savings of 4 octets.  Half price! And that’s assuming that you don’t use the smaller entropy/GISS encodings which would give an even greater savings.

Tony

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 15 Apr 2022 22:46:48 +0200

Hi Tony,

> If you’re going to involve the control plane, then it would seem like
Robert’s proposal (or what I understand of it),
> would imply that you push EVERYTHING into the control plane.  Why bother
with data plane encodings at all?

I know You know it but just for the record to the list in my proposal data
plane still needs to contain a fixed 4+4=8 octet reference.

4 octets fora special reserved *single* label and next 4 to contain the
actual reference ID to any special handling. Almost identical to today's
ELI/EL encoding.

That tuple would be always above first top most label on the stack.

And either all nodes would inspect it or only those which based on the top
most label can tell if there is something to inspect or not.

It is essentially an idea stolen from pretty common use of pointers to
memory :)  The actions and metadata distributed outside of data plane would
be a tuple: destination_node+referenece+action_data/metadata/parms etc ...

Happy Easter !
Robert

On Fri, 15 Apr 2022 at 22:04, Tony Li <[email-masked]> wrote:

> Please explain that.  As previously discussed, parsing the full stack to
> find the bottom of the stack requires more read/load operations. That would
> seem to mandate poorer performance.
>
> *[HS] Assuming the label stack depth is L,  it needs L steps to reach the
> EH.  The question here is: what’s the reasonable value of L?   The total
> parsing cost and latency also need to include the parsing of the AD items.
> If we include this into consideration, we would see PSD would still win
> overall. Please see pg. 7 of the slides for the analysis **https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1
> <https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1>*
>
>
>
> You’re assuming a TCAM and your only metric here seems to be the storage
> cost.  Many folks don’t do parsing using a TCAM and the performance cost is
> in the reads, not in the storage.
>
>
> *Moreover, in another
> draft https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt
> <https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt> we
> describe a way to avoid unnecessary scan of the label stack with the help
> from the control plane.*
>
>
>
> If you’re going to involve the control plane, then it would seem like
> Robert’s proposal (or what I understand of it), would imply that you push
> EVERYTHING into the control plane.  Why bother with data plane encodings at
> all?
>
>
> For the EH encoding efficiency, I appreciate your input. Currently it
> follows the IPv6 EH type (i.e., using NH + LENGTH to delineate every EH).
> There could certainly be room for improvement.
>
>
>
> Ok.  If I understand your proposal correctly, there are 4 octets of
> overhead (HEH) at the start. Then, for each network action, there would
> seem to be at least 3 octets of overhead, plus any associated data, plus
> alignment.
>
> *[HS] It’s right if you think 8 octets are needed for Entropy and GISS *
>
>
>
> If I understand your proposal, and we want >8 bits for both of these
> fields, then each would require 3 octets of overhead. Storing the EL/GIS
> value might take 2 or 3 octets.  Alignment would take you to 8.
>
> *(BTW, I wonder why we want to redefine entropy. We have already have a
> standard for it and we don’t need to do anything more about it ).*
>
>
>
> It’s true that we have a standard for it. It’s fairly common, so then the
> interesting question is what happens when it is present in the label stack
> along with MNA?  If we do not incorporate EL into MNA, then an
> implementation has to deal with both MNA and EL independently.  What order
> are they in? How do they interact? If we use the standard ELI/EL encoding,
> that’s 8 octets just for entropy.  OTOH, if we incorporate EL into MNA,
> then when using both, we can collapse the encoding.  For example, in the
> example above, entropy is 4 octets, resulting in a savings of 4 octets.
> Half price! And that’s assuming that you don’t use the smaller entropy/GISS
> encodings which would give an even greater savings.
>
> Tony
>
>

---

From: Haoyu Song <[email-masked]>
Date: Sat, 16 Apr 2022 00:26:40 +0000

Hi Tony,  please see inline comments with [HS2]

Best regards,
Haoyu

Sent: Friday, April 15, 2022 1:04 PM

Hi Haoyu,

[HS] Assuming the label stack depth is L,  it needs L steps to reach the EH.  The question here is: what's the reasonable value of L?   The total parsing cost and latency also need to include the parsing of the AD items. If we include this into consideration, we would see PSD would still win overall. Please see pg. 7 of the slides for the analysis https://trac.ietf.org/trac/mpls/wiki/2022-01-27-agenda#no1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrac.ietf.org%2Ftrac%2Fmpls%2Fwiki%2F2022-01-27-agenda%23no1&data=05%7C01%7Chaoyu.song%40futurewei.com%7C0f1c419a6fba45b48e2908da1f1b1ff2%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856498622812126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n9vdFVxMQS2gVOBgD9DMhSO5%2FKLAgSv2L%2FHINECufaw%3D&reserved=0>

You're assuming a TCAM and your only metric here seems to be the storage cost.  Many folks don't do parsing using a TCAM and the performance cost is in the reads, not in the storage.

[HS2] TCAM is actually the better choice to store states involving bitmaps. Otherwise, parsing bitmap requires extra calculations in addition to storage, making it even less favorable.

Moreover, in another draft https://www.potaroo.net/ietf/all-ids/draft-andersson-mpls-eh-label-stack-operations-03.txt<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.potaroo.net%2Fietf%2Fall-ids%2Fdraft-andersson-mpls-eh-label-stack-operations-03.txt&data=05%7C01%7Chaoyu.song%40futurewei.com%7C0f1c419a6fba45b48e2908da1f1b1ff2%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637856498622812126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GKad3KmRVlaZmhNeDXCxKth5FfKlgYDkfK%2FqtYFI%2BIo%3D&reserved=0> we describe a way to avoid unnecessary scan of the label stack with the help from the control plane.

If you're going to involve the control plane, then it would seem like Robert's proposal (or what I understand of it), would imply that you push EVERYTHING into the control plane.  Why bother with data plane encodings at all?

[HS2] We propose to use explicit FEC to tell there's no EHs in the packet to avoid the label stack scanning in such cases. This doesn't nullify the need to carry extra metadata for some user packets to support some in-network functions.

Let's suppose that we want to encode NFFRR, entropy, and GISS in one packet.  By my math, the cost is:

EHI: 4 octets
HEH: 4 octets
NFFRR: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 28 octets

Do I have that right?

[HS] It's right if you think 8 octets are needed for Entropy and GISS

FAI: 4 octets

Total: 12 octets

[HS2] Header overhead is an important dimension for comparison.  To make it apple to apple, let's first clarify several points.

  1.  If an action doesn't need any extra data, there's no point to allocate an EH for it. A flag in EHI is sufficient (this applies to the NFFRR case).
  2.  You are using the bitmap encoding since you mention FAI. Bitmap is more compact than Type/Length for sure. Here the implication is that each ISD must be 4 octets long, otherwise it will need every node to understand the size of every data item, further complicating the design.
  3.  For extensibility, how many ISD data items is planned to be supported? If it exceeds the one label capacity, you will need to extend it. So the actual overhead of FAI could be 8 octets instead of 4.
Let's temporarily put all these details aside and assume the minimum sized FAI. Basically, EH has a fixed 4 byte overhead for HEH. For each use case that does need metadata, EH will add 4 more byte overhead. This is the header overhead comparison. (In your example, it's 24 bytes PSD vs. 12 bytes ISD)
Now let's look at the other dimensions.

  1.  We know some use cases requiring data too big to fit in stack, so PSD is needed anyway. PSD can be used for the use cases with smaller metadata size, although it's relatively inefficient as discussed above. So essentially we can have just one mechanism all everything but to support ISD we end up with two.
  2.  Bitmap, albeit succinct, is inflexible and less extensible. The semantics and order are fixed at the design time; the total size of each data item must be equal to a label size to make bitmap useful in finding the corresponding data item. Use cases identified in the future can only use unused bitmap bits, regardless of its importance. If a use case requires to extend its data size, it's out of luck; if some common use cases also apply to the other types of networks (especially for those still under investigation), the use cases can be seriously limited by such size constraint which hampers design sharing and interoperability.
  3.  Parsing based on bitmap significantly bloats either the parser size or the parsing latency or both (as shown in my analysis). People can write pseudocode by themselves to verify. This is bad for both software and hardware data plane implementation. No wonder bitmap is never used for header encoding before (it's sometimes used in a header for its sub-structure, which is related to the header processing but not the header parsing).
  4.  The encoding of each ISD item itself is awkward. You have 30 bits and only 30 bits at your disposal with a BoS bit in between. It creates a hole in data, which is bad for both software and hardware.

(BTW, I wonder why we want to redefine entropy. We have already have a standard for it and we don't need to do anything more about it ).

It's true that we have a standard for it. It's fairly common, so then the interesting question is what happens when it is present in the label stack along with MNA?  If we do not incorporate EL into MNA, then an implementation has to deal with both MNA and EL independently.  What order are they in? How do they interact? If we use the standard ELI/EL encoding, that's 8 octets just for entropy.  OTOH, if we incorporate EL into MNA, then when using both, we can collapse the encoding.  For example, in the example above, entropy is 4 octets, resulting in a savings of 4 octets.  Half price! And that's assuming that you don't use the smaller entropy/GISS encodings which would give an even greater savings.

[HS2] I think the new design won't nullify the EL/ELI standard so they need to coexist in an incremental deployment scenario given EL/ELI might have been realized (It's possible to provide yet another entropy solution using the new mechanism but the necessity is subject to further discussion IMO). But they don't need to interact with each other because each has its clear function and operational procedure.

Tony

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Apr 2022 18:21:36 -0700

Hi Haoyu,

> [HS2] TCAM is actually the better choice to store states involving bitmaps. Otherwise, parsing bitmap requires extra calculations in addition to storage, making it even less favorable. 

Ok, I disagree. I would MUCH rather have normal ALU functionality.

> [HS2] We propose to use explicit FEC to tell there’s no EHs in the packet to avoid the label stack scanning in such cases. This doesn’t nullify the need to carry extra metadata for some user packets to support some in-network functions.  

I will wait to see a concrete complete proposal before commenting.

> [HS2] Header overhead is an important dimension for comparison.  To make it apple to apple, let’s first clarify several points.
> If an action doesn’t need any extra data, there’s no point to allocate an EH for it. A flag in EHI is sufficient (this applies to the NFFRR case).

Ok, that saves you 4 octets, but costs you a bit in EHI. There’s only a finite number of bits available.  What do you do when they’re consumed?

> You are using the bitmap encoding since you mention FAI. Bitmap is more compact than Type/Length for sure. Here the implication is that each ISD must be 4 octets long, otherwise it will need every node to understand the size of every data item, further complicating the design.  

In FAI, the length of the ISD is defined based on the action and may have different lengths depending on bit combinations. Please see the EG bits, for example.

Yes, determining the length requires reading the full ISD.

> For extensibility, how many ISD data items is planned to be supported? If it exceeds the one label capacity, you will need to extend it. So the actual overhead of FAI could be 8 octets instead of 4. 

Absolutely true. Allocate popular functions first. :) If there are too many actions, then we could also consider a second SPL.

> Let’s temporarily put all these details aside and assume the minimum sized FAI. Basically, EH has a fixed 4 byte overhead for HEH. For each use case that does need metadata, EH will add 4 more byte overhead. This is the header overhead comparison. (In your example, it’s 24 bytes PSD vs. 12 bytes ISD)

So EH is twice as expensive as FAI.

> Now let’s look at the other dimensions.
> We know some use cases requiring data too big to fit in stack, so PSD is needed anyway. PSD can be used for the use cases with smaller metadata size, although it’s relatively inefficient as discussed above. So essentially we can have just one mechanism all everything but to support ISD we end up with two.

That’s true.  We’re trading added complexity for better performance.

>   
> Bitmap, albeit succinct, is inflexible and less extensible. The semantics and order are fixed at the design time; the total size of each data item must be equal to a label size to make bitmap useful in finding the corresponding data item.

That’s incorrect. The size can vary. An implementation must support the NAI that are set.  Handling cases where a node only knows some of them still needs to be discussed, but I favor just not supporting that case at all.

> Use cases identified in the future can only use unused bitmap bits, regardless of its importance. If a use case requires to extend its data size, it’s out of luck; if some common use cases also apply to the other types of networks (especially for those still under investigation), the use cases can be seriously limited by such size constraint which hampers design sharing and interoperability.

Yes, future actions can only allocate unused bitmap bits.  Reusing existing bitmap bits would overlap with existing functions.  Your statement seems to be a non-sequitur.

A new action may define whatever length of ISD it likes, including being variable length. It’s not recommended, but it is possible.

>     
> Parsing based on bitmap significantly bloats either the parser size or the parsing latency or both (as shown in my analysis). People can write pseudocode by themselves to verify. This is bad for both software and hardware data plane implementation. No wonder bitmap is never used for header encoding before (it’s sometimes used in a header for its sub-structure, which is related to the header processing but not the header parsing).

Only if you insist on using a TCAM for parsing. From my perspective, that’s the exactly wrong tool for the job. If you use conventional ALU, then parsing become a straightforward ‘if’ chain.

We are not obligated to optimize the solution for the wrong tools.

> The encoding of each ISD item itself is awkward. You have 30 bits and only 30 bits at your disposal with a BoS bit in between. It creates a hole in data, which is bad for both software and hardware.

We already discussed this. Yes, it’s not optimal, but it’s only a few instructions to fix, if necessary.

> [HS2] I think the new design won’t nullify the EL/ELI standard so they need to coexist in an incremental deployment scenario given EL/ELI might have been realized (It’s possible to provide yet another entropy solution using the new mechanism but the necessity is subject to further discussion IMO). But they don’t need to interact with each other because each has its clear function and operational procedure.
> 

Well, you could do things that way, but IMHO, that’s suboptimal.  If you’re including MNA, then you’re already limting yourself to MNA capable nodes. You might as well take the benefit of the better encoding.

Tony

---

From: Tianran Zhou <[email-masked]>
Date: Sat, 16 Apr 2022 07:42:49 +0000

Yes, starting with use cases is good. Then we should figure out what are really necessary.
For example, in your list, entropy, I do not see the requirement to reinvent the wheel.
I do not think DETNET(including bounded delay) need neither. There is already specifications for MPLS.
We are not going to review them all here.

For those use cases really need a new MPLS extension, PSD could be a generic way to achieve.
There may further be optimization need, optionally. It could be ad hoc, case by case.
I do not think we really need nor can have a generic ISD mechanism.
I get this conclusion also based on my analysis of existing ISD solution. There are too many constraints in stack. I see complexities added to existing limited resource, which is the case that ISD want to tackle.

Tianran

From: John E Drake [mailto:jdrake=[email-masked]]
Sent: Friday, April 15, 2022 9:25 PM
To: Tianran Zhou <[email-masked]>; Tony Li <[email-masked]>
Cc: [email-masked]
Subject: RE: [mpls] Concerns about ISD

Hi,

Yours Irrespectively,

John

T.

Yours Irrespectively,

John

Hi,

Comments in line.

Tianran

Hi Tony,

Best,
Tianran

Hi Tianran,

Tony

---

From: Tianran Zhou <[email-masked]>
Date: Sat, 16 Apr 2022 07:58:17 +0000

It’s better if we can put all the solutions in the plate. Use the same use cases, scenarios, and criteria to compare.
And to see, how ISD can achieve, and how ISD could be better than PSD.

Tianran

From: Tony Li [mailto:[email-masked]] On Behalf Of Tony Li
Sent: Saturday, April 16, 2022 9:22 AM
To: Haoyu Song <[email-masked]>
Cc: John E Drake <[email-masked]>; Tianran Zhou <[email-masked]>; [email-masked]
Subject: Re: [mpls] Concerns about ISD

Hi Haoyu,

EHI: 4 octets
HEH: 4 octets
NFFRR: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 28 octets

Do I have that right?

FAI: 4 octets

Total: 12 octets

  1.  If an action doesn’t need any extra data, there’s no point to allocate an EH for it. A flag in EHI is sufficient (this applies to the NFFRR case).

  1.

  1.
  2.  For extensibility, how many ISD data items is planned to be supported? If it exceeds the one label capacity, you will need to extend it. So the actual overhead of FAI could be 8 octets instead of 4.

  1.

  1.  We know some use cases requiring data too big to fit in stack, so PSD is needed anyway. PSD can be used for the use cases with smaller metadata size, although it’s relatively inefficient as discussed above. So essentially we can have just one mechanism all everything but to support ISD we end up with two.

  1.
  2.  Bitmap, albeit succinct, is inflexible and less extensible. The semantics and order are fixed at the design time; the total size of each data item must be equal to a label size to make bitmap useful in finding the corresponding data item.

  1.  Use cases identified in the future can only use unused bitmap bits, regardless of its importance. If a use case requires to extend its data size, it’s out of luck; if some common use cases also apply to the other types of networks (especially for those still under investigation), the use cases can be seriously limited by such size constraint which hampers design sharing and interoperability.

  1.
  2.  Parsing based on bitmap significantly bloats either the parser size or the parsing latency or both (as shown in my analysis). People can write pseudocode by themselves to verify. This is bad for both software and hardware data plane implementation. No wonder bitmap is never used for header encoding before (it’s sometimes used in a header for its sub-structure, which is related to the header processing but not the header parsing).

  1.
  2.  The encoding of each ISD item itself is awkward. You have 30 bits and only 30 bits at your disposal with a BoS bit in between. It creates a hole in data, which is bad for both software and hardware.

Tony

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 16 Apr 2022 15:38:43 +0200

Hi Tianran,

I do not think DETNET(including bounded delay) need neither. There is
> already specifications for MPLS.
>

That is a very interesting and important point. I just looked at RFC8964
and surprise ... no need to change MPLS architecture. Just use a service
label concept and carry service info outside of the data plane.

What I called Forwarding Behaviour ID they call service label (analogy to
other types of service labels already carried in MPLS). The
difference between RFC8964 and FBI approach seems to be about types of
services (aka actions) to execute on the packets. Also FBI concept works
with domain wide labels while service labels are pretty much local to a
given node.

So a bit curious what the above architectures are missing which would
really justify to define, develop and deploy a new MPLS data plane ?

Kind regards,
Robert

PS. For those who are claiming that use a post stack data PSD should be the
only option let me observe that if this solution is supposed to work with
SR-MPLS the label stack of pure transport may be pretty large. So number of
nodes may not be capable to fetch PSD (we invented EL just because transit
LSRs were not even able to read IP header). That to me is a practical
blocker for this approach if MNA will continue.

---

From: Stewart Bryant <[email-masked]>
Date: Sat, 16 Apr 2022 16:47:10 +0100

> On 16 Apr 2022, at 14:38, Robert Raszuk <[email-masked]> wrote:
> 
> Hi Tianran,
> 
> That is a very interesting and important point. I just looked at RFC8964 and surprise ... no need to change MPLS architecture. Just use a service label concept and carry service info outside of the data plane. 
> 
> What I called Forwarding Behaviour ID they call service label (analogy to other types of service labels already carried in MPLS). The difference between RFC8964 and FBI approach seems to be about types of services (aka actions) to execute on the packets. Also FBI concept works with domain wide labels while service labels are pretty much local to a given node. 
> 
> So a bit curious what the above architectures are missing which would really justify to define, develop and deploy a new MPLS data plane ? 
> 
> Kind regards,
> Robert
> 
> PS. For those who are claiming that use a post stack data PSD should be the only option let me observe that if this solution is supposed to work with SR-MPLS the label stack of pure transport may be pretty large. So number of nodes may not be capable to fetch PSD (we invented EL just because transit LSRs were not even able to read IP header). That to me is a practical blocker for this approach if MNA will continue. 
> 

Detnet is really an overlay system.

The delivery label takes the packet to the next hop in the Detnet layer. This is popped and the service label exposed. The service label is then processed. Then if the packet is to continue a new service label is pushed followed by a new delivery label.

In a PHPed network where every node is a service label the service label is always ToS. However the operation in not really swap as in MSPW, the service label and the CW are examined and processed.

Note that Detnet does not support many of the features being asked for in NA/MIAD.

- Stewart

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 16 Apr 2022 22:21:40 +0200

Hi Stewart,

Yes - exactly. And I fail to see a reason by MNA/NA/MIAD can not be an
overlay too from transport point of view.

Cheers,
R.

On Sat, 16 Apr 2022 at 17:47, Stewart Bryant <[email-masked]>
wrote:

> PS. For those who are claiming that use a post stack data PSD should be
> the only option let me observe that if this solution is supposed to work
> with SR-MPLS the label stack of pure transport may be pretty large. So
> number of nodes may not be capable to fetch PSD (we invented EL just
> because transit LSRs were not even able to read IP header). That to me is a
> practical blocker for this approach if MNA will continue.
>
>
> The delivery label takes the packet to the next hop in the Detnet layer.
> This is popped and the service label exposed. The service label is then
> processed. Then if the packet is to continue a new service label is pushed
> followed by a new delivery label.
>
> In a PHPed network where every node is a service label the service label
> is always ToS. However the operation in not really swap as in MSPW, the
> service label and the CW are examined and processed.
>