Subject: [Detnet-dp-dt] BIER: TE or not TE? (was BIER packet duplication question)

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 25 Feb 2016 08:32:02 +0000

I can easily agree with that, Greg.

There are really 3 aspects:

1.      Classical BIER vs. BIER-TE

2.      Path computation (centralized or IGP)

3.      Data plane signaling

The coolest thing with BIER is probably that it creates an abstraction that clearly decouples those aspects.
So even though it seems that classical BIER comes with classical IGPs and BIER-TE comes with PCE and/or Constrained IGP computations, it does not have to be that way.
We can really have any combination of the 3 elements above. And whether we go for TE or classical, most of the text that you have written applies.

The way I differentiate classical BER vs. TE is this:

-        Classical BIER is a multicast technique where bits indicate end points, the leaves in a multicast tree.

-        BIER-TE is a unicast technique where bits indicate adjacencies, which are really an abstraction of the topological segments between intermediate nodes. For DetNet, segments are typically between replication points and elimination points.
Based on the categorization above, my reading of how you propose to use classical BIER is that in fact you are describing BIER TE.

If that's correct then we could reorganize the 2 BIER sections so that your piece focusses on topological and routing aspects (item 2 above) and my piece focusses on what the BIER bits represent and how they are used.

Does that seem right?

Pascal

From: Gregory Mirsky [mailto:[email-masked]]
Sent: jeudi 25 février 2016 00:57
To: Pascal Thubert (pthubert) <[email-masked]>
Cc: [email-masked]
Subject: RE: [Detnet-dp-dt] BIER packet duplication question

Hi Pascal,
I don't think that "classical" BIER cannot use traffic engineered explicitly routed routing underlay. The BIER architecture draft<https://datatracker.ietf.org/doc/draft-ietf-bier-architecture/?include_text=1> points that the BIER domain may have multiple routing underlays (routing underlay is analogous to forwarding adjacencies in MPLS). One underlay that map BFR's IP address to set of one or more paths can be calculated by IGP's SPF. With such underlay being perhaps more common case, nothing precludes from constructing another underlay by CSPF with specific set of constraints or use, for example, Steiner tree for multicast distribution. The BIER Architecture document notes that a particular BIER sub-domain can have only single routing underlay but the same underlay may be shared by multiple sub-domains. In case of MPLS data plane, BIER-MPLS label identifies the BIER sub-domain and thus identifies the routing adjacency that may be constructed of explicitly routed TE-LSPs. I think that that what DetNet may use for some use cases. For cases where timing constraints are not very stringent TE LSPs of the routing underlay may be loosely routed.

                Regards,
                                Greg

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Thursday, February 18, 2016 5:50 AM
To: Gregory Mirsky
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: [Detnet-dp-dt] BIER packet duplication question

Hello Greg:

With the classical BIER as described in the architecture, bits indicate a destination so it is not designed for nailing down TE paths.  It can be done if we have 2 domains per flow, and a route per domain that is computed centrally, but that looks like an overkill.

This is why we are proposing a BIER-TE where the bits indicate segments. Listening to you at the call, this proposal seems to be what you are really after. I'm happy to talk more when you can.

Also,  I agree with you that overloading the entropy field is probably a bad idea.

Cheers,

Pascal

From: Gregory Mirsky [mailto:[email-masked]]
Sent: jeudi 18 février 2016 03:07
To: Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: [Detnet-dp-dt] BIER packet duplication question

Hi Pascal,
thank you for providing additional context to slides. I'll comment on them in more details later.
I've talked with Tony P, Ice should have no problem identifying him, about some proposals in BIER you've mentioned earlier, e.g. re-use of Entropy field. I'll point that BIER Architecture document is explicit about interpretation of bits in the BitString. These are indexes of BFERs in the given BIER sub-domain (sub-domain is limited to 256 BFRs) and not links, logical or physical (that could be Segment Routing underlay to BIER domain). Thus we cannot reach the same BFER in the given BIER sub-domain over disjoint path or, in other words, the BFER can receive only single copy of the payload in the given sub-domain. Hence my proposal to use multiple, several BIER sub-domains overlayed over the same BIER domain.
Yes, as I've noted, I gave only an example of how BIER may be used to support DetNet packet replication and termination to improve probability of delivery within predetermined time.

                Regards,
                                Greg

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Wednesday, February 17, 2016 9:27 AM
To: Gregory Mirsky
Cc: [email-masked]<mailto:[email-masked]>
Subject: FW: [Detnet-dp-dt] BIER packet duplication question

Hello Greg:

Here's some text related to the slides in the github since the previous call. Should I place text in the document about that? Would that be a different section?

Also: I heard you saying that you talked to someone and concluded that a certain thing does not work, but I could not hear what that was. Could you please elaborate on that one -sorry for this- ?

Finally, the BIER domain can be seen as a topology information. When we use a PCE, we can do with just 2 of them, blue and red, say. Each bicasted flow will be sent twice, a blue copy and a red copy.

I wish to make sure that the text does not give the impression that the BIER domains are a physical separation, or that you have to define 2 new domains for each new flow. In the simplest case, all BIER nodes belong to both domains, but happen to participate to one only for a particular bi-casted flow (same source and destination), as long as the paths can be maintained non-congruent.

For a multicast flow, a node may have to forward both a blue and a red copy even when a full non-congruency is achieved. This is because the non-congruency needs to be per destination (which is much simpler) as opposed to a tree wide property. The whole trick is to minimize that case so as to optimize the efficiency of the multicast. That's really a PCE game, the nodes are obedient slaves.

Cheers,

Pascal

From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: vendredi 29 janvier 2016 13:35
To: Gregory Mirsky <[email-masked]<mailto:[email-masked]>>; Olivier Marce <[email-masked]<mailto:[email-masked]>>; Jouni Korhonen <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Cc: IJsbrand Wijnands (iwijnand) <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Detnet-dp-dt] BIER packet duplication question

Hello Greg:

The value of the BIER header in this proposal is in the replication and elimination domain, as opposed to flow identification and sequencing.  BIER is used to identify the segments that are activated (by the controller) and those where transmission failed (which can be returned to the controller for action). The value is that the replication can be controlled and monitored with the granularity of a packet and a segment.

BIER also enables to abstract what a segment is. A segment can be an Ethernet hop, and it can be an IPv6 loose source routed path. In the latter case the source route state must be provisioned in the replicator node that serves the particular bit(s), together with the timing or shaping information for that particular flow.

In more details:

-        The controller activates replication by setting bits associated with segments

-         the eliminator function strips the source route headers and performs a bitwise AND on the bitmaps from the various copies of the packet that it receives.

-        For each bit that it serves, the replicator function forms a packet where it inserts the source route header for the next segment that corresponds to the bit, inserts a BIER header (e.g. the result of the AND operation by the eliminator function), zeroes out the bit in the BIER header and transmits along the SR path.

To your points:

1)      With MPLS, the BIER behavior is identified implicitly with the MPLS label which is used to find the state associated to the BIER bits, but I agree that explicit is probably better.

2)      The association of the bit with an address (such as a loopback address in the router) does not depend on whether we use this router for DetNet or classical BIER. I would not necessarily want to change any of  this. I expect that the controller knows that the router is capable of DetNet before it makes it a segment endpoint, and only segment endpoints need to look into the BIER header which is placed after the source route header.

3)      I understand that the proto field in BIER allows us to define what the protocol inside is. The original proposal was to define a DetNet header for the flow ID (and next header), but then the new header that we define could have more, like flow ID and sequence number as I understand you are suggesting with your point on CW. In that case, we'd be independent on the encapsulation, and the flow/seq encoding would be independent on whether we use the technique above to do replication and elimination, like our proposed BIER encoding.

Does that make sense?

Pascal

From: Gregory Mirsky [mailto:[email-masked]]
Sent: jeudi 28 janvier 2016 19:03
To: Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>>; Olivier Marce <[email-masked]<mailto:[email-masked]>>; Jouni Korhonen <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Cc: IJsbrand Wijnands (iwijnand) <[email-masked]<mailto:[email-masked]>>
Subject: RE: [Detnet-dp-dt] BIER packet duplication question

Hi Pascal, et. al,
thank you for very interesting proposal. I do have concern about re-definition of the Entropy field in the BIER encapsulation for MPLS network. In my opinion it creates another class of BIER Forwarding Routers (BFRs) that interpret the field differently. As result, as I think of it, this behavior must be advertised in IGP so that non-compliant BFRs, those that interpret it exclusively as defined in draft-ietf-bier-mpls-encapsulation, be excluded from BFR role. But that doesn't mean that I don't see BIER as suitable data plane for DetNet. I do believe though that using separate BIER sub-domain with disjoint (level of disjointness may be one of constraints in path computation in addition to latency, jitter and loss ratio) paths would be appropriate way to use BIER infrastructure for DetNet. And I do believe that sequence number belong in distinct DetNet layer encapsulation (we likely find that it should have more than just Sequence Number).
I'll note that use of the Sequence Number field, as per RFC 4448, is optional as well as use of PW CW in Ethernet PW.Section 2.7 RFC 7079 gives, though by now historical view, of how generic support of PW CW in Ethernet PW been. Hence, I do believe that distinct DetNet encapsulation is justified and the right way to go.

                Regards,
                                Greg

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Saturday, January 23, 2016 8:04 AM
To: Olivier Marce; Gregory Mirsky; Jouni Korhonen; [email-masked]<mailto:[email-masked]>
Cc: IJsbrand Wijnands (iwijnand)
Subject: RE: [Detnet-dp-dt] BIER packet duplication question

I visited Ice (cc) this week to discuss the applicability of BIER.

Indeed we found a great match but that will need some explaining and we'll do a write up if this team wishes to hear about it.

The catch is that the method we have in mind leverage Cisco IPR, which Cisco would announce with the usual terms as soon as we publish a draft 00. Is that acceptable?

In very short we can decline BIER-TE and use the bits to control at the source which of the possible replication points will effectively do it (setting bits associated to L2 links/L3 segments) and mark along the way which replicators effectively made a replication. The value is that it is always possible to identify which copy by which replication point a packet is, and at the point of arrival, we can determine which segment failed on a per packet basis, and use that to de/reactivate replication points (by setting the bits).

The BIER header is of variable size, we can use a model with 64 bits if 64 segments are enough for us, but we can also have 128 or 256 segments in very complex paths.

      0                   1                   2                   3

      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     |0 1 0 1|  Ver  |  Len  |              Entropy                  |

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     |                BitString  (first 32 bits)                     ~

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     ~                                                               ~

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     ~                BitString  (last 32 bits)                      |

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     |OAM|     Reserved      | Proto |            BFIR-id            |

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                           Figure 1: BIER Header

The entropy can be used to store a wrapping packet sequence number within a flow.

That's certainly enough to encode the gap between 2 flows.

The source of the flow is indicated by the BFIR-id.

The Flow itself can be indicated in the encapsulating protocol, in which case we'd need to design one way per encaps.

An alternate is to use a Proto  to encode the Flow ID after the BIER header.

What do you guys think?

Pascal

> -----Original Message-----

> From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of

> Olivier Marce

> Sent: vendredi 22 janvier 2016 17:11

> To: Gregory Mirsky <[email-masked]<mailto:[email-masked]>>; Jouni Korhonen

> <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

> Subject: Re: [Detnet-dp-dt] BIER packet duplication question

> Hi (from Nokia now)

> I certainly missed some earlier discussion but so far the link between DetNet

> and BIER  is not clear for me.

> Meanwhile I'm looking at time related technologies, familiarisation only, up

> to now. I wonder if state of the art solutions are accurate only to support

> what is expected in DetNet.

> Maybe we could have a page on Git listing considered technologies and how

> they can/could contribute to DetNet ?

> Regards

> Le 13/01/2016 01:44, Gregory Mirsky a écrit :

> > Hi Jouni,

> > BIER does not allow packet duplication within the given BIER sub-domain

> that in case of MPLS dataplane is identified by BIER-MPLS label at the

> bottom of label stack. Duplication of packets can be realized by computing,

> centralized or distributed, distribution trees with desired level of node and

> link disjointness, exclusion. BIER supports use of multiplicity of tree

> computation algorithms.

> >          Regards,

> >                         Greg

> > -----Original Message-----

> > From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of

> > Jouni Korhonen

> > Sent: Tuesday, January 12, 2016 9:44 AM

> > To: [email-masked]<mailto:[email-masked]>

> > Subject: [Detnet-dp-dt] BIER packet duplication question

> > I did some "familiarization" with BIER and it is not instantly clear to me

> how the packet duplication in intermediate BFRs would work in a case the

> destination is a single BF(E)R?

> > - Jouni

> --

> !! Happy Nok'year !!

> --

> Olivier Marcé

> Nokia Bell Labs France

> _______________________________________________

> Detnet-dp-dt mailing list

> [email-masked]<mailto:[email-masked]>

> https://www.ietf.org/mailman/listinfo/detnet-dp-dt

---

From: Gregory Mirsky <[email-masked]>
Date: Thu, 25 Feb 2016 22:38:42 +0000

Hi Pascal,
I think that we've converged on the BIER and, correct me if I misinterpreted, agreed that the traffic engineering for BIER can be achieved by different means. The only important difference between what we can call "classical" BIER and BIER-TE is interpretation of bits in the BitString that consequently affects scaling property and types of services, use cases each can support. (I'll note that the two are incompatible in the data plane and, in my view, present different models of the data plane, but that discussion would be more appropriate for BIER WG.)
I've started reading the draft with a pencil but probably wait with sending my notes for another day or two so that Janos would kindly help to put in my latest text (I owe you one more) and you may make changes to the BIER section. Or would you suggest I send my notes sooner?

                Regards,
                                Greg

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Thursday, February 25, 2016 12:32 AM
To: Gregory Mirsky
Cc: [email-masked]
Subject: BIER: TE or not TE? (was BIER packet duplication question)

3.      Data plane signaling

Does that seem right?

Pascal

Hi Pascal,

                Regards,
                                Greg

Hello Greg:

Cheers,

Pascal

Hi Pascal,

                Regards,
                                Greg

Hello Greg:

Cheers,

Pascal

Hello Greg:

In more details:

To your points:

Does that make sense?

Pascal

Hi Pascal, et. al,

                Regards,
                                Greg

      0                   1                   2                   3

     ~                                                               ~

                           Figure 1: BIER Header

What do you guys think?

Pascal

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Feb 2016 18:02:08 -0500

On 2/25/2016 5:38 PM, Gregory Mirsky wrote:
> I’ve started reading the draft with a pencil but probably wait with
> sending my notes for another day or two 
Gregg,
    It would be really good to get your comments/input on the other
sections to the same level of "energy" as bier -- particularly  in some
other areas where you have some good experience - e.g., in MPLS and MPLS
TE....

Lou

---

From: Gregory Mirsky <[email-masked]>
Date: Thu, 25 Feb 2016 23:04:22 +0000

Hi Lou,
of course, I'll continue with the notes on non-BIER sections and will share with the team.

	Regards,
		Greg

-----Original Message-----
From: Lou Berger [mailto:[email-masked]] 
Sent: Thursday, February 25, 2016 3:02 PM
To: Gregory Mirsky; Pascal Thubert (pthubert)
Cc: [email-masked]
Subject: Re: [Detnet-dp-dt] BIER: TE or not TE? (was BIER packet duplication question)

On 2/25/2016 5:38 PM, Gregory Mirsky wrote:
> I've started reading the draft with a pencil but probably wait with 
Gregg,
    It would be really good to get your comments/input on the other sections to the same level of "energy" as bier -- particularly  in some other areas where you have some good experience - e.g., in MPLS and MPLS TE....

Lou

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Feb 2016 18:10:03 -0500

Excellent, thank you.  The BIER discussion seem to moving towards SOS...

On 2/25/2016 6:04 PM, Gregory Mirsky wrote:
> [email-masked]

---

From: Gregory Mirsky <[email-masked]>
Date: Sat, 27 Feb 2016 06:21:00 +0000

Hi Lou and the team,

your wish is my command.

Attached is the copy with my notes and edits (except for BIER/BIER-TE section). I think that I've two questions I need to clarify:

*         can time synchronization requirement be fulfilled by comprehensive delay measurement, active and/or passive, that characterizes a DetNet flow;

*         can packet traceroute, as well, be fulfilled by active OAM traceroute that is guaranteed to fate share with, be in-band with a DetNet flow.

Regards,

        Greg

-----Original Message-----
From: Lou Berger [mailto:[email-masked]]
Sent: Thursday, February 25, 2016 3:02 PM
To: Gregory Mirsky; Pascal Thubert (pthubert)
Cc: [email-masked]
Subject: Re: [Detnet-dp-dt] BIER: TE or not TE? (was BIER packet duplication question)

Gregg,

Lou

---

From: Jouni Korhonen <[email-masked]>
Date: Fri, 26 Feb 2016 23:01:45 -0800

Thanks for the early review. I have few quick comments/questions here:

* "This is not property of PWs ut only of TDMoIP.” -> even for TDMoIP/CESoPSN RTP is described as part of the payload encapsulation layer and imho can be considered to be part of PWs when used. RFC3985 states that PW encapsulation layer may contain Payload Convergence, Timing and Sequencing.

* "Cannot happen as PW would be between the same pair of Pes and thus guaranteed to be different since assigned by the same Pes. Outer labels, on the other hand, may be identical since LSPs traverse different LSRs. But I don’t see how that is important. MPLS OAM, in ECMP doesn’t use IP 5-tuple, is in-band with data and LSP Ping may be used to trace the path.” -> this goes back to the presentation last IETF and the duplication of PWs could actually benefit if the PW labels were the same for duplicates. I acknowledge this is not the behaviour today and we can make such a change if needed into existing PWE3. Thus the language “even if the..” in the text.

- Jouni

> On 26 Feb 2016, at 22:21, Gregory Mirsky <[email-masked]> wrote:
> 
> Hi Lou and the team,
> your wish is my command.
> Attached is the copy with my notes and edits (except for BIER/BIER-TE section). I think that I’ve two questions I need to clarify:
> ·         can time synchronization requirement be fulfilled by comprehensive delay measurement, active and/or passive, that characterizes a DetNet flow;
> ·         can packet traceroute, as well, be fulfilled by active OAM traceroute that is guaranteed to fate share with, be in-band with a DetNet flow.
>  
> Regards,
>         Greg
>  
> <draft-dt-detnet-dp-alt+greg.docx>_______________________________________________