Subject: [Detnet-dp-dt] DetNet layering

From: János Farkas <[email-masked]>
Date: Wed, 2 Mar 2016 15:56:32 +0100

Hi,

 From the current version of the draft we can see that all chapters have 
issues with some of the criteria.
The reason for that is that we have not introduced appropriate layering 
for dealing with DetNet flows.
We may consider the introduction of two layers to structure the analysis 
better.

However, the criteria are well selected and appropriate for our data 
plane analysis, no intent to change it. It was well discussed and 
concluded.

The two proposed layers are:
- Det-layer: a shim layer to carry DetNet flow specific attributes
- Net layer: provides the networking between the DetNet flow endpoints

We could map the data plane options to these two layers as follows:
+------+
| Det | PW, RTP, UDP, GRE, L2TP
+------+
| Net | IPv6, IPv4, MPLS
+------+

(incomplete list for the moment)

Most criteria apply only for one of the layers. The criteria can be 
mapped as follows:
- Det-layer:
     #1 Encapsulation and overhead
     #2 Stream identification (Flow-ID)
     #3 Packet sequencing
     #6 Operations and management
     #7 Time synchronization (time stamping)
     #8 Class and quality of service capabilities (Det specific)
     #10 Technical maturity
- Net-layer
     #1 Encapsulation and overhead
     #2 Stream identification (Egress-ID)
     #4 Explicit routes
     #5 Packet replication and deletion
     #6 Operations and management
     #7 Time synchronization (time/phase sync)
     #8 Class and quality of service capabilities (Net specific)
     #9 Packet traceability
     #10 Technical maturity

Some explanation for items appearing in both layers:
- Stream identification: Stream-ID = Flow-ID + Egress-ID
For example, for PW over MPLS "Flow-ID=PW-label" and 
"Egress-ID=LSP-label-of-egress-node".

- QoS has aspects that are specific to the Det-layer and other aspects 
to the Net-layer
For example, for PW over MPLS "Det-specific = TC-field-of-PW-label" and 
"Net-specific = TC-field of-LSP-labels".
There may be a mapping between the QoS fields of the two layers.

- Time sync: Det-layer can add a time stamp to the packets of the flow, 
whereas Net-layer provides time/phase sync among the network nodes.

Such a structuring would make the draft more easy to follow. For 
instance, it would decrease cross-references, repeated text that we have 
at the moment.

Maybe we could discuss this a bit on the call today.

Regards,
Janos

---

From: János Farkas <[email-masked]>
Date: Fri, 4 Mar 2016 13:50:02 +0100

Hi,

Initial text on layering has been added to the draft in github. It is an 
individual section; Section 3 for now.
Review please.

Cheers,
Janos

On 3/2/2016 3:56 PM, János Farkas wrote:
> Hi,
>
> From the current version of the draft we can see that all chapters 
> have issues with some of the criteria.
> The reason for that is that we have not introduced appropriate 
> layering for dealing with DetNet flows.
> We may consider the introduction of two layers to structure the 
> analysis better.
>
> For example, for PW over MPLS "Det-specific = TC-field-of-PW-label" 
> and "Net-specific = TC-field of-LSP-labels".
> - Time sync: Det-layer can add a time stamp to the packets of the 
> flow, whereas Net-layer provides time/phase sync among the network nodes.
>
> instance, it would decrease cross-references, repeated text that we 
> have at the moment.
>
> _______________________________________________
> Detnet-dp-dt mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet-dp-dt

---

From: Jouni Korhonen <[email-masked]>
Date: Fri, 4 Mar 2016 07:58:36 -0800

Thanks Janos for a quick work,

I'll have a read and come back with comments if any.

- Jouni

3/4/2016, 4:50 AM, János Farkas kirjoitti:
> Hi,
>
> Initial text on layering has been added to the draft in github. It is 
> an individual section; Section 3 for now.
> Review please.
>
> Cheers,
> Janos
>
>
>> - QoS has aspects that are specific to the Det-layer and other 
>> aspects to the Net-layer

---

From: Jouni Korhonen <[email-masked]>
Date: Sun, 6 Mar 2016 17:45:19 -0800

Janos,

I had another read on the new Section 3. I really like where this is heading to. Few comments below:

* I would argue "#5 Packet replication and deletion (for seamless redundancy)” also belongs to Det layer, depending on which layer the detnet stream and duplication detection is done in the first place.

- Jouni

> On 04 Mar 2016, at 04:50, János Farkas <[email-masked]> wrote:
> 
> Hi,
> 
> Initial text on layering has been added to the draft in github. It is an individual section; Section 3 for now.
> Review please.
> 
> Cheers,
> Janos
> 
> 
>> From the current version of the draft we can see that all chapters have issues with some of the criteria.
>> The reason for that is that we have not introduced appropriate layering for dealing with DetNet flows.
>> We may consider the introduction of two layers to structure the analysis better.
>> 
>> However, the criteria are well selected and appropriate for our data plane analysis, no intent to change it. It was well discussed and concluded.
>> 
>> Most criteria apply only for one of the layers. The criteria can be mapped as follows:
>> - Det-layer:
>> For example, for PW over MPLS "Flow-ID=PW-label" and "Egress-ID=LSP-label-of-egress-node".
>> 
>> - QoS has aspects that are specific to the Det-layer and other aspects to the Net-layer
>> For example, for PW over MPLS "Det-specific = TC-field-of-PW-label" and "Net-specific = TC-field of-LSP-labels".
>> - Time sync: Det-layer can add a time stamp to the packets of the flow, whereas Net-layer provides time/phase sync among the network nodes.
>> 
>> Such a structuring would make the draft more easy to follow. For instance, it would decrease cross-references, repeated text that we have at the moment.
>>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Mar 2016 07:52:21 +0000

I agree with Jouni, Janos.

Regards,

Pascal

> Le 7 mars 2016 à 02:45, Jouni Korhonen <[email-masked]> a écrit :
> 
> 
> Janos,
>

---

From: János Farkas <[email-masked]>
Date: Mon, 7 Mar 2016 13:39:43 +0100

Hi Jouni and Pascal,

It is a valid approach to have "#5 Packet replication and deletion (for 
seamless redundancy)” in the Det layer (as well). I'm open. ;-)

The idea behind the current text is that #3 Packet sequencing belongs to 
the Det layer. However, the functionality of replicating packets and 
eliminating duplicates, i.e. #5, is rather a networking functionality, 
that's why it was added to the Net layer. Nonetheless, #5 relies on #3. 
That is, the function of #5 even being in Net layer, has to use 
attribute in the Det layer, i.e. #3. I admit it is not that clean. The 
Det and the Net layers are closely related, we cannot draw a hard line 
between them, that's why the different line in the figure. It is not 
only "#5 Packet replication and deletion" that need to leverage both layers.

Nevertheless, I'm open to change. We can move #5 to the Det layer, or 
have it at both the Det and the Net layers.

Thanks,
Janos

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Mar 2016 16:56:45 +0000

Hello Janos

This makes sense; 

quick question: do you see another application to the sequence like reordering after balancing over non equally latent paths? If yes then sequencing is a tool of multiple usages. If no then sequencing is a tool that is a component of the elimination function. In both case they are of different nature, tool vs. function.

Take care,

Pascal

> Le 7 mars 2016 à 13:39, János Farkas <[email-masked]> a écrit :
> 
> Hi Jouni and Pascal,
> 
> It is a valid approach to have "#5 Packet replication and deletion (for seamless redundancy)” in the Det layer (as well). I'm open. ;-)
> 
> The idea behind the current text is that #3 Packet sequencing belongs to the Det layer. However, the functionality of replicating packets and eliminating duplicates, i.e. #5, is rather a networking functionality, that's why it was added to the Net layer. Nonetheless, #5 relies on #3. That is, the function of #5 even being in Net layer, has to use attribute in the Det layer, i.e. #3. I admit it is not that clean. The Det and the Net layers are closely related, we cannot draw a hard line between them, that's why the different line in the figure. It is not only "#5 Packet replication and deletion" that need to leverage both layers.
> 
> Nevertheless, I'm open to change. We can move #5 to the Det layer, or have it at both the Det and the Net layers.
> 
> Thanks,
> Janos
> 
>

---

From: Jouni Korhonen <[email-masked]>
Date: Mon, 7 Mar 2016 10:24:12 -0800

Hi Janos,

3/7/2016, 4:39 AM, János Farkas kirjoitti:
> Hi Jouni and Pascal,
>
> It is a valid approach to have "#5 Packet replication and deletion 
> (for seamless redundancy)” in the Det layer (as well). I'm open. ;-)
>
> The idea behind the current text is that #3 Packet sequencing belongs 
> to the Det layer. However, the functionality of replicating packets 
> and eliminating duplicates, i.e. #5, is rather a networking 
> functionality, that's why it was added to the Net layer. Nonetheless, 
> #5 relies on #3. That is, the function of #5 even being in Net layer, 
> has to use attribute in the Det layer, i.e. #3. I admit it is not that 
> clean. The Det and the Net layers are closely related, we cannot draw 
> a hard line between them, that's why
This is the issue I was also having my concerns. We took a very 
preliminary peek how to actually implement some of these and it is not 
always clear how the layering goes in a "clean way".

> the different line in the figure. It is not only "#5 Packet 
> replication and deletion" that need to leverage both layers.
Agree.
> have it at both the Det and the Net layers.
I would have it at both layers for the time being.. subject to removal 
once we (or I get more insight), though.

- jouni
>>> * I would argue "#5 Packet replication and deletion (for seamless 
>>> redundancy)” also belongs to Det layer, depending on which layer the 
>>> detnet stream and duplication detection is done in the first place.
>>>
>>>
>>> - Jouni
>>>
>>>
>>>> Initial text on layering has been added to the draft in github. It 
>>>> is an individual section; Section 3 for now.
>>>> Review please.
>>>>
>>>> Cheers,
>>>> Janos
>>>>
>>>>
>>>>>  From the current version of the draft we can see that all 
>>>>> chapters have issues with some of the criteria.
>>>>> However, the criteria are well selected and appropriate for our 
>>>>> data plane analysis, no intent to change it. It was well discussed 
>>>>> and concluded.
>>>>>
>>>>> For example, for PW over MPLS "Det-specific = 
>>>>> TC-field-of-PW-label" and "Net-specific = TC-field of-LSP-labels".
>>>>> instance, it would decrease cross-references, repeated text that 
>>>>> we have at the moment.
>>>>>

---

From: Jouni Korhonen <[email-masked]>
Date: Mon, 7 Mar 2016 10:30:33 -0800

Pascal,

Sequence numbers can potentially have other useful uses than reordering 
and detecting duplicates. However, then again those start to be 
application/use case specific and probably better be done on top of Det 
layer (yes, ~redundant information on multiple layers). For example, a 
sequence number could also be associated to the specific time event.

- Jouni

---

From: Gregory Mirsky <[email-masked]>
Date: Mon, 7 Mar 2016 19:03:32 +0000

Hi Janos, et. al,

apologies for the belated comments on the new Section 3:

·         Edge nodes of a network domain are the peers at the Det layer. I'd characterize Det as overlay network so that it can have internal nodes to the transport layer, underlay

·         "Core nodes of a network domain are involved in the Net layer." I think that transport is the layer on its own and its relationship to the physical network is not known and, I think, is not important.

·         Figure 1.

o   Are the lists to give example or considered to be exhaustive?

o   if we define Det layer as distinctive layer with its own characteristics and, I believe, encapsulation, then shouldn't it be empty in the figure?

·         "Both the Det and the Net layers are provided by a single technology ..." I think that this is too strong statement and may be better positioned as question that would be investigated and concluded upon later.

·         "Different criteria apply for the Det layer and the Net layer, however, some of the criteria are valid for both layers." I think that "criteria" may be interpreted as "requirements" and lists that follow are these requirements. Is that so?

·         "Operations and management (capabilities)" is this reference to 'F' and 'P' in the FCAPS? In that case, OAM is more common acronym to refer to Fault Management and Performance Measurement.

·         "Time synchronization (e.g., time stamping)" time stamping does require any degree of clock synchronization.

·         "#8 Class and quality of service capabilities (Det specific)" Class of Service is marking while Quality of Service is shaping and policing, scheduling policies. CoS is request while QoS is actual realization of PHB.

Regards,

        Greg

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of János Farkas
Sent: Monday, March 07, 2016 4:40 AM
To: Pascal Thubert (pthubert); Jouni Korhonen
Cc: [email-masked]
Subject: Re: [Detnet-dp-dt] DetNet layering

Hi Jouni and Pascal,

It is a valid approach to have "#5 Packet replication and deletion (for seamless redundancy)" in the Det layer (as well). I'm open. ;-)

The idea behind the current text is that #3 Packet sequencing belongs to the Det layer. However, the functionality of replicating packets and eliminating duplicates, i.e. #5, is rather a networking functionality, that's why it was added to the Net layer. Nonetheless, #5 relies on #3.

That is, the function of #5 even being in Net layer, has to use attribute in the Det layer, i.e. #3. I admit it is not that clean. The Det and the Net layers are closely related, we cannot draw a hard line between them, that's why the different line in the figure. It is not only "#5 Packet replication and deletion" that need to leverage both layers.

Thanks,

Janos

>> Le 7 mars 2016 à 02:45, Jouni Korhonen <[email-masked]<mailto:[email-masked]>> a écrit :

>> * I would argue "#5 Packet replication and deletion (for seamless redundancy)" also belongs to Det layer, depending on which layer the detnet stream and duplication detection is done in the first place.

>>> On 04 Mar 2016, at 04:50, János Farkas <[email-masked]<mailto:[email-masked]>> wrote:

>>>> - Net layer: provides the networking between the DetNet flow

>>>> endpoints

>>>> - Stream identification: Stream-ID = Flow-ID + Egress-ID For

>>>> example, for PW over MPLS "Flow-ID=PW-label" and "Egress-ID=LSP-label-of-egress-node".

>>>> aspects to the Net-layer For example, for PW over MPLS "Det-specific = TC-field-of-PW-label" and "Net-specific = TC-field of-LSP-labels".

>>>> [email-masked]<mailto:[email-masked]>

>>> [email-masked]<mailto:[email-masked]>

>> [email-masked]<mailto:[email-masked]>

[email-masked]<mailto:[email-masked]>

---

From: János Farkas <[email-masked]>
Date: Tue, 8 Mar 2016 15:58:20 +0100

Hi Jouni,

We can have #5 at both layers, fine by me.

Regards,
Janos

On 3/7/2016 7:24 PM, Jouni Korhonen wrote:
>> has to use attribute in the Det layer, i.e. #3. I admit it is not 
>> that clean. The Det and the Net layers are closely related, we cannot 
>> draw a hard line between them, that's why
>>>> redundancy)” also belongs to Det layer, depending on which layer 
>>>> the detnet stream and duplication detection is done in the first 
>>>> place.
>>>>
>>>>
>>>> - Jouni
>>>>
>>>>
>>>>>> data plane analysis, no intent to change it. It was well 
>>>>>> discussed and concluded.
>>>>>>

---

From: János Farkas <[email-masked]>
Date: Tue, 8 Mar 2016 15:59:24 +0100

Hi Pascal and Jouni,

I agree with Jouni, sequence number can have multiple purpose.

Regards,
Janos

On 3/7/2016 7:30 PM, Jouni Korhonen wrote:
> Pascal,
>
> Sequence numbers can potentially have other useful uses than 
> reordering and detecting duplicates. However, then again those start 
> to be application/use case specific and probably better be done on top 
> of Det layer (yes, ~redundant information on multiple layers). For 
> example, a sequence number could also be associated to the specific 
> time event.
>
> - Jouni
>
>> quick question: do you see another application to the sequence like 
>> reordering after balancing over non equally latent paths? If yes then 
>> sequencing is a tool of multiple usages. If no then sequencing is a 
>> tool that is a component of the elimination function. In both case 
>> they are of different nature, tool vs. function.
>>
>> Take care,
>>
>> Pascal
>>
>>> The idea behind the current text is that #3 Packet sequencing 
>>> belongs to the Det layer. However, the functionality of replicating 
>>> packets and eliminating duplicates, i.e. #5, is rather a networking 
>>> functionality, that's why it was added to the Net layer. 
>>> Nonetheless, #5 relies on #3. That is, the function of #5 even being 
>>> in Net layer, has to use attribute in the Det layer, i.e. #3. I 
>>> admit it is not that clean. The Det and the Net layers are closely 
>>> related, we cannot draw a hard line between them, that's why the 
>>> different line in the figure. It is not only "#5 Packet replication 
>>> and deletion" that need to leverage both layers.
>>>
>>> Nevertheless, I'm open to change. We can move #5 to the Det layer, 
>>> or have it at both the Det and the Net layers.
>>>
>>> Thanks,
>>> Janos
>>>
>>>
>>>>>> Initial text on layering has been added to the draft in github. 
>>>>>> It is an individual section; Section 3 for now.
>>>>>> Review please.
>>>>>>
>>>>>> Cheers,
>>>>>> Janos
>>>>>>
>>>>>>
>>>>>>> - Time sync: Det-layer can add a time stamp to the packets of 
>>>>>>> the flow, whereas Net-layer provides time/phase sync among the 
>>>>>>> network nodes.
>>>>>>>

---

From: János Farkas <[email-masked]>
Date: Tue, 8 Mar 2016 16:36:08 +0100

Hi Greg,

On 3/7/2016 8:03 PM, Gregory Mirsky wrote:
> ·Edge nodes of a network domain are the peers at the Det layer. I’d 
> characterize Det as overlay network so that it can have internal nodes 
> to the transport layer, underlay
>
A true overlay provides exactly what is stated: the edge nodes are peers.
We can of course remove the sentence if it is confusing.

> ·“Core nodes of a network domain are involved in the Net layer.” I 
> think that transport is the layer on its own and its relationship to 
> the physical network is not known and, I think, is not important.
>
I agree.
The point was that core nodes take part in routing, it cannot be only 
done by the edge nodes, unless they are adjacent neighbors.
We can delete the sentence if it is confusing.

> ·Figure 1.
>
> oAre the lists to give example or considered to be exhaustive?
>
just examples

> oif we define Det layer as distinctive layer with its own 
> characteristics and, I believe, encapsulation, then shouldn’t it be 
> empty in the figure?
>
I think it should not be empty.
In my understanding, the goal is to evaluate and investigate how much 
existing data plane can support DetNet flows.
The idea behind introducing the Det layer and Net layer is to make that 
exercise easier, or at least easier to document. Our analysis so far has 
shown that there is no single technology that can provide all that is 
needed for DetNet. MPLS is not enough on its own, the same holds for 
IPv4, IPv6, PW, etc. We ended up in the discussion many times "but if we 
combine it with ...". It seems that combinations are promising. However, 
we cannot really document each possible permutation. Distinguishing Det 
layer from Net layer helps to address combinations more easy, and 
identify which feature to look where.

> ·“Both the Det and the Net layers are provided by a single technology 
> …” I think that this is too strong statement and may be better 
> positioned as question that would be investigated and concluded upon 
> later.
>
This is just my bad phrasing.
The intention was to express that in some solutions Net layer is 
provided by a single technology. In some solutions the Net layer is 
provided by a single technology. However, in some solutions multiple 
technologies are used to provide either the Net layer or the Det layer, 
or both.
How about rephrasing the sentence to:
"The Net layer can be provided by a single technology in some solutions. 
The Det layer can be provided by a single technology in some solutions. 
For example, the Det layer is PW and the Net layer is MPLS in case of PW 
over MPLS."

> ·“Different criteria apply for the Det layer and the Net layer, 
> however, some of the criteria are valid for both layers.” I think that 
> “criteria” may be interpreted as “requirements” and lists that follow 
> are these requirements. Is that so?
>
Section 3 just follows Section 2, which uses "criteria" right now.

> ·“Operations and management (capabilities)” is this reference to ‘F’ 
> and ‘P’ in the FCAPS? In that case, OAM is more common acronym to 
> refer to Fault Management and Performance Measurement.
>
Section 3 just follows Section 2, which uses "#6 Operations and 
management" at the moment. If Section 2 gets updated, then we need to 
update Section 3 accordingly. (if we keep Section 3)

> ·“Time synchronization (e.g., time stamping)” time stamping does 
> require any degree of clock synchronization.
>
Yes.
The idea was that time synchronization has multiple aspects. Clock 
synchronization belongs to the Net layer. Time stamping rather belongs 
to the Det layer, I think.

> ·“#8 Class and quality of service capabilities (Det specific)” Class 
> of Service is marking while Quality of Service is shaping and 
> policing, scheduling policies. CoS is request while QoS is actual 
> realization of PHB.
>
Making it clear is the job of Section 2.

Thanks and regards,
Janos

> #5 relies on #3.
> >> Le 7 mars 2016 à 02:45, Jouni Korhonen <[email-masked] 
> <mailto:[email-masked]>> a écrit :
>
> >>
>
> >>
>
> >> Janos,
>
> >>
>
> >>> On 04 Mar 2016, at 04:50, János Farkas <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> >>>
>
> >>> Hi,
>
> >>>
>
> data plane analysis, no intent to change it. It was well discussed and 
> concluded.
>
> >>>>
>
> >>>> aspects to the Net-layer For example, for PW over MPLS 
> "Det-specific = TC-field-of-PW-label" and "Net-specific = TC-field 
> of-LSP-labels".
>
> >>>> [email-masked] <mailto:[email-masked]>
>
> >>> [email-masked] <mailto:[email-masked]>
>
> >> [email-masked] <mailto:[email-masked]>
>
> [email-masked] <mailto:[email-masked]>
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 8 Mar 2016 15:49:01 +0000

I thought that too Janos, so we agree.

The point I was making is that we are trying to segregate things in layer and hitting a snag.
The suggestion was that maybe the snag is that we are not comparing apple to apple. 

We are comparing a signaling tool (here a sequence number) with a functionality (here packet redundancy). 

If we accept that the tool is useful for multiple functionalities, then we can accept that this functionalities are of different layers. Which effectively allows us to place replication and elimination at the net layer when other usages of the sequence may be of the det layer or even as jouni suggests at the app layer.

With this understanding, I'm happy to converge with your initial thought that replication, retry, elimination and reordering belong to the net layer.

Does that help?

Pascal

> From: János Farkas [mailto:[email-masked]]
> Sent: mardi 8 mars 2016 15:59
> To: Jouni Korhonen <[email-masked]>; Pascal Thubert
> (pthubert) <[email-masked]>
> >>> (for seamless redundancy)" in the Det layer (as well). I'm open. ;-)
> >>>
> >>>>> redundancy)" also belongs to Det layer, depending on which layer

---

From: Gregory Mirsky <[email-masked]>
Date: Tue, 8 Mar 2016 21:14:48 +0000

Hi Janos,
thank you for the most detailed responses. Please find my follow-up notes in-line and tagged GIM>>.

                Regards,
                                Greg

From: Janos Farkas
Sent: Tuesday, March 08, 2016 7:36 AM
To: Gregory Mirsky
Cc: Pascal Thubert (pthubert); Jouni Korhonen; [email-masked]
Subject: Re: [Detnet-dp-dt] DetNet layering

Hi Greg,

Hi Janos, et. al,

·         Edge nodes of a network domain are the peers at the Det layer. I'd characterize Det as overlay network so that it can have internal nodes to the transport layer, underlay
A true overlay provides exactly what is stated: the edge nodes are peers.
We can of course remove the sentence if it is confusing.
GIM>> peer-to-peer may be the most common overlay but BIER and SFC are examples of more generic overlay that includes transient overlay nodes.

·         "Core nodes of a network domain are involved in the Net layer." I think that transport is the layer on its own and its relationship to the physical network is not known and, I think, is not important.
I agree.
The point was that core nodes take part in routing, it cannot be only done by the edge nodes, unless they are adjacent neighbors.
We can delete the sentence if it is confusing.
GIM>> There could be several models to provide DetNet functionality. If underlay supports all requirements, then there may not be need for intermediate nodes to support DetNet. In case transport layer provides only part of required services, e.g. replication but not elimination of duplicates, then intermediate nodes with DetNet functionality may be reasonable alternative.

·         Figure 1.

o   if we define Det layer as distinctive layer with its own characteristics and, I believe, encapsulation, then shouldn't it be empty in the figure?
I think it should not be empty.
In my understanding, the goal is to evaluate and investigate how much existing data plane can support DetNet flows.
The idea behind introducing the Det layer and Net layer is to make that exercise easier, or at least easier to document. Our analysis so far has shown that there is no single technology that can provide all that is needed for DetNet. MPLS is not enough on its own, the same holds for IPv4, IPv6, PW, etc. We ended up in the discussion many times "but if we combine it with ...". It seems that combinations are promising. However, we cannot really document each possible permutation. Distinguishing Det layer from Net layer helps to address combinations more easy, and identify which feature to look where.
GIM>> I think that what would be in Det layer depends upon services provided by the Net, transport layer. For example, BIER supports replication but not elimination of duplicates at pre-determined nodes in the domain. Then, in my view, sequence number handling is in Det layer since Net layer, e.g. BIER is not acting on it at all. And similarly for time stamp, mandatory or optional. It may be more in place in Det layer as transport compliance with particular SLA requested for the given DetNet flow must be monitored by transport layer Performance Measurement OAM.

·         "Both the Det and the Net layers are provided by a single technology ..." I think that this is too strong statement and may be better positioned as question that would be investigated and concluded upon later.
This is just my bad phrasing.
The intention was to express that in some solutions Net layer is provided by a single technology. In some solutions the Net layer is provided by a single technology. However, in some solutions multiple technologies are used to provide either the Net layer or the Det layer, or both.
How about rephrasing the sentence to:
"The Net layer can be provided by a single technology in some solutions. The Det layer can be provided by a single technology in some solutions. For example, the Det layer is PW and the Net layer is MPLS in case of PW over MPLS."
GIM>> If we state that Det presents itself as distinct layer, then all listed technologies, PW, MPLS, are examples of Net, not Det, in my view. Unless, of course, we introduce Det(Net) PW.

·         "Operations and management (capabilities)" is this reference to 'F' and 'P' in the FCAPS? In that case, OAM is more common acronym to refer to Fault Management and Performance Measurement.
Section 3 just follows Section 2, which uses "#6 Operations and management" at the moment. If Section 2 gets updated, then we need to update Section 3 accordingly. (if we keep Section 3)
GIM>> We can refer to RFC 6291 in Section 2.

·         "Time synchronization (e.g., time stamping)" time stamping does require any degree of clock synchronization.
Yes.
The idea was that time synchronization has multiple aspects. Clock synchronization belongs to the Net layer. Time stamping rather belongs to the Det layer, I think.
GIM>> I agree and think that we should separate, distinguish Clock Synchronization in the network from Time Stamping (is it at the ingress only or else?) DetNet flow.

Thanks and regards,
Janos

Regards,

        Greg

-----Original Message-----
From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of János Farkas
Sent: Monday, March 07, 2016 4:40 AM
To: Pascal Thubert (pthubert); Jouni Korhonen
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet-dp-dt] DetNet layering

Hi Jouni and Pascal,

Thanks,

Janos

---

From: János Farkas <[email-masked]>
Date: Thu, 10 Mar 2016 19:35:39 +0100

Hi Jouni,

To be honest, I still have some concerns in having #5 Packet replication 
and deletion at the Det layer.

I think #3 Packet sequencing belongs to the Det layer.
#3 Packet sequencing is clearly needed for seamless redundancy.

However, #5 Packet replication and deletion is part of the Net layer.

We also need Stream ID for seamless redundancy.  Stream ID = Flow ID 
(Det layer) + Egress ID (Net Layer).

Let's take a look at PW over MPLS as an example, e.g. 
http://www.ietf.org/proceedings/94/slides/slides-94-detnet-4.pptx. ;-)
p11: says:
”stream splitting” at LSP level -> streams have the same PW label but 
different outer labels on the PSN tunnel."
In my reading it is having the Packet replication at the Net layer.
p12:
"PW Instance receives packets from multiple LSPs that have the same PW 
label."
In my reading, it is that deletion is not at PW layer, i.e., not at the 
Det layer
p16 is crucial:
"All packets on a specific PW Instance share the same PW Control Word 
Sequence Number space."
That is, sequence numbering is at PW, i.e., at Det layer.

As the replication happens at Net layer, the deletion can only happen at 
Net layer.
However, the device has to take a look at the Sequence Number being at 
the Det layer in order to be able to perform the proper deletion/merging 
action at Net layer.

My 2 cents.

Cheers,
Janos

---

From: "Norman Finn (nfinn)" <[email-masked]>
Date: Thu, 10 Mar 2016 19:06:27 +0000

I’ve got one serious concern about the layering section in the draft that was checked in this morning.

=== From the text: ===
   The criteria for the Det layer:

      #1 Encapsulation and overhead
      #2 Stream identification (Flow ID part of the DetNet stream)
      #3 Packet sequencing (sequence number)
      #6 Operations and management (capabilities)
      #7 Time synchronization (e.g., time stamping)
      #8 Class and quality of service capabilities (Det specific)
      #10 Technical maturity

   The criteria for the Net layer:

      #1 Encapsulation and overhead
      #2 Stream identification (Egress ID)
      #4 Explicit routes (network path)
      #5 Packet replication and deletion (for seamless redundancy)
      #6 Operations and management (capabilities)
      #7 Time synchronization (time/phase sync of nodes)
      #8 Class and quality of service capabilities (Net specific)
      #9 Packet traceability (verification purposes)
      #10 Technical maturity
======

We have here a problem that I ran into in the early drafts of IEEE P802.1CB — confusing duplicate deletion with the merging of the two member streams.  (I’ll use the current P802.1CB terminology, because I don’t think we’ve define one, yet — “Member Streams” are the duplicated streams that were duplicate from, and are merged into, the “Compound Stream”.)

Trying to merge our two streams of thought :) I get:

 - Packet sequencing and packet deletion are peers of each other at the Det layer.

 - Stream splitting and stream merging are peers of each other at the Net layer.

It is critical that the packet sequencing (which may add a tag or a control word or whatever) and the duplicate deletion (which is likely the thing that removes any added tag or control word) are peers.  The packet replication can be little more than normal multicast distribution by whatever means you are using for that in the data plane.  The merging of the Member Streams is a DetNet novelty for most data plane technologies.  But, that merging is not the same thing as duplicate deletion.  For example, you could merge the streams and deliver all of the packets to an end station and let the end station delete the duplicates.  At any rate, the sequencing and deleting are peers, and you can’t have one peer in Det and the other in Net.

— Norm

---

From: Jouni Korhonen <[email-masked]>
Date: Thu, 10 Mar 2016 11:09:23 -0800

Hi Janos,

Inline..

3/10/2016, 10:35 AM, János Farkas kirjoitti:
> Hi Jouni,
>
> To be honest, I still have some concerns in having #5 Packet 
> replication and deletion at the Det layer.
>
Replication at the Net layer is ok.

> In my reading, it is that deletion is not at PW layer, i.e., not at 
> the Det layer
> p16 is crucial:
> As the replication happens at Net layer, the deletion can only happen 
> at Net layer.
> the Det layer in order to be able to perform the proper 
> deletion/merging action at Net layer.
The PW instance, which has "Det layer" steroids would receive packets 
that have the same SN.. All labels are likely popped away at this point. 
What is left is the "context" set by the last popped label (in this case 
would be the PW label that also could act as a stream identifier) and 
the CW. The information from the popped labels is likely to be 
completely gone and looking ahead of your current label context is 
likely not to work. So if I want to do duplicate discard I would have 
issues to do it in other than Det layer. replication is another story 
and easier in this context and thus doable at Net layer on my egress 
pipeline.

- Jouni

>>> related, we cannot draw a hard line between them, that's why
>> preliminary peek how to actually implement some of these and it is 
>> not always clear how the layering goes in a "clean way".
>>
>> I would have it at both layers for the time being.. subject to 
>> removal once we (or I get more insight), though.
>>
>> - jouni
>>>
>>> Thanks,
>>> Janos
>>>
>>>

---

From: János Farkas <[email-masked]>
Date: Fri, 11 Mar 2016 15:02:53 +0100

Hi Norm,

I've been waiting for you to chime in as you have gone through all these 
for 802.1CB.
I agree with you.

Thanks,
Janos

On 3/10/2016 8:06 PM, Norman Finn (nfinn) wrote:
> We have here a problem that I ran into in the early drafts of IEEE 
> P802.1CB — confusing duplicate deletion with the merging of the two 
> member streams.  (I’ll use the current P802.1CB terminology, because I 
> don’t think we’ve define one, yet — “Member Streams” are the 
> duplicated streams that were duplicate from, and are merged into, the 
> “Compound Stream”.)
>
> It is critical that the packet sequencing (which may add a tag or a 
> control word or whatever) and the duplicate deletion (which is likely 
> the thing that removes any added tag or control word) are peers.  The 
> packet replication can be little more than normal multicast 
> distribution by whatever means you are using for that in the data 
> plane.  The merging of the Member Streams is a DetNet novelty for most 
> data plane technologies.  But, that merging is not the same thing as 
> duplicate deletion.  For example, you could merge the streams and 
> deliver all of the packets to an end station and let the end station 
> delete the duplicates.  At any rate, the sequencing and deleting are 
> peers, and you can’t have one peer in Det and the other in Net.
>
> — Norm

---

From: János Farkas <[email-masked]>
Date: Fri, 11 Mar 2016 15:06:50 +0100

Hi Jouni,

Well, it is not straightforwards.

Of course, you strip many headers by the edge node when a packet is 
received from the network and is about to be sent out on the UNI. I 
believe striping headers by an edge node does not move a feature from 
one layer to the above one.

For the moment, think of a duplicate eliminating point in the middle of 
a network domain, i.e. a core node. As it was used before, let's use PW 
over MPLS as an example. The core node will look into the sequence 
number provided by the PW, but will not change the PW.

Cheers,
Janos

On 3/10/2016 8:09 PM, Jouni Korhonen wrote:
>> "PW Instance receives packets from multiple LSPs that have the same 
>> PW label."
>> However, the device has to take a look at the Sequence Number being 
>> at the Det layer in order to be able to perform the proper 
> that have the same SN.. All labels are likely popped away at this 
> point. What is left is the "context" set by the last popped label (in 
> this case would be the PW label that also could act as a stream 
> identifier) and the CW. The information from the popped labels is 
> likely to be completely gone and looking ahead of your current label 
> context is likely not to work. So if I want to do duplicate discard I 
> would have issues to do it in other than Det layer. replication is 
> another story and easier in this context and thus doable at Net layer 
> on my egress pipeline.
>
> - Jouni
>
>>
>> My 2 cents.
>>
>> Cheers,
>> Janos
>>
>>
>>>> Nonetheless, #5 relies on #3. That is, the function of #5 even 
>>>> being in Net layer, has to use attribute in the Det layer, i.e. #3. 
>>>> I admit it is not that clean. The Det and the Net layers are 
>>>> closely related, we cannot draw a hard line between them, that's why
>>>>>>>> Such a structuring would make the draft more easy to follow. 
>>>>>>>> For instance, it would decrease cross-references, repeated text 
>>>>>>>> that we have at the moment.
>>>>>>>>

---

From: Gregory Mirsky <[email-masked]>
Date: Fri, 11 Mar 2016 16:04:13 +0000

Hi Norm,
I very much agree with layering of functions you’ve proposed. The one modification I’d like us to consider is when the DetNet shim/tag, i.e. Det encapsulation, gets added. If the tag carries not only sequence number but flow identifier, payload type information then I think that it would be reasonable it to be added by an ingress DetNet node and removed by egress node(s). Packet replication may be done by Det node or supported by multicast transport layer. Elimination of duplicates, per model you’ve proposed, at Det layer, but may not lead to the Det tag removal. The Det tag, i.e. encapsulation, to be removed at the egress node.
Very much interested in your opinion on this.

                Regards,

From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Norman Finn (nfinn)
Sent: Thursday, March 10, 2016 11:06 AM
To: Janos Farkas; [email-masked]
Subject: Re: [Detnet-dp-dt] DetNet layering

=== From the text: ===

— Norm

---

From: Jouni Korhonen <[email-masked]>
Date: Fri, 11 Mar 2016 11:09:21 -0800

Janos,

One thing here.. if you go back to my presentation from the last meeting 
I modelled (in case of MPLS PW) stream split/merge/duplicate elimination 
in a middle of network using multi-segment pseudowire. I have no desire 
to make a 'normal' P router in anyway DetNet specific or even aware, 
specifically when it comes to stream splitting/merging and duplicate 
elimination. (I do recon than MS-PW reference stack model does not 
bother with encapsulation layer so some changes/clarifications would be 
needed there to make to look into encapsulation, which includes things 
like sequencing, as well).

- Jouni

3/11/2016, 6:06 AM, János Farkas kirjoitti:
> For the moment, think of a duplicate eliminating point in the middle 
> of a network domain, i.e. a core node. As it was used before, let's 
> use PW over MPLS as an example. The core node will look into the 
> sequence number provided by the PW, but will not change the PW.
>
> Cheers,
> Janos
>
>
>>> ”stream splitting” at LSP level -> streams have the same PW label 
>>> but different outer labels on the PSN tunnel."
>>> "All packets on a specific PW Instance share the same PW Control 
>>> Word Sequence Number space."
>>> As the replication happens at Net layer, the deletion can only 
>>> happen at Net layer.
>>>>> belongs to the Det layer. However, the functionality of 
>>>>> replicating packets and eliminating duplicates, i.e. #5, is rather 
>>>>> a networking functionality, that's why it was added to the Net 
>>>>> layer. Nonetheless, #5 relies on #3. That is, the function of #5 
>>>>> even being in Net layer, has to use attribute in the Det layer, 
>>>>> i.e. #3. I admit it is not that clean. The Det and the Net layers 
>>>>> are closely related, we cannot draw a hard line between them, 
>>>>> that's why
>>>>>>> * I would argue "#5 Packet replication and deletion (for 
>>>>>>> seamless redundancy)” also belongs to Det layer, depending on 
>>>>>>> which layer the detnet stream and duplication detection is done 
>>>>>>> in the first place.
>>>>>>>
>>>>>>>
>>>>>>> - Jouni
>>>>>>>
>>>>>>>
>>>>>>>>> However, the criteria are well selected and appropriate for 
>>>>>>>>> our data plane analysis, no intent to change it. It was well 
>>>>>>>>> discussed and concluded.
>>>>>>>>>
>>>>>>>>> For instance, it would decrease cross-references, repeated 
>>>>>>>>> text that we have at the moment.
>>>>>>>>>

---

From: "Norman Finn (nfinn)" <[email-masked]>
Date: Mon, 14 Mar 2016 01:29:48 +0000

I agree.  The encaps would usually be added at sequencing time and removed at deleting time, but not necessarily.

— Norm

From: Gregory Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Saturday, March 12, 2016 at 00:04 AM
To: Norman Finn <[email-masked]<mailto:[email-masked]>>, János Farkas <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [Detnet-dp-dt] DetNet layering

Hi Norm,

                Regards,

From: Detnet-dp-dt [mailto:[email-masked]] On Behalf Of Norman Finn (nfinn)
Sent: Thursday, March 10, 2016 11:06 AM
To: Janos Farkas; [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet-dp-dt] DetNet layering

=== From the text: ===

— Norm