Subject: [Detnet] IPv6 encapsulation in dataplane doc

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 7 Feb 2018 13:27:09 +0000

Dear all

This is about the IPv6 encapsulation and more precisely

                        Therefore, if a DetNet-aware end system only

   inserted the DetNet Destination Option into the IPv6 but e.g., a

   DetNet Edge node is configured to enforce an explicit route for the

   IPv6 packet using a source routing header, then it has no other

   possibility than add an outer tunneling IPv6 header with required

   extension headers in it.  The processing of IPv6 packets in a DetNet

   Edge node is discussed further in Section 6.4.1<https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-01#section-6.4.1>.

With the current spec, a source sends a DetNet packet as

                    +---------------------------------+
                    |                                 |
                    |           DetNet Flow           |
                    |             Payload             |
                    |                                 |
                    /---------------------------------\
                    H   Optional DetNet DstOpt Hdr    H
                    \---------------------------------/
                    |          IPv6 header            |
                    |     (with set Flow label)       |
                    +---------------------------------+

And then the ingress node needs to re-encapsulate as

                    +---------------------------------+

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    /---------------------------------\

                    H        DetNet DstOpt Hdr        H
                    \---------------------------------/
                    |          IPv6 header            |
                    |     (with set Flow label)       |
                    +=================================+

                    |          Routing header         |

                    /---------------------------------\

                    H        DetNet DstOpt Hdr        H

                    \---------------------------------/

                    |          IPv6 header            |

                    |     (with set Flow label)       |

                    +---------------------------------+

This creates a duplication of the DetNet Destination Option.

There are alternatives

a)  whereby the packet is tunneled from the source to the detnet ingress, and based on its state the DetNet ingress accepts the packet, processes it and then resends it. The tunneled version of this could be:

                    +---------------------------------+
                    |                                 |
                    |           DetNet Flow           |
                    |             Payload             |
                    |                                 |
                    +---------------------------------+
                    |          IPv6 header            |
                    |   (dest = final destination)    |
                    /=================================\
                    H   Optional DetNet DstOpt Hdr    H
                    \---------------------------------/
                    |          IPv6 header            |
                    |   (dest = DetNet ingress edge)  |
                    |     (with set Flow label)       |
                    +---------------------------------+

Which allows the ingress to tunnel to the egress as follows:
                    +---------------------------------+
                    |                                 |
                    |           DetNet Flow           |
                    |             Payload             |
                    |                                 |
                    +---------------------------------+
                    |          IPv6 header            |
                    |      (to final destination)     |
                    +=================================+

                    |          Routing header         |

                    /---------------------------------\
                    H   Optional DetNet DstOpt Hdr    H
                    \---------------------------------/
                    |          IPv6 header            |
                    |   (dest = DetNet egress edge)   |
                    |     (with set Flow label)       |
                    +---------------------------------+

b)  whereby the PREF is done by the end nodes and the tunnel is transport only, meaning that there are 2 tunnels A and B and that the source sends twice a packet like this:

                    +---------------------------------+
                    |                                 |
                    |           DetNet Flow           |
                    |             Payload             |
                    |                                 |
                    /---------------------------------\
                    H   Optional DetNet DstOpt Hdr    H
                    \---------------------------------/
                    |          IPv6 header            |
                    |   (dest = DetNet ingress edge X)|
                    |     (with set Flow label)       |
                    +---------------------------------+

And then the ingress node needs to re-encapsulate as

                    +---------------------------------+

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    /---------------------------------\

                    H        DetNet DstOpt Hdr        H
                    \---------------------------------/
                    |          IPv6 header            |
                    |   (dest = final destination)    |
                    |     (with set Flow label)       |
                    +=================================+

                    |          Routing header         |

                    +---------------------------------+

                    |          IPv6 header            |

                    |   (dest = DetNet egress edge X) |

                    |     (with set Flow label)       |

                    +---------------------------------+

Cheers,

Pascal

---

From: Lou Berger <[email-masked]>
Date: Wed, 07 Feb 2018 09:59:42 -0500

Pascal/all,

At the last interim a proposal was made to simplify IP processing, at least 
for the initial detnet solution, by leaving PREF to the subnet/transport 
layers (i.e., TSN and MPLS) and providing DetNet flow identification based 
on typical IP 5-tuple perhaps + dscp. This approach has several benefits 
beyond simplification, notably it will work for both ipv4 and IPv6, and 
doesn't require any modification to encapsulation / formats.

It would be really valuable to get feedback from the whole working group if 
this simplification is acceptable or has unacceptable limitations.

Lou

On February 7, 2018 8:28:02 AM "Pascal Thubert (pthubert)" 
<[email-masked]> wrote:

>    Edge node is discussed further in Section 
>    6.4.1<https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-01#section-6.4.1>.
>
>
> a)  whereby the packet is tunneled from the source to the detnet ingress, 
> and based on its state the DetNet ingress accepts the packet, processes it 
> and then resends it. The tunneled version of this could be:
>
> b)  whereby the PREF is done by the end nodes and the tunnel is transport 
> only, meaning that there are 2 tunnels A and B and that the source sends 
> twice a packet like this:
>
> ----------
> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet
>

---

From: "Jouni" <[email-masked]>
Date: Thu, 8 Feb 2018 10:49:16 +0200

Hi Pascal,

Thanks for reading the draft and elaborating the option handling further. In
the current draft Section 6.4.1 elaborates handling/processing of options
further than in the introductory part of the options and packet formats. The
alternative a) below is already described in the draft to some extent - thus
I fully agree. The alternative b) would also fit but there is no explicit
description for it yet - again I fully agree. The figure 11 is heading
exactly where your alternatives point at.

Thnaks,

                jouni

From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert
(pthubert)
Sent: Wednesday, February 7, 2018 15:27 PM
To: [email-masked]
Subject: [Detnet] IPv6 encapsulation in dataplane doc

Dear all

                        Therefore, if a DetNet-aware end system only
   inserted the DetNet Destination Option into the IPv6 but e.g., a
   DetNet Edge node is configured to enforce an explicit route for the
   IPv6 packet using a source routing header, then it has no other
   possibility than add an outer tunneling IPv6 header with required
   extension headers in it.  The processing of IPv6 packets in a DetNet
   Edge node is discussed further in Section 6.4.1
<https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-01#section-6.4.1> .

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

There are alternatives 

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          Routing header         |

                    |          IPv6 header            |

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

Cheers,

Pascal

---

From: "Jouni" <[email-masked]>
Date: Thu, 8 Feb 2018 11:08:09 +0200

Hi,

Yes, we discussed this to an extent in the last call. I've been thinking
this a bit more lately (cross country skiing conditions have been great thus
you got a lot of time while on the skiing tracks ;) Few concerns/points to
think I see here are:

* End systems cannot participate to DetNet service layer packet elimination
& duplication business. If they still do e.g. using DetNet DstOpt that is
separate from subnet/transport layer provided service.

* Different subnet/transport layer segments in the DetNet domain are likely
to use their own sequencing and duplication & elimination solutions. I am
not sure how independent those will be e.g., is the sequence numbering
unified across or only per segment. The former is not IMO easy and the
latter easily reduces to single points between segments to handle number
book keeping properly.  

Regardless the above I am keen to explore this alternative approach..

-          Jouni

From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: Wednesday, February 7, 2018 17:00 PM
To: Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Pascal/all,

Lou

On February 7, 2018 8:28:02 AM "Pascal Thubert (pthubert)"
<[email-masked] <mailto:[email-masked]> > wrote:

Dear all

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

There are alternatives 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          Routing header         |

                    |          IPv6 header            |

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

Cheers,

Pascal

_______________________________________________
detnet mailing list
[email-masked] <mailto:detnet%40ietf.org> 
https://www.ietf.org/mailman/listinfo/detnet

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 8 Feb 2018 10:45:55 +0000

I fully agree, Jouni ;

For your first point: In the case of wireless, PRP (a form of PREF) is a critical feature for industrial deployments; this is shipping and deployed today. The link that needs protection the most is the first hop (the UNI from/to the device) since it is over Wi-Fi. If the PREF starts at the network edge (say the controller), then most of the benefit is lost. So DetNet must enable E2E Services, in particular over IPv4 which is dominant in industrial use cases. What DetNet could bring to the art could that a single wireless transmission received by 2 APs end up on 2 parallel end-to-end paths.

For your second point: there are a number of potential issues if the sequencing is not done by the source end point. If a packet is received at one edge and not at another, then the sequences will differ over the 2 paths and reconciliation will be impossible. Packets received out of order at the ingress will be delivered out of order at the egress meaning that whatever support for in-order-delivery (IOD) is defeated.

My conclusion is that the end points must contribute to the effort. They must stream the data at a reasonable pace, and if Service Layer functions are expected from the network,  they really should indicate the flow ID and the sequence number from their perspective.

Now what can we expect from current end points?

-         Existing industrial devices can claim a rough need for a given rate but were not necessarily designed to provide guarantees for burstiness in DetNet terms. Failure to comply to DetNet terms may cause loss/reclassification by ingress filtering in the edge router, which would defeat one purpose of DetNet for reliability. For a close-to-acceptable sources, one can imagine a jitter absorption feature at the DetNet ingress edge router. But that also mean an additional latency, which may defeat another purpose of DetNet of in-time-delivery.

-         TCP is adverse to DetNet, and will not maintain the optimal rate; in any case, the sense of sequence that can be inferred from the offset can be defeated by retries.  In order to support TCP endpoints, one would probably need to devise a L4-proxy function (ala DLSW) that terminates TCP at the ingress edge and maps it into something more DetNet friendly.

-         UDP does not provide sequencing information - this is probably the use case for the DetNet Destination Option in the inner header.

-         Arguably RTP can signal both flow and sequence, but does not have network coding, does not expect multipath, and I'm not sure if the rate control work has gone beyond the research papers. Then again, packets may be lost or reclassified at the ingress edge.

-         This leaves rate control (e.g. adaptive bitrate streaming) to the application layer (typically over HTTP), meaning that each application as to cover for the network deficiencies in its own ways. Which is sad because it adds complexity to each and every application, a complexity that is not the core business of the app and could be handled in common at a lower layer.

DetNet has the power to simplify the application, remove jitter recovery buffers and the like, by transporting the near-perfect rate in near-perfect time.  It would be better that the end points contribution happens without impacting each and every time sensitive application, by providing DetNet Service Layer functionality at the L4 transport.
Flow identification (by port), IOD and duplicate elimination are classical L4-transport features (e.g. TCP). MP-TCP already distributes packets over different paths. It is not a gigantic conceptual step to place DetNet Service Layer features in a L4 transport. If a DetNet friendly transport -to be designed- adds DetNet Service Layer capabilities like PREF and/or network coding in the end points, 2 parallel DetNet Transport Layer circuits based on the discussion at the interim would enable the end-to-end service without knowing it. So 'd say that the proposal is still useful, but depends on L4 transport capabilities that would be fully transparent to the DetNet signaling and would not use the Destination Option.

Alternatively, say we want DetNet to provide the S-level functions, e.g. PREF. Considering that we expect that an edge router can read the ports, then it can figure what the L4 transport can be recognized and that for some transports, e.g. RTP and the suggested new transport, the edge router can derive a sequence from the transport header in a non-ambiguous way.

All the best,

Pascal

From: Jouni [mailto:[email-masked]]
Sent: jeudi 8 février 2018 10:08
To: 'Lou Berger' <[email-masked]>; Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: RE: [Detnet] IPv6 encapsulation in dataplane doc

Hi,

Yes, we discussed this to an extent in the last call. I've been thinking this a bit more lately (cross country skiing conditions have been great thus you got a lot of time while on the skiing tracks ;) Few concerns/points to think I see here are:
* End systems cannot participate to DetNet service layer packet elimination & duplication business. If they still do e.g. using DetNet DstOpt that is separate from subnet/transport layer provided service.
* Different subnet/transport layer segments in the DetNet domain are likely to use their own sequencing and duplication & elimination solutions. I am not sure how independent those will be e.g., is the sequence numbering unified across or only per segment. The former is not IMO easy and the latter easily reduces to single points between segments to handle number book keeping properly.

-         Jouni

From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: Wednesday, February 7, 2018 17:00 PM
To: Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Pascal/all,

At the last interim a proposal was made to simplify IP processing, at least for the initial detnet solution, by leaving PREF to the subnet/transport layers (i.e., TSN and MPLS) and providing DetNet flow identification based on typical IP 5-tuple perhaps + dscp. This approach has several benefits beyond simplification, notably it will work for both ipv4 and IPv6, and doesn't require any modification to encapsulation / formats.

It would be really valuable to get feedback from the whole working group if this simplification is acceptable or has unacceptable limitations.

Lou

On February 7, 2018 8:28:02 AM "Pascal Thubert (pthubert)" <[email-masked]<mailto:[email-masked]>> wrote:
Dear all

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    H        DetNet DstOpt Hdr        H

                    |          Routing header         |

                    H        DetNet DstOpt Hdr        H

                    |          IPv6 header            |

There are alternatives

                    |          Routing header         |

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    H        DetNet DstOpt Hdr        H

                    |          Routing header         |

                    |          IPv6 header            |

Cheers,

Pascal
_______________________________________________
detnet mailing list
[email-masked]<mailto:detnet%40ietf.org>
https://www.ietf.org/mailman/listinfo/detnet

---

From: "Jouni" <[email-masked]>
Date: Thu, 8 Feb 2018 16:36:16 +0200

Hi Pascal,

Thanks for good input (again) and I agree in most parts – I do have
reservations for L4 solution since it would be per L4 protocol. One
clarification to my ”first point”. In my example the not-mentioned
assumption was that the edge node is the first node doing duplication &
elimination at the subnet/transport layer. If that DetNet subnet/transport
layer can extended all way to end systems (like with current TSN in 802.1
space) then some of the issues are relaxed.

-          Jouni

From: Pascal Thubert (pthubert) [mailto:[email-masked]] 
Sent: Thursday, February 8, 2018 12:46 PM
To: Jouni <[email-masked]>; 'Lou Berger' <[email-masked]>;
[email-masked]
Subject: RE: [Detnet] IPv6 encapsulation in dataplane doc

I fully agree, Jouni ;

For your first point: In the case of wireless, PRP (a form of PREF) is a
critical feature for industrial deployments; this is shipping and deployed
today. The link that needs protection the most is the first hop (the UNI
from/to the device) since it is over Wi-Fi. If the PREF starts at the
network edge (say the controller), then most of the benefit is lost. So
DetNet must enable E2E Services, in particular over IPv4 which is dominant
in industrial use cases. What DetNet could bring to the art could that a
single wireless transmission received by 2 APs end up on 2 parallel
end-to-end paths.

For your second point: there are a number of potential issues if the
sequencing is not done by the source end point. If a packet is received at
one edge and not at another, then the sequences will differ over the 2 paths
and reconciliation will be impossible. Packets received out of order at the
ingress will be delivered out of order at the egress meaning that whatever
support for in-order-delivery (IOD) is defeated.

My conclusion is that the end points must contribute to the effort. They
must stream the data at a reasonable pace, and if Service Layer functions
are expected from the network,  they really should indicate the flow ID and
the sequence number from their perspective.

-          Existing industrial devices can claim a rough need for a given
rate but were not necessarily designed to provide guarantees for burstiness
in DetNet terms. Failure to comply to DetNet terms may cause
loss/reclassification by ingress filtering in the edge router, which would
defeat one purpose of DetNet for reliability. For a close-to-acceptable
sources, one can imagine a jitter absorption feature at the DetNet ingress
edge router. But that also mean an additional latency, which may defeat
another purpose of DetNet of in-time-delivery.

-          TCP is adverse to DetNet, and will not maintain the optimal rate;
in any case, the sense of sequence that can be inferred from the offset can
be defeated by retries.  In order to support TCP endpoints, one would
probably need to devise a L4-proxy function (ala DLSW) that terminates TCP
at the ingress edge and maps it into something more DetNet friendly.

-          UDP does not provide sequencing information - this is probably
the use case for the DetNet Destination Option in the inner header.

-          Arguably RTP can signal both flow and sequence, but does not have
network coding, does not expect multipath, and I’m not sure if the rate
control work has gone beyond the research papers. Then again, packets may be
lost or reclassified at the ingress edge.

-          This leaves rate control (e.g. adaptive bitrate streaming) to the
application layer (typically over HTTP), meaning that each application as to
cover for the network deficiencies in its own ways. Which is sad because it
adds complexity to each and every application, a complexity that is not the
core business of the app and could be handled in common at a lower layer. 

DetNet has the power to simplify the application, remove jitter recovery
buffers and the like, by transporting the near-perfect rate in near-perfect
time.  It would be better that the end points contribution happens without
impacting each and every time sensitive application, by providing DetNet
Service Layer functionality at the L4 transport.

Flow identification (by port), IOD and duplicate elimination are classical
L4-transport features (e.g. TCP). MP-TCP already distributes packets over
different paths. It is not a gigantic conceptual step to place DetNet
Service Layer features in a L4 transport. If a DetNet friendly transport -to
be designed- adds DetNet Service Layer capabilities like PREF and/or network
coding in the end points, 2 parallel DetNet Transport Layer circuits based
on the discussion at the interim would enable the end-to-end service without
knowing it. So ‘d say that the proposal is still useful, but depends on L4
transport capabilities that would be fully transparent to the DetNet
signaling and would not use the Destination Option. 

Alternatively, say we want DetNet to provide the S-level functions, e.g.
PREF. Considering that we expect that an edge router can read the ports,
then it can figure what the L4 transport can be recognized and that for some
transports, e.g. RTP and the suggested new transport, the edge router can
derive a sequence from the transport header in a non-ambiguous way. 

All the best,

Pascal

From: Jouni [mailto:[email-masked]] 
Sent: jeudi 8 février 2018 10:08
To: 'Lou Berger' <[email-masked] <mailto:[email-masked]> >; Pascal
Thubert (pthubert) <[email-masked] <mailto:[email-masked]> >;
[email-masked] <mailto:[email-masked]> 
Subject: RE: [Detnet] IPv6 encapsulation in dataplane doc

Hi,

Yes, we discussed this to an extent in the last call. I’ve been thinking
this a bit more lately (cross country skiing conditions have been great thus
you got a lot of time while on the skiing tracks ;) Few concerns/points to
think I see here are:

-          Jouni

From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: Wednesday, February 7, 2018 17:00 PM
To: Pascal Thubert (pthubert) <[email-masked]
<mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Pascal/all,

Lou

Dear all

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

There are alternatives 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          Routing header         |

                    |          IPv6 header            |

 

                    |                                 |

                    |           DetNet Flow           |

                    |             Payload             |

                    |                                 |

                    |          IPv6 header            |

                    |          IPv6 header            |

                    |          Routing header         |

Cheers,

Pascal

---

From: Lou Berger <[email-masked]>
Date: Fri, 9 Feb 2018 11:23:56 -0500

Hi Jouni,

On 02/08/2018 04:08 AM, Jouni wrote:
> this a bit more lately (cross country skiing conditions have been great
> thus you got a lot of time while on the skiing tracks ;) Few
> concerns/points to think I see here are:

I'm jealous!

> 
> * End systems cannot participate to DetNet service layer packet
> elimination & duplication business. If they still do e.g. using DetNet
> DstOpt that is separate from subnet/transport layer provided service.

True, end station participation in PREF wouldn't be possible in this
simplified approach for IP, but the end stations *could* still
participate in the equivalent function provided by a sub-net layer such
as TSN. So end to end detnet traffic protection is not possible, but it
doesn't mean it can't be provided by lower network layers.  The gain
here is a simplified and unified IP solution and simplified host
processing/support, albeit with a loss of one of the three end to end
detnet service attributes.

> * Different subnet/transport layer segments in the DetNet domain are
> likely to use their own sequencing and duplication & elimination
> solutions. I am not sure how independent those will be e.g., is the
> sequence numbering unified across or only per segment. The former is not
> IMO easy and the latter easily reduces to single points between segments
> to handle number book keeping properly.  

Agreed.  The question for the WG is this a reasonable compromise in
order to in our *initial* solution defined by the WG.  -- Nothing says
we can't come back and improve on this in the future, but it will allow
us to get *something* useful defined with less complexity.

> Regardless the above I am keen to explore this alternative approach..

Great - look forward to hear the results!
Lou
  
> 
> -          Jouni
> 
>  
> 
>  
> 
> *From:* detnet [mailto:[email-masked]] *On Behalf Of *Lou Berger
> *Sent:* Wednesday, February 7, 2018 17:00 PM
> *To:* Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
> *Subject:* Re: [Detnet] IPv6 encapsulation in dataplane doc
> 
>  
> 
> Pascal/all,
> 
> At the last interim a proposal was made to simplify IP processing, at
> least for the initial detnet solution, by leaving PREF to the
> subnet/transport layers (i.e., TSN and MPLS) and providing DetNet flow
> identification based on typical IP 5-tuple perhaps + dscp. This approach
> has several benefits beyond simplification, notably it will work for
> both ipv4 and IPv6, and doesn't require any modification to
> encapsulation / formats.
> 
> It would be really valuable to get feedback from the whole working group
> if this simplification is acceptable or has unacceptable limitations.
> 
> Lou
> 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Dear all
> 
>      
> 
>     <https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-01#section-6.4.1>.
> 
>      
> 
>      
> 
>     a)  whereby the packet is tunneled from the source to the detnet
>     ingress, and based on its state the DetNet ingress accepts the
>     packet, processes it and then resends it. The tunneled version of
>     this could be:
> 
>      
> 
>     b)  whereby the PREF is done by the end nodes and the tunnel is
>     transport only, meaning that there are 2 tunnels A and B and that
>     the source sends twice a packet like this:
> 
>      
>

---

From: Jouni <[email-masked]>
Date: Sun, 11 Feb 2018 22:06:02 +0200

Hi,

Inline..

> On 09 Feb 2018, at 18:23, Lou Berger <[email-masked]> wrote:
> 
> Hi Jouni,
> 
> 

Yes, that’s true (as I already clarified in my latest mail to Pascal).

> us to get *something* useful defined with less complexity.

I would first specify the required steps that make IP as-is work within one subnet/transport layer segment. That should be an achievable goal with a limitation of e.g. a single interconnection point between subnet/transport layer segments of different types/solutions. 

After that look into solving multiple interconnection points.. and there the L4 path that Pascal mentioned or that IPv6 DetNet DstOpt of mine could be way forward.

- Jouni

---

From: Lou Berger <[email-masked]>
Date: Sun, 11 Feb 2018 16:26:51 -0500

hi,

> Yes, that’s true (as I already clarified in my latest mail to Pascal).

yes.  It's good for all to remember that DetNet is just as much (if not
more) about supporting explicit routes with congestion protection and
latency control....

> I would first specify the required steps that make IP as-is work within one subnet/transport layer segment. That should be an achievable goal with a limitation of e.g. a single interconnection poin…

sure. as long as one of the 'attached stations' can be a router.

> After that look into solving multiple interconnection points.. and there the L4 path that Pascal mentioned or that IPv6 DetNet DstOpt of mine could be way forward.

yes, this would be good follow on work...

Cheers,
Lou

---

From: Jouni <[email-masked]>
Date: Sun, 11 Feb 2018 23:42:53 +0200

> more) about supporting explicit routes with congestion protection and

Indeed..

> sure. as long as one of the 'attached stations' can be a router.

Yes. That’s the plan at least from my side.

- Juoni

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 12 Feb 2018 10:30:00 +0000

Hello Jouni and Lou:

Just to make sure we are on the same line:

- IPvX (==v4/v6) would signal the flow implicitly - derived from the tuples- but not the source sequencing. No need to change IP.

- DetNet may provide PREF/IOD from the edge ingress. DetNet would provide its own sequencing based on reception order at ingress edge.

- The  ingress edge may be a NIC card in the originating node. It would be operating below IPvX and possibly equipped with 2 Ethernet ports and capable of PREF.

- Anything above IP is beyond charter. A upper layer sequence may be visible in the L4-transport or at the application layer but DetNet will not look for it. A snooping edge device doing DPI may dig in the packet and find information for packet identification and ordering, for a particular application, and that is beyond the interests of the WG.

- A L4 transport may be designed to provide PREF, IOD, FEC (network coding) and whatever else needed. This would be done in TSV Area with DetNet's blessing.

Do we agree?

Pascal

-----Original Message-----
From: Jouni [mailto:[email-masked]] 
Sent: dimanche 11 février 2018 22:43
To: Lou Berger <[email-masked]>
Cc: Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

>>>> elimination & duplication business. If they still do e.g. using 
>>>> DetNet DstOpt that is separate from subnet/transport layer provided service.
>>> 
>>> participate in the equivalent function provided by a sub-net layer 
>>> such as TSN. So end to end detnet traffic protection is not 
>>> possible, but it doesn't mean it can't be provided by lower network 
>>> layers.  The gain here is a simplified and unified IP solution and 
>>> simplified host processing/support, albeit with a loss of one of the 
>>> three end to end detnet service attributes.
>> 
>> Yes, that's true (as I already clarified in my latest mail to Pascal).
>> 
> 

Indeed..

>>>> * Different subnet/transport layer segments in the DetNet domain 
>>>> are likely to use their own sequencing and duplication & 
>>>> elimination solutions. I am not sure how independent those will be 
>>>> e.g., is the sequence numbering unified across or only per segment. 
>>>> The former is not IMO easy and the latter easily reduces to single 
>>>> points between segments to handle number book keeping properly.
>>> 
>>> order to in our *initial* solution defined by the WG.  -- Nothing 
>>> says we can't come back and improve on this in the future, but it 
>>> will allow us to get *something* useful defined with less complexity.
>> 

Yes. That's the plan at least from my side.

- Juoni

>>>> At the last interim a proposal was made to simplify IP processing, 
>>>> at least for the initial detnet solution, by leaving PREF to the 
>>>> subnet/transport layers (i.e., TSN and MPLS) and providing DetNet 
>>>> flow identification based on typical IP 5-tuple perhaps + dscp. 
>>>> This approach has several benefits beyond simplification, notably 
>>>> it will work for both ipv4 and IPv6, and doesn't require any 
>>>> modification to encapsulation / formats.
>>>> 
>>>> It would be really valuable to get feedback from the whole working 
>>>> group if this simplification is acceptable or has unacceptable limitations.
>>>> 
>>>> Lou
>>>>

---

From: Lou Berger <[email-masked]>
Date: Mon, 12 Feb 2018 11:25:57 -0500

Hi Pascal,

> - IPvX (==v4/v6) would signal the flow implicitly - derived from the tuples- but not the source sequencing. No need to change IP.
if you substitute "encode" with "signal", I agree.

> - DetNet may provide PREF/IOD from the edge ingress. DetNet would provide its own sequencing based on reception order at ingress edge.
I'm not sure edge ingress is right here.  an IP host may still be a 
DetNet edge, but PREF may on start at the DetNet/MPLS ingress.

> - The  ingress edge may be a NIC card in the originating node. It would be operating below IPvX and possibly equipped with 2 Ethernet ports and capable of PREF.
are you talking 802.1cb or detnet PREF? if the former, yes, if the 
latter this is FFS for IP only detnet.

> - Anything above IP is beyond charter. A upper layer sequence may be visible in the L4-transport or at the application layer but DetNet will not look for it.
Yes (from my personal view)
> A snooping edge device doing DPI may dig in the packet and find information for packet identification and ordering, for a particular application, and that is beyond the interests of the WG.
I don't think there has any proposal for this discussed in the WG either 
way.

> - A L4 transport may be designed to provide PREF, IOD, FEC (network coding) and whatever else needed. This would be done in TSV Area with DetNet's blessing.
agreed, as would be any changes in TCP congestion control modes to take 
advantage of DetNet allocations.

> Do we agree?
I hope so!

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 12 Feb 2018 16:36:36 +0000

Hi Pascal,

I have a lot of sympathy on your view.
- OK: "DetNet would provide its own sequencing based on reception order at ingress edge."
- OK: "ingress edge may be a NIC card"
- OK: "avoid DPI"

However I have some questions for clarification on your final conclusions:
- Why no seq.num in IP? That would mean no PREF, no IOD function within the DetNet domain
Which is an IP PSN.
- What would be the gain from a "L4 transport designed to provide PREF, IOD, FEC, ... "?
I mean that would be a duplicate, as in the DetNet network You would have the same function.
Checking L4 within the DetNet domain would require DPI, what I guess we do not want.
- I am also somewhat confused by your WiFi example. I mean in TSN/DetNet scenarios 
end-systems are always assumed to be in sync with their controller(s). So I do not agree with
this statement: "Existing industrial devices can claim a rough need for a given rate but were 
not necessarily designed to provide guarantees for burstiness in DetNet terms." 

Thanks & Cheers
Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: 2018. február 12. 11:30
To: Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Hello Jouni and Lou:

Do we agree?

Pascal

-----Original Message-----
From: Jouni [mailto:[email-masked]] 
Sent: dimanche 11 février 2018 22:43
To: Lou Berger <[email-masked]>
Cc: Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Indeed..

- Juoni

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 12 Feb 2018 14:14:30 -0500

Lou and Pascal,

On the most recent interim call, we discussed an encapsulation for IPv4 and
IPv6 that reused the MPLS encapsulation inside UDP inside IP (ala RFC
7520). This works for both v4 and v6, obviously.  Or at least that’s how I
interpreted the conversation at the time and the following notes from the
Etherpad:

Jouni: Good point. Prob with IP is have been narrow focused on carrying
DetNet info. But if can assume that in a DetNet domain for routed traffic,
transport below is TSN or equivalent, e.g. MPLS, then what we do at IP
layer is just routing or mapping flows into MPLS tunnels. An underlay and
then just do routing.
Lou: Is that assuming that PREF always done by e.g. TSN layer?
Jouni: Yes.
Jouni: Then starts looking like over PWs as we discussed earlier.
Lou: Others find this approach acceptable? This proposal fits within our
charter and process, and is more like what some hoped to see from our
process. Do we, the people building the solution, like this approach or
find it objectionable?
Jouni: Starting to like this idea.
Lou: So next rev of draft will propose largely common approach on v4 and
v6, i.e. 5-tuple, depends on TSN and MPLS to provide transport and PREF
functions?

So is this still the case, and what we should expect to see in the next
revision of the draft?

Thanks,
Andy

On Mon, Feb 12, 2018 at 11:25 AM, Lou Berger <[email-masked]> wrote:

>> - IPvX (==v4/v6) would signal the flow implicitly - derived from the
>> tuples- but not the source sequencing. No need to change IP.
>>
> - DetNet may provide PREF/IOD from the edge ingress. DetNet would provide
>> its own sequencing based on reception order at ingress edge.
>>
> I'm not sure edge ingress is right here.  an IP host may still be a DetNet
> edge, but PREF may on start at the DetNet/MPLS ingress.
>
> - The  ingress edge may be a NIC card in the originating node. It would be
>> operating below IPvX and possibly equipped with 2 Ethernet ports and
>> capable of PREF.
>>
> are you talking 802.1cb or detnet PREF? if the former, yes, if the latter
> this is FFS for IP only detnet.
>
> - Anything above IP is beyond charter. A upper layer sequence may be
>> visible in the L4-transport or at the application layer but DetNet will not
>> look for it.
>>
>> A snooping edge device doing DPI may dig in the packet and find
>> information for packet identification and ordering, for a particular
>> application, and that is beyond the interests of the WG.
>>
> - A L4 transport may be designed to provide PREF, IOD, FEC (network
>> coding) and whatever else needed. This would be done in TSV Area with
>> DetNet's blessing.
>>
>>>> I would first specify the required steps that make IP as-is work within
>>>> one subnet/transport layer segment. That should be an achievable goal with
>>>> a limitation of e.g. a single interconnection point between
>>>> subnet/transport layer segments of different types/solutions.
>>>>
>> After that look into solving multiple interconnection points.. and there
>>>> the L4 path that Pascal mentioned or that IPv6 DetNet DstOpt of mine could
>>>> be way forward.
>>>>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 12 Feb 2018 14:15:16 -0500

Whoops, that was a typo, should be RFC 7510, not 7520.

Cheers,
Andy

On Mon, Feb 12, 2018 at 2:14 PM, Andrew G. Malis <[email-masked]> wrote:

> Lou and Pascal,
>
> On the most recent interim call, we discussed an encapsulation for IPv4
> and IPv6 that reused the MPLS encapsulation inside UDP inside IP (ala RFC
>> - The  ingress edge may be a NIC card in the originating node. It would
>>> be operating below IPvX and possibly equipped with 2 Ethernet ports and
>>> capable of PREF.
>>>
>>>>> I would first specify the required steps that make IP as-is work
>>>>> within one subnet/transport layer segment. That should be an achievable
>>>>> goal with a limitation of e.g. a single interconnection point between

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 12 Feb 2018 19:31:22 +0000

Hello Balázs

- Why no seq.num in IP? That would mean no PREF, no IOD function within DetNet domain Which is an IP PSN.

> Sorry, I was unclear and I agree with you. There would not be one provided the source, but then the ingress edge would need to encapsulate the packet. In which case it would set the next  as destination and insert a DestOpt before the routing header if any. 

> Note that if have not (Yet? Should we???) looked at a way to express the ladder as a routing header. Till then, the most sensible thing is to send the packet unicast to the next Relay node (next relays if this node is a replicator). We still may indicate intermediate transit nodes as a routing header though. The DestOpt would then be after the routing header, not before.

--------------------

> I would say that these functions are closely related to traditional capabilities of a L4 transport. TCP reorders the bytes in the stream and provides ARQ which is an alternate to FEC. MP TCP distributes packets over different paths. Doing those functions at L4 allows real end to end service even if there is no specialized NIC. It keeps the IP layer intact which seemed to be a goal in that discussion. It allows features like Network Coding which are certainly easier to do high in the stack of a host system than in a line card or a fast path in a router.

- Checking L4 within the DetNet domain would require DPI, what I guess we do not want.

> Architecturally speaking, the L4 transport implements a detnet service layer though it would not be done at DetNet WG. When that L4 transport is enabled, some service layer functions provided by the network would not be used, or would rely on their own sequencing. I'm not excluding that vendors could propose some DPI for very specific applications, e.g. recognizing a particular industrial protocol and routing it differently, say over a DetNet path. But this is an implementation game and I do not see what's there for us to standardize.

------------------------

- I am also somewhat confused by your WiFi example. I mean in TSN/DetNet scenarios end-systems are always assumed to be in sync with their controller(s). So I do not agree with this statement: "Existing industrial devices can claim a rough need for a given rate but were not necessarily designed to provide guarantees for burstiness in DetNet terms." 

> I mean that an application on a device that was designed before TSN and that does not connect to the controller and that may not participate to PTP may still benefit from DetNet. But even if the application needs are roughly known, the granularity of the TSPEC may not be the one TSN expects. There is a need for an adaptation to absorb the burst that a strict ingress filtering may treat too violently. Reference here is to CIR in frame relay networks serving TCP and SNA flows. Disserving would be a more proper term, I have a long history on implementing that, and I would like to avoid going through that pain again.

What I meant with Wi-Fi is different; I showed that in some cases we really need the PREF from the source node if it is a wireless node. Because the losses mostly happen on the radio hop. I had in mind that with a form of multicast, the source STA could send a single packet to 2 or more Ingress Routers connected to different APs, and then all the received copies would follow their own DetNet path.

Take care,

Pascal

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Balázs Varga A
Sent: lundi 12 février 2018 17:37
To: Pascal Thubert (pthubert) <[email-masked]>; Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Hi Pascal,

However I have some questions for clarification on your final conclusions:
- Why no seq.num in IP? That would mean no PREF, no IOD function within the DetNet domain Which is an IP PSN.
- What would be the gain from a "L4 transport designed to provide PREF, IOD, FEC, ... "?
I mean that would be a duplicate, as in the DetNet network You would have the same function.
Checking L4 within the DetNet domain would require DPI, what I guess we do not want.
- I am also somewhat confused by your WiFi example. I mean in TSN/DetNet scenarios end-systems are always assumed to be in sync with their controller(s). So I do not agree with this statement: "Existing industrial devices can claim a rough need for a given rate but were not necessarily designed to provide guarantees for burstiness in DetNet terms." 

Thanks & Cheers
Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: 2018. február 12. 11:30
To: Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Hello Jouni and Lou:

Do we agree?

Pascal

-----Original Message-----
From: Jouni [mailto:[email-masked]]
Sent: dimanche 11 février 2018 22:43
To: Lou Berger <[email-masked]>
Cc: Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Indeed..

- Juoni

---

From: Lou Berger <[email-masked]>
Date: Mon, 12 Feb 2018 15:45:52 -0500

I thought I heard IP over mpls not mpls over IP, although this is certainly 
possible but my bet is that this a third document or at least a new section.

I expect that this will come up on Wednesdays interim...

Lou

---

From: Balázs Varga A <[email-masked]>
Date: Tue, 13 Feb 2018 09:01:41 +0000

Hello Pascal,
Thanks, that was great clarification and now I see your valid points much better.
Cheers
Bala'zs

-----Original Message-----
From: Pascal Thubert (pthubert) [mailto:[email-masked]] 
Sent: 2018. február 12. 20:31
To: Balázs Varga A <[email-masked]>; Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: RE: [Detnet] IPv6 encapsulation in dataplane doc

Hello Balázs

--------------------

------------------------

Take care,

Pascal

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Balázs Varga A
Sent: lundi 12 février 2018 17:37
To: Pascal Thubert (pthubert) <[email-masked]>; Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Hi Pascal,

Thanks & Cheers
Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: 2018. február 12. 11:30
To: Jouni <[email-masked]>; Lou Berger <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Hello Jouni and Lou:

Do we agree?

Pascal

-----Original Message-----
From: Jouni [mailto:[email-masked]]
Sent: dimanche 11 février 2018 22:43
To: Lou Berger <[email-masked]>
Cc: Pascal Thubert (pthubert) <[email-masked]>; [email-masked]
Subject: Re: [Detnet] IPv6 encapsulation in dataplane doc

Indeed..

- Juoni