Subject: [Detnet] Gap Analysis for DetNet

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 13 Nov 2014 00:35:29 +0000

My own 2 cents based on the DetNet BoF meeting:

We differentiated the need for:

1)      a blueprint of the big picture, at the level of the interaction models over a number of interfaces that we defined, and

2)      the model for the data that has to be exchanged or stored in the devices as part of the execution of the protocols.

I'm happy to say that we did not get a knee jerk like the IEEE is imposing its design on IETF or something like that.

OTOH, there were still reactions that we have to overcome.

1)      People did not necessarily figure that Norm and I were talking about the same problem.

2)      This may relate to some lack of common terminology, e.g. what's a flow?

3)      Also, people were saying that (from arguably a broad perspective) all this has been done or is being done somewhere. Metro Ethernet Forum was mentioned and I'm unsure how much overlap we have there. And then work are PCE and CCAMP was discussed.

The final words (of wisdom) from the A-D is that we need a gap analysis.

Interaction model

In that gap analysis we need to differentiate work that needs to be done in a new WG (quiet clearly the blueprint and then some new data models) from what should be done in other WGs (typically extensions of existing protocols such as PCEP or stuff over foo).

The arch blueprint we seem to agree upon is based on a controller model and looks like this:

Interactions happen over a Northbound interface with the app/service layer, over a Southbound interface for the Network Interface, and over a UNI interface between the user NICs and the network elements.

Questions in the UNI interface:
* Whether we can live without one is one question
* The interaction is also TBD:
   - From the network to user NIC, it could be as simple as a periodic list of established paths, or we could add async info on state change, per-flow tick, etc...
   - The other way, it could be async readiness to receive on the Listener side.

Questions on the SB I/F:
* Interaction between the Sender or an operator and the PCE are certainly in scope.
* Interactions between applications will probably depend on applications and we should stay away from there.
* Finally, whether the Receiver side should interact with the PCE and what that interaction should be is in scope but left to be determined at this point.

Questions on the NB I/F
* Interaction between the network and the controller for the purpose of gathering capabilities, topology and metrics is required. Apparently there is work at MEF for a data model. Classical PCE gets information by passively listening to IS-IS or similar.
* Whether interaction between the Controller and the NIC is useful is TND
* From there we proposed 2 abstract approaches for the blueprint, and either we agree on one asap, or we need to provide a gap analysis for both:

"CCAMP" approach.

This is the case where the controller injects the flow information at one end point with some kind of source route path and associated state. A control packet flows to the other end of the path, provisioning on the way.

It was mentioned that the CCAMP WG is looking at (rechartering for?) work that would be useful to implement this approach.

"PCE" approach
-------------------
This is the case where the controller injects the flow information to all nodes on the way individually. It is unclear whether the controller needs also talk to the end point.

In this case, there is probably work that we can leverage from the PCE WG.

Data models

We found that we need to provide data models for:

* the TSpec that describes the flow at the app/service layer
* The state that must be installed in each Hop.
* the path control information that is passed over the UNI interface

This sort of object exist all over the place, usually without all the information that we require such as precise timing. I' m unclear which example we should be using for gap analysis and potential extensions. Ideas welcome!

We also need to work on tagging in the packet for:
* flow identification
* time stamping
* sequencing, for the purpose of replication and elimination

For this, there is at least one proposal on the table, from Norm, of reusing MPLS tagging. We certainly need to work with NVO3 and SFC to study the interaction and the orders of tags, between overlays, service chaining and DetNet.

What do you think?

Pascal

---

From: "Anca Zamfir (ancaz)" <[email-masked]>
Date: Mon, 17 Nov 2014 12:13:05 +0000

Hello Pascal,
Looks good, thanks for the notes. A few comments and questions that you may want to address in the analysis:

  *   the controller needs to know where the endpoints are attached to and, I believe, some of their characteristics. One example that comes to mind is the replicate/ duplicate-elimination ability for seamless redundancy. (same applies for the network devices).
  *   for the PCE approach, an interaction is needed from controller (or App?) with the talker sometime after the flow has been scheduled on all devices on path, in order to instruct the talker to start transmission.
  *   similarly for the listener to start listening (I think you have this one mentioned below)
  *   are the requirements for establishing paths/ schedules in an initial setup only? Or are we considering dynamic flows as well?
  *   regarding the control messaging used for provisioning in CCAMP case - is there a need for a 2-way or a 3-way messaging ala RSVP to satisfy the requirements?
  *   what are the class(es) of devices that need to be considered for the endpoints? What is the order of magnitude for the number of endpoints? What types of interfaces are used to connect them to the network elements?

It would be also useful to have a control flow sequence, maybe start with one usecase.
Regarding the CCAMP case, there is a PCE functionality in the controller that it is required in order to determine the source routes, correct? IMO it's better to focus on PCE model first and consider the CCAMP case after.

cheers,
-anca

From: "Pascal Thubert (pthubert)" <[email-masked]<mailto:[email-masked]>>
Date: Thursday, November 13, 2014 1:35 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [Detnet] Gap Analysis for DetNet

I’m happy to say that we did not get a knee jerk like the IEEE is imposing its design on IETF or something like that.

2)      This may relate to some lack of common terminology, e.g. what’s a flow?

3)      Also, people were saying that (from arguably a broad perspective) all this has been done or is being done somewhere. Metro Ethernet Forum was mentioned and I’m unsure how much overlap we have there. And then work are PCE and CCAMP was discussed.

Interaction model

   - From the network to user NIC, it could be as simple as a periodic list of established paths, or we could add async info on state change, per-flow tick, etc…

Questions on the SB I/F:

Questions on the NB I/F

“CCAMP” approach.

“PCE” approach
-------------------

Data models

This sort of object exist all over the place, usually without all the information that we require such as precise timing. I’ m unclear which example we should be using for gap analysis and potential extensions. Ideas welcome!

What do you think?

Pascal

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 24 Nov 2014 15:17:46 +0000

Hello Anca:

Please find inline

From: Anca Zamfir (ancaz)
Sent: lundi 17 novembre 2014 13:13
To: Pascal Thubert (pthubert); [email-masked]
Subject: Re: [Detnet] Gap Analysis for DetNet

Hello Pascal,

[PT] Yes, certainly...

  *   for the PCE approach, an interaction is needed from controller (or App?) with the talker sometime after the flow has been scheduled on all devices on path, in order to instruct the talker to start transmission.
  *   similarly for the listener to start listening (I think you have this one mentioned below)
[PT] Yes, for both. This is typically what the UNI would be about, like a FM LMI on steroids. Probably a hint of timing info as well would allow the client to send at an optimum time to avoid queueing and shaping artifacts at the first switch.

  *   are the requirements for establishing paths/ schedules in an initial setup only? Or are we considering dynamic flows as well?
[PT] I expect that AvNU already has variable bit rate requirements, but all in all, we seem to reserve for the peak, as if it was all constant bit rate. I'm not sure we can be any deterministic without that. OTOH, we need to aggressively pursue the inefficiencies, and reuse bandwidth that would be wasted in reservation. At least in the case of 6TiSHC, this is a clear requirement that is discussed in the architecture.

  *   regarding the control messaging used for provisioning in CCAMP case - is there a need for a 2-way or a 3-way messaging ala RSVP to satisfy the requirements?
[PT] Need to dig CCAMP, but that's certainly something like that, though the way of the ack could be via the PCE so packets always flow in a same direction inside the deterministic network (which may or may not have symmetrical properties for transmission)..

  *   what are the class(es) of devices that need to be considered for the endpoints? What is the order of magnitude for the number of endpoints? What types of interfaces are used to connect them to the network elements?
[PT] IoT is in scope, as well as video servers. IoT bring in large numbers, and video large flows. Both types leading to congestion loss unless deterministic properties are provided. The shaper may differ though. I see a lot of value in the Cyclical Queuing and Forwarding (CQF) for large scale IoT monitoring.

[PT] Correct, and why not?

All the best,

Pascal

cheers,
-anca

Interaction model

Questions on the SB I/F:

Questions on the NB I/F

"CCAMP" approach.

"PCE" approach
-------------------

Data models

What do you think?

Pascal

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 24 Nov 2014 15:59:04 +0000

In the case of 6TiSCH, we usually have a guard time fo +/- 500ms. A time Slot is about 10-15ms long, time enough for a 127 bytes frame + ack including various waits and time to crypto.
This is very different from the scale of Time-Triggered Ethernet, to the point where one would ask why we would need to be deterministic on an Ethernet that federates large 6TiSCH networks.
And that's when the numbers and the peristaltic scheduling come into play, making sure that vast amounts of very small flows are equally spread and do not generate a congestion loss on the backbone.

Cheers,

Pascal

From: David Mozes [mailto:[email-masked]]
Sent: lundi 24 novembre 2014 16:50
To: Pascal Thubert (pthubert); Anca Zamfir (ancaz); [email-masked]
Subject: RE: [Detnet] Gap Analysis for DetNet

What is  the windows' time accuracy  for transmitting  if there is any ?

From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: Monday, November 24, 2014 5:18 PM
To: Anca Zamfir (ancaz); [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet] Gap Analysis for DetNet

Hello Anca:

Please find inline

From: Anca Zamfir (ancaz)
Sent: lundi 17 novembre 2014 13:13
To: Pascal Thubert (pthubert); [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet] Gap Analysis for DetNet

Hello Pascal,

All the best,

Pascal

cheers,
-anca

Interaction model

Questions on the SB I/F:

Questions on the NB I/F

"CCAMP" approach.

"PCE" approach
-------------------

Data models

What do you think?

Pascal

---

From: David Mozes <[email-masked]>
Date: Mon, 24 Nov 2014 15:49:42 +0000

From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: Monday, November 24, 2014 5:18 PM
To: Anca Zamfir (ancaz); [email-masked]
Subject: Re: [Detnet] Gap Analysis for DetNet

Hello Anca:

Please find inline

Hello Pascal,

All the best,

Pascal

cheers,
-anca

Interaction model

Questions on the SB I/F:

Questions on the NB I/F

"CCAMP" approach.

"PCE" approach
-------------------

Data models

What do you think?

Pascal

---

From: David Mozes <[email-masked]>
Date: Mon, 24 Nov 2014 16:19:51 +0000

What is the maximum link speed that we are targeting  ?
Looks that it need to related to this  , right  ?

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Monday, November 24, 2014 5:59 PM
To: David Mozes; Anca Zamfir (ancaz); [email-masked]
Subject: RE: [Detnet] Gap Analysis for DetNet

Cheers,

Pascal

From: David Mozes [mailto:[email-masked]]
Sent: lundi 24 novembre 2014 16:50
To: Pascal Thubert (pthubert); Anca Zamfir (ancaz); [email-masked]<mailto:[email-masked]>
Subject: RE: [Detnet] Gap Analysis for DetNet

Hello Anca:

Please find inline

Hello Pascal,

All the best,

Pascal

cheers,
-anca

Interaction model

Questions on the SB I/F:

Questions on the NB I/F

"CCAMP" approach.

"PCE" approach
-------------------

Data models

What do you think?

Pascal

---

From: "Patrick Wetterwald (pwetterw)" <[email-masked]>
Date: Mon, 24 Nov 2014 16:44:18 +0000

Some comments:

If you look at the Utilities requirement draft:

Deterministic Networking Uitilities requirements
draft-wetterwald-detnet-utilities-reqs-01

Important use cases required a maximum latency of 4 to 10 ms and a jitter less than 250 usec. The distance between the 2 end points may be hundreds of km.

More important is that it requires a symmetric delay. Meaning that the difference between the latency for a packet from A to B  and B to A must be less than 250 usec as well.
This has an impact on the computation of paths and reverse paths. The end devices are both talkers and listeners with the timing requirements just listed above.

We may have stronger requirements in industrial automation as well.

Thanks,

Patrick

From: David Mozes <[email-masked]<mailto:[email-masked]>>
Date: Monday 24 November 2014 17:19
To: Pascal THUBERT <[email-masked]<mailto:[email-masked]>>, "Anca Zamfir (ancaz)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Detnet] Gap Analysis for DetNet

From: Pascal Thubert (pthubert) [mailto:[email-masked]]
Sent: Monday, November 24, 2014 5:59 PM
To: David Mozes; Anca Zamfir (ancaz); [email-masked]<mailto:[email-masked]>
Subject: RE: [Detnet] Gap Analysis for DetNet

In the case of 6TiSCH, we usually have a guard time fo +/- 500ms. A time Slot is about 10-15ms long, time enough for a 127 bytes frame + ack including various waits and time to crypto.
This is very different from the scale of Time-Triggered Ethernet, to the point where one would ask why we would need to be deterministic on an Ethernet that federates large 6TiSCH networks.
And that’s when the numbers and the peristaltic scheduling come into play, making sure that vast amounts of very small flows are equally spread and do not generate a congestion loss on the backbone.

Cheers,

Pascal

What is  the windows’ time accuracy  for transmitting  if there is any ?

Hello Anca:

Please find inline

Hello Pascal,

[PT] Yes, certainly…

  *   are the requirements for establishing paths/ schedules in an initial setup only? Or are we considering dynamic flows as well?
[PT] I expect that AvNU already has variable bit rate requirements, but all in all, we seem to reserve for the peak, as if it was all constant bit rate. I’m not sure we can be any deterministic without that. OTOH, we need to aggressively pursue the inefficiencies, and reuse bandwidth that would be wasted in reservation. At least in the case of 6TiSHC, this is a clear requirement that is discussed in the architecture.

  *   regarding the control messaging used for provisioning in CCAMP case - is there a need for a 2-way or a 3-way messaging ala RSVP to satisfy the requirements?
[PT] Need to dig CCAMP, but that’s certainly something like that, though the way of the ack could be via the PCE so packets always flow in a same direction inside the deterministic network (which may or may not have symmetrical properties for transmission)..

All the best,

Pascal

cheers,
-anca

Interaction model

Questions on the SB I/F:

Questions on the NB I/F

“CCAMP” approach.

“PCE” approach
-------------------

Data models

What do you think?

Pascal

---

From: Pat Kinney <[email-masked]>
Date: Mon, 24 Nov 2014 11:12:35 -0600

I believe that Pascal meant to state that in 6TiSCH slot guard times are often 0.5 ms (500 μs).  However, I note that timing within a slot is usually tighter, such as ± 0.1 ms for start of a transmission.  In IEEE 802.15.4 timing accuracies range from 20x10-6  to 50x10-6.

Pat Kinney

Pat Kinney
Kinney Consulting LLC
IEEE 802.15 WG vice chair, TG chair
ISA100.11a WG chair
O: +1.847.960.3715
[email-masked] <mailto:[email-masked]>

On 24, Nov2014, at 9:59, Pascal Thubert (pthubert) <[email-masked]> wrote:

Cheers,
 
Pascal

 
From: detnet [mailto:[email-masked] <mailto:[email-masked]>] On Behalf Of Pascal Thubert (pthubert)
Sent: Monday, November 24, 2014 5:18 PM
To: Anca Zamfir (ancaz); [email-masked] <mailto:[email-masked]>
Subject: Re: [Detnet] Gap Analysis for DetNet
 
Hello Anca:
 
Please find inline
 
From: Anca Zamfir (ancaz) 
Sent: lundi 17 novembre 2014 13:13
To: Pascal Thubert (pthubert); [email-masked] <mailto:[email-masked]>
Subject: Re: [Detnet] Gap Analysis for DetNet
 
Hello Pascal,
Looks good, thanks for the notes. A few comments and questions that you may want to address in the analysis:
the controller needs to know where the endpoints are attached to and, I believe, some of their characteristics. One example that comes to mind is the replicate/ duplicate-elimination ability for seamless redundancy. (same applies for the network devices).
[PT] Yes, certainly…
for the PCE approach, an interaction is needed from controller (or App?) with the talker sometime after the flow has been scheduled on all devices on path, in order to instruct the talker to start transmission.
similarly for the listener to start listening (I think you have this one mentioned below)
[PT] Yes, for both. This is typically what the UNI would be about, like a FM LMI on steroids. Probably a hint of timing info as well would allow the client to send at an optimum time to avoid queueing and shaping artifacts at the first switch.
are the requirements for establishing paths/ schedules in an initial setup only? Or are we considering dynamic flows as well?
[PT] I expect that AvNU already has variable bit rate requirements, but all in all, we seem to reserve for the peak, as if it was all constant bit rate. I’m not sure we can be any deterministic without that. OTOH, we need to aggressively pursue the inefficiencies, and reuse bandwidth that would be wasted in reservation. At least in the case of 6TiSHC, this is a clear requirement that is discussed in the architecture.
regarding the control messaging used for provisioning in CCAMP case - is there a need for a 2-way or a 3-way messaging ala RSVP to satisfy the requirements?
[PT] Need to dig CCAMP, but that’s certainly something like that, though the way of the ack could be via the PCE so packets always flow in a same direction inside the deterministic network (which may or may not have symmetrical properties for transmission)..
what are the class(es) of devices that need to be considered for the endpoints? What is the order of magnitude for the number of endpoints? What types of interfaces are used to connect them to the network elements?
[PT] IoT is in scope, as well as video servers. IoT bring in large numbers, and video large flows. Both types leading to congestion loss unless deterministic properties are provided. The shaper may differ though. I see a lot of value in the Cyclical Queuing and Forwarding (CQF) for large scale IoT monitoring.

 
All the best,
 
Pascal

cheers,
-anca 

From: "Pascal Thubert (pthubert)" <[email-masked] <mailto:[email-masked]>>
Date: Thursday, November 13, 2014 1:35 AM
To: "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
Subject: [Detnet] Gap Analysis for DetNet

 

 
Interaction model

 
 
Questions on the SB I/F:
 
Questions on the NB I/F

“CCAMP” approach.

 
“PCE” approach
-------------------

Data models

 

 
What do you think?
 
Pascal
_______________________________________________
detnet mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet

---

From: "Anca Zamfir (ancaz)" <[email-masked]>
Date: Wed, 26 Nov 2014 15:47:42 +0000

Hello Pascal,
Thanks for the responses. One clarification below.

From: "Pascal Thubert (pthubert)" <[email-masked]<mailto:[email-masked]>>
Date: Monday, November 24, 2014 4:17 PM
To: Anca Zamfir <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [Detnet] Gap Analysis for DetNet

...

My question was not clear but you answer an interesting one that should be included in the gap analysis. Back to my question.

There are some scenarios (like the factory floor one) where a "static" setup rather than a dynamic flow setup scenario may apply. That is, the controller does not have to deal with the case where new flows need to be scheduled or old flows removed. For lack of better term let's call this "schedule computation on initial setup". In this case the controller runs "offline" trying to find the optimal schedulers for a predetermined set of flows (in general being multi-constraint, path computation is NP-complete, will probably take a long time, possible to improve with bounded metrics but still slow). Once computed, schedules are downloaded and network starts running. If flows need to be added/ changed/ deleted, new schedulers are computed from start, everything is stopped, re-downloaded during maintenance window and restart. This can be optimized but fundamentally the scheme stays the same in nature. The result is optimal schedulers.

In a "dynamic scheduler" scenario, the controller/ PCE could get new requests (delete or add flows) live and recompute the schedules starting from the existing one, with minimize-disruption-level potentially a new criteria for computation. Another one may be anticipate-the-worst.
Maybe controller uses heuristics for an acceptable computation response time. The requirements and algorithms in this case may be different than the one above.

Does it make sense? Do both of these need to be considered?

cheers,
-anca