Subject: [Detnet] data plane framework editorial pass

From: "Grossman, Ethan A." <[email-masked]>
Date: Tue, 1 Oct 2019 02:38:17 +0000

Hi All, (particularly Balazs as editor of the draft),
I hate to be the guy who waits until WG LC to actually read the draft carefully and then comes with a bunch of comments, but here I am; hopefully I will get into gear sooner with the other data plane drafts, but as document shepherd I have to read carefully, and I figured as long as I was noticing things I would write them down.
So as a result there are many comments below; thankfully they are mostly grammatical, the sort that the IETF Editorial will surely make if we don't correct them, so I figure we should bite the bullet and do them now rather than later. There are even a few actual "what does this mean" type of questions. But on the whole the draft is in quite good shape and I plan to start my "shepherd writeup" soon, given that I have gone over the draft carefully now.
The form I am using for my comments is like this:
Issue type (e.g. wording, grammar, abbreviation, etc):
"text in question"

? "Ethan's suggestion on how to fix it:"
I hope this is helpful. If it isn't, just let me know and for the remaining drafts I'll be less picky (or whatever your suggestion to me is). And of course I may have gotten the wrong end of the stick on some of the items, so please feel free to push back if I got the meaning wrong.
Ethan (as data plane framework draft document shepherd).
---------------------------------------
Wording:
"Abstract
This document provides an overall framework for the Deterministic Networking data plane.  It covers concepts and considerations that are generally common to any Deterministic Networking data plane specification."

  *   Say DetNet instead of the more generic Deterministic Networking since it is specific to our architecture, not to all of deterministic networking in the world.
Wording:
"1.  Introduction
   Deterministic Networking (DetNet)"

  *   Similarly, say "DetNet (Deterministic Networking)" .
Wording:
"The forwarding sub-layer is used to provide congestion protection (low loss, assured latency, and limited out-of-order delivery) and leverages Traffic Engineering mechanisms."

  *   Improve clarity:
The forwarding sub-layer leverages Traffic Engineering mechanisms to provide congestion protection (low loss, assured latency, and limited out-of-order delivery).
Grammar:
"service sub-layer"

  *   Service is a proper name, s/b capitalized.

Wording:
"It also describes the forwarding sub-layer that is used to   eliminate (or reduce) contention loss and provide bounded latency for   DetNet flows."

  *   Redundant with above paragraph. s/b just "...the Forwarding sub-layer". "assured latency" vs "bounded latency".
Meaning:
" Different traffic types, or application flows, can be mapped on top   of DetNet."

  *   Are these two separate things? I gather they are? Maybe "Different traffic types and application flows can be mapped on top  of DetNet". Below it then says
"3.  DetNet Data Plane Overview
   This document describes how application flows, or app-flows, are  carried over DetNet networks."
This sounds like they are synonyms? But an App Flow is not differentiated by "traffic type" - we differentiate based on individual flow. So I think we should remove "traffic types" above.
Wording:
 "functions  related to the control plane"

  *   Are we standardizing on controller plane vs control plane, or are these different grammatical roles?
Wording:
"The forwarding sub-layer provides the quality underpin needed by the   DetNet flow."

  *   Is underpin a real word? Sounds like we mean QoS?
Abbreviation:
"An example of   this is Packet Replication, Elimination, and Ordering (PREOF)    functions see Section 4.3."

  *   Not first use of PREOF, should not spell out here.
Grammar:
"The method of instantiating each of the layers is specific to the   particular DetNet data plane method.  There may be more than one   approach that is applicable to a given bearer network type."

  *   I would combine these into a single sentence with comma, as: "The method of instantiating each of the layers is specific to the particular DetNet data plane method, and more than one approach may be applicable to a given bearer network type".
Wording:
"3.1.  Data Plane Characteristics
   There are two major characteristics to the data plane:
   1.  Data plane technology: The DetNet data plane is provided by the DetNet service and forwarding sub layers."
--> Please don't tell me this again. Eliminate this sentence.
Wording:
"Namely"

  *   Remove this word, already said "specifically" which is better anyway.
Abbreviations:
"S-label and d-CW"

  *   Need to add these to Abbrev section.
Structure:
"There are two major characteristics to the data plane:

  1.  Data plane technology:"

  *   I would say these should not be a 2-item numbered list, they should be a topic sentence listing both, followed by subsections, as is done for Encapsulation but not for Technology. Also one says Data Plane Technology, the other just Encapsulation - should be consistent - So we could have:
3.1.  Data Plane Characteristics
There are two major characteristics to the data plane: the technology and the encapsulation, as discussed below.
3.1.1 Technology
...
3.1.2 Encapsulation
...

Grammar:
"The encapsulation of the DetNet flows allows them to be sent over a   data plane technology other than their native type.  Encapsulation is   essential if, for example, it is required to send Ethernet TSN stream    as a DetNet Application over a data plane such as MPLS."

  *   Consolidate: "The encapsulation of a DetNet flow allows it to be sent over a   data plane technology other than its native type. For example, an Ethernet TSN app flow can be sent as a DetNet app flow over MPLS."
Grammar:
"meta-data"

  *   s/b metadata - not a hyphenated word.
Clarity:
"The DetNet data plane can provide or carry meta-data:
   1.  Flow-ID
   2.  Sequence Number
... Both of these metadata are required ..."

  *   I initially read this as "they are both required" but then it turns out they are not. Rephrase:
"The DetNet data plane supports a Flow-ID (for identification of the flow or aggregate flow) and/or a Sequence Number (for PREOF) for each DetNet flow. The DetNet Service sub-layer requires both; the DetNet forwarding sub-layer requires only Flow-ID. Metadata can also be used for OAM indications and instrumentation of DetNet data plane operation."
Grammar:
"it is anticipated that more than one encapsulation   may be deployed for example GRE"

  *   Need comma after "deployed,"
Wording:
"[RFC7657] also provides useful background on the delivery differentiated services (DiffServ)"

  *   I don't think "the delivery" belongs there? Delete? ("...useful background on Differentiated Services (Diffserv)..." (following RFC7657 format).
Word meaning:
"It is possible to include such information in a native IP   packet explicitly, or implicitly."

  *   I understand "explicitly" to mean there is additional metadata, vs "implicitly" to mean that it is inferred from the existing fields e.g. 6-tuple. However above, it says "Some MPLS examples of implicit metadata include the sequence number    for use by the PREOF function" - to me that sounds odd. Can we please reconcile use of the words "explicit" vs "implicit" and/or explain the meaning of each. We also have explicit/implicit paths (for which I don't see a clear definition from a simple google search):
"MPLS provides the ability to forward traffic over implicit and   explicit paths to the point in the network where the next DetNet   service sub-layer action needs to take place."
"Reservation and Allocation of resources:" (and similar lines below)

  *   Perhaps these paragraphs should be 3rd level headers, as opposed to one-sentence paragraphs with ":" at the end. This would also cause them to be included in the table of contents. Perhaps having a bullet list at the top would be useful? Then one could move to here the statement from below: "Several of these capabilities are expanded upon in more detail in sections below."
Clarity:
"This can eliminate packet contention and loss"

  *   I would explicitly say "This can eliminate packet contention and packet loss".
Grammar:
"This also can reduce jitter for the DetNet traffic"

  *   Eliminate superfluous "the": "This also can reduce jitter for DetNet traffic"

Clarity:
"DetNet flows are assumed to behave with respect to the reserved traffic profile.  If other traffic shares the link resources, the use of (queuing, policing, shaping) policies can be used to ensure that the allocation of resources reserved for DetNet is met.   Queuing and shaping of DetNet traffic could be required to ensure that DetNet traffic does not exceed its reserved profile but this would impact the DetNet service characteristics."

  *   I think this could be made clearer. Maybe:
"DetNet flows are assumed to behave with respect to the reserved traffic profile.  If other traffic shares the link resources, the use of policies (such as queuing, policing, shaping) can be applied to the "other" traffic to ensure that the allocation of resources reserved for DetNet is not compromised.  Queuing and shaping of DetNet traffic to ensure that it does not exceed its reserved profile (for example by dropping packets) may be necessary to prevent fault conditions, but should not be used under normal conditions as this could compromise the DetNet QoS."
"Network coding"

  *   Proper name, Network Coding, s/b capitalized.
"packet by packet"

  *   s/b hyphenated "packet-by-packet".
Grammar:
"Since Detnet leverages many different forwarding sub-layers, those  technologies also support a number of tools to troubleshoot connectivity for example, to support identification of misbehaving  flows."

  *   Try:
 "Detnet leverages many different forwarding sub-layers, each of which supports various tools to troubleshoot connectivity, for example identification of misbehaving flows."
Grammar:
"At the service layer again there are existing mechanisms to troubleshoot or monitor flows.  Many of these mechanisms exist for IP and MPLS networks.  A client of a DetNet service can introduce any monitoring applications which can detect and monitor delay and loss."

  *   Try:
"The DetNet Service layer can leverage existing mechanisms to troubleshoot or monitor flows, such as those in use by IP and MPLS networks.  At the Application layer a client of a DetNet service can use existing techniques to detect and monitor delay and loss."
Grammar:
"Recognize flow(s) for analytics:"

  *   Try
"Flow recognition for analytics:
Grammar:
"To a large degree this follows the logic in the previous section. Analytics can be inherited from the two sub-layers.  At the DetNet service edge packet and bit counters e.g. sent, received, dropped, and out of sequence are maintained."

  *   Try:
"Network analytics can be inherited from the technologies of the Service and Forwarding sub-layers.  At the DetNet service edge, packet and bit counters (e.g. sent, received, dropped, and out-of-sequence) can be maintained."
Grammar:
"Correlate events with flows:"
      The provider of a DetNet service may allow other capabilities to monitor flows such as more detail loss statistics and time stamping of events.  The details of these capabilities are currently out of scope for this document."

  *   Try:
"Correlation of events with flows:"
      The provider of a DetNet service may provide other capabilities to monitor flows, such as more detailed loss statistics and time stamping of events.  The details of these capabilities are currently out of scope for this document."
3.6.1
Grammar:
"in the case of network congestion or some failures."

  *   Try:
"in the case of network congestion or equipment failure."

Abbreviation error:
"combinations of PRF, PRE, and POF."

  *   Try:
"combinations of PRF, PEF, and POF."

'"This example also illustrates 1:1 protection scheme meaning there is traffic and path for each segment of the end to end path."

  *   I do not understand what this means; "there is traffic and path"?
Grammar: (Ring Service Protection is a proper name, capitalized, "rings" is not).
'"Many of the same concepts apply however Rings"

  *   "Many of the same concepts apply, however rings"
Grammar:
"The DetNet data plane also allows for the aggregation of DetNet flows, to improved scaling by reducing the state per hop."

  *   Try:
"The DetNet data plane also allows for the aggregation of DetNet flows, which can improve scalability by reducing the per-hop state."
Grammar:
"If bandwidth reservations are used, the sum of the reservations should be the sum of all the individual reservations, in other words, the reservations should not create an over subscription of bandwidth reservation. If maximum delay bounds are used the system should ensure that the aggregate does not exceed the delay bounds of the individual flows."

  *   Try:
"If bandwidth reservations are used, the sum of the reservations should be the sum of all the individual reservations; in other words, the reservations should not add up to an over-subscription of bandwidth reservation. If maximum delay bounds are used, the system should ensure that the aggregate does not exceed the delay bounds of the individual flows."
Grammar:
"Encapsulation can either be in the same service type or in a different service type see Figure 3 for example.  When an encapsulation is used the choice of reserving a maximum resource level and then tracking the services in the aggregated service or adjusting the aggregated resources as the services are added is implementation and technology specific."

  *   Try:
"Encapsulation can either be in the same service type or in a different service type (see Figure 3, for example).  When an encapsulation is used, the choice of reserving a maximum resource level and then tracking the services in the aggregated service, versus adjusting the aggregated resources as the services are added, is implementation- and technology-specific."
In the following, I don't know what "conditions where general requirements are not satisfied" means, please clarify:
"DetNet flows at edges must be able to handle rejection to an aggregation group due to lack of resources as well as conditions where general requirements are not satisfied."
Grammar:
"For the data plane flows may be aggregated for treatment based on shared characteristics such as 6-tuple."

  *   Need comma after "For the data plane,".
Grammar:
"MPLS aggregation similarly has data plane and controller plane aspects.  In the case of MPLS flows are often tunneled in a forwarding sub-layer and reservation is associated with that MPLS tunnel."

  *   Try (I am not certain this is the correct meaning?):
"MPLS aggregation also has data plane and controller plane aspects.  MPLS flows are often tunneled in a forwarding sub-layer, under the reservation associated with that MPLS tunnel."
Grammar:
"3.6.3.  End-System Specific Considerations"

  *   Hyphenate "End-System-Specific"
Abbreviation: "DN" for DetNet as in "which are not provided by DN functions".

  *   "DN" is not defined as an abbreviation, but is also used in Fig 2 and Fig 3. In any of these cases either spell out DetNet or define DN as an abbreviation. I would just replace these kind of instances of "DN" with "DetNet".
Grammar:
"For example, a DetNet MPLS domain the DN functions use the d-CWs, S-Labels and F-Labels to provide DetNet services."

  *   Try:
"For example, in a DetNet MPLS domain the DetNet functions use the d-CWs, S-Labels and F-Labels to provide DetNet services."
Terminology: "edge system"

  *   Don't we say "edge node"? I don't see "edge system" anywhere else in this draft. If it is used elsewhere so we are going to use it, I would think it should be hyphenated "edge-system" like "end-system". Or maybe "edge system" in this context should be "end-system".
Abbreviation: "TDM technologies"

  *   TDM not defined.
Something is wrong grammatically, run-on sentence at best, but I can't figure out how to fix it, please help:
"Flow aggregation includes aggregation accomplished through the use of hierarchical LSPs in MPLS and tunnels, in the case of IP, MPLS and TSN, all of which aggregate multiple DetNet flows into a single new DetNet flow."

  *   Maybe something like:
"Flow aggregation includes aggregation accomplished through the use of hierarchical LSPs in MPLS, and through IP or TSN tunnels. IP, MPLS and TSN are each capable of aggregating multiple DetNet flows into a single new DetNet flow."
Grammar:
"Depending on the specific technology the assigned resources are updated and distributed in the databases preventing over subscription."

  *   Try:
 "Depending on the specific technology, the assigned resources are updated and distributed in the databases, preventing over-subscription."
Question about the following:
"While the DetNet IP data plane must support bidirectional DetNet flows, there are no special bidirectional features with respect to the data plane other than the need for the two directions of a co-
   routed bidirectional flow to take the same path.  That is to say that bidirectional DetNet flows are solely represented at the management and control plane levels, without specific support or knowledge within the DetNet data plane."

  *   There was discussion about use cases that wanted symmetrical delays in both directions; is that guaranteed by taking "the same path"? Presumably it is not implied by "fate sharing"?
Wording:
"DetNet's use of PREOF may increase the complexity of using co-routing bidirectional flows, since if PREOF is used, then the replication points in one direction would have to match the elimination points in the other direction, and vice versa, and the optimal points for these functions in one direction may not match the optimal points in the other subsequent to the network and traffic constraints."

  *   Try:
"DetNet's use of PREOF may increase the complexity of using co-routing bidirectional flows, since if PREOF is used, then the replication points in one direction would have to match the elimination points in the other direction, and vice versa. In such cases the optimal points for these functions in one direction may not match the optimal points in the other, due to network and traffic constraints."
---------------------------------------