Subject: [Detnet] IP Solution problem: Use of DSCP and ECN fields in IP headers for detnet flow identification

From: "Black, David" <[email-masked]>
Date: Sun, 4 Nov 2018 16:27:22 +0000

We have a problem here ...

The detnet IP solution draft (draft-ietf-detnet-dp-sol-ip-01.txt) has this to say about
use of DSCP and ECN fields in IP headers for detnet flow identification:

6.1.1.4.  IPv4 Type of Service and IPv6 Traffic Class Fields

   These fields are used to support Differentiated Services [RFC2474]
   and Explicit Congestion Notification [RFC3168].  Implementations of
   this document MUST support DetNet flow identification based on the
   IPv4 Type of Service field when processing IPv4 packets, and the IPv6
   Traffic Class Field when processing IPv6 packets.  Implementations
   MUST support bimask based matching, where one (1) values in the
   bitmask indicate which subset of the bits in the field are to be used
   in determining a match.  Note that a zero (0) value as a bitmask
   effectively means that these fields are ignored.

That bitmask approach won't work, as it violates both RFC 2474 and RFC 3168.

Starting with ECN (RFC 3168) - the 2-bit ECN field is intended to enable ECN functionality to be
applied to any flow, and the contents of the ECN field can be changed by any router.  Using ECN
field values to identify separate flows is wrong, see Section 5 of RFC 3168, which specifies the
current use of that field..

That leaves the 6-bit DSCP field, which is defined by RFC 2474.  The above bitmask approach is
prohibited by the following paragraph in section 3 of RFC 2474:

   Implementors should note that the DSCP field is six bits wide.  DS-
   compliant nodes MUST select PHBs by matching against the entire 6-bit
   DSCP field, e.g., by treating the value of the field as a table index
   which is used to select a particular packet handling mechanism which
   has been implemented in that device.  The value of the CU field MUST
   be ignored by PHB selection.  The DSCP field is defined as an
   unstructured field to facilitate the definition of future per-hop
   behaviors.

The CU field is now the ECN field (see RFC 3168).  My reading is that the current section 6.1.1.4 text in
the IP solutions draft has managed to violate all three "MUST" requirements in that RFC 2474
paragraph, which is impressive ... and not in a good way.

I suggest that several things be done:

a)      Abandon use of the ECN field for detnet flow identification.

b)      For the DSCP field, change from a bitmask approach to a list of DSCPs.

a.      I would note that a carefully chosen DSCP list can be implemented via a bitmask.

c)      Define one or more Diffserv PHBs that realize DetNet behavior.

a.      I suspect that much of the content needed for this already exists in the
detnet drafts, so this should not be a "from scratch" exercise.

I would also caution that the current IP solution draft text on 6-tuples for flow identification appears
to allow multiple separate detnet flows that differ only in DSCP to use the same IP 5-tuple (source &
destination addresses, transport protocol, source & destination ports).  I believe that this also ought
to be prohibited, as Diffserv uses 5-tuples for flow identification - see the definition and use of the
term "microflow" in RFC 2475.

A quick glance at the MPLS solution suggests that it does not have an analogous problem with the TC
field in labels as the TC field does not appear to be used for detnet flow identification.

Thanks, --David
----------------------------------------------------------------
David L. Black, Senior Distinguished Engineer
Dell EMC, 176 South St., Hopkinton, MA  01748
+1 (774) 350-9323 New    Mobile: +1 (978) 394-7754
[email-masked]<mailto:[email-masked]>
----------------------------------------------------------------

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 5 Nov 2018 04:12:50 +0000

Hi David,

Many thanks for your comments and suggestions. This is definitely something
we must fix.

Just two general statement as background:

- ECN was not considered so far to be useful for DetNet flows. DetNet flows
expect zero congestion loss.

DetNet sources do not consider to react on ECN.

- Masking for flow identification was considered as a general rule for the
“tuples” (i.e. not DSCP specific).

So, regarding your proposals

A, No usage of ECN for flow identification: AGREE

B, DSCP list instead of bitmask: AGREE, it can provide the same result.

C, New DiffServ PHB for DetNet: Agree in principle, let’s discuss the
details

Thanks

Bala’zs

From: detnet <[email-masked]> On Behalf Of Black, David
Sent: Sunday, November 4, 2018 11:27 PM
To: [email-masked]
Subject: [Detnet] IP Solution problem: Use of DSCP and ECN fields in IP
headers for detnet flow identification

We have a problem here …

The detnet IP solution draft (draft-ietf-detnet-dp-sol-ip-01.txt) has this
to say about

That bitmask approach won’t work, as it violates both RFC 2474 and RFC 3168.

Starting with ECN (RFC 3168) – the 2-bit ECN field is intended to enable ECN
functionality to be

applied to any flow, and the contents of the ECN field can be changed by any
router.  Using ECN

field values to identify separate flows is wrong, see Section 5 of RFC 3168,
which specifies the

That leaves the 6-bit DSCP field, which is defined by RFC 2474.  The above
bitmask approach is

   behaviors.

The CU field is now the ECN field (see RFC 3168).  My reading is that the
current section 6.1.1.4 text in

the IP solutions draft has managed to violate all three “MUST” requirements
in that RFC 2474

paragraph, which is impressive … and not in a good way.

a.	Abandon use of the ECN field for detnet flow identification. 
b.	For the DSCP field, change from a bitmask approach to a list of
DSCPs.

a.	I would note that a carefully chosen DSCP list can be implemented
via a bitmask.

c.	Define one or more Diffserv PHBs that realize DetNet behavior.

a.	I suspect that much of the content needed for this already exists in
the
detnet drafts, so this should not be a “from scratch” exercise.

I would also caution that the current IP solution draft text on 6-tuples for
flow identification appears

to allow multiple separate detnet flows that differ only in DSCP to use the
same IP 5-tuple (source &

destination addresses, transport protocol, source & destination ports).  I
believe that this also ought

to be prohibited, as Diffserv uses 5-tuples for flow identification - see
the definition and use of the

term “microflow” in RFC 2475.

A quick glance at the MPLS solution suggests that it does not have an
analogous problem with the TC

field in labels as the TC field does not appear to be used for detnet flow
identification.

Thanks, --David

[email-masked] <mailto:[email-masked]>

---

From: "Black, David" <[email-masked]>
Date: Tue, 6 Nov 2018 03:48:58 +0000

Hi Varga,

I'm pleased that we mostly agree, and I'll be happy to discuss the PHB further.

This discussion may also provide insight into my concern about 5-tuple vs. 6-tuple flow identification - it looks like we have two different sorts of "flow identification" under discussion:

- Identification of a flow in the network in general (network flow or microflow in RFC 2475).

- Identification of a flow that undergoes DetNet processing at a DetNet node (DetNet flow).

At the moment, we're using 5-tuples (src & dst addresses, protocol, src & dst ports) for network flow identification and 6-tuples (5-tuple + DSCP) for Detnet flow identification.  In other words, we're effectively saying that a network flow (defined by a 5-tuple), becomes a DetNet flow when a DetNet DSCP is added to request DetNet forwarding by the network.  There's a level of indirection here caused by the Diffserv architecture - a DetNet DSCP designates a DetNet PHB, and any a packet marked with a DetNet DSCP receives DetNet forwarding at a DetNet node.

That said, my concern is that the 6-tuple term for a DetNet flow seems counter-intuitive to me, even though the DetNet forwarding plane in a DetNet node may well be matching on all 6 elements involved in order to decide whether to provide DetNet forwarding to each arriving packet.

IMHO, a better description would be that a network flow (5-tuple) is being uniformly marked with a DetNet DSCP to request DetNet treatment from the network (and presumably DetNet router hardware is matching on DetNet DSCPs to dispatch inbound packets to DetNet forwarding logic instead of conventional forwarding logic).   The counter-intuitivity of the 6-tuple term arises in the error case when someone uses a "clever" marking of a single 5-tuple flow with a mixture of DetNet and non-DetNet DSCPs.  I believe that the result of that "cleverness" is not multiple flows (which would be implied by the "6-tuple" term), even though it may look like that in a DetNet node implementation.  I think the result is better described as a single not-entirely-DetNet flow (defined by a 5-tuple) that receives bizarre forwarding treatment at DetNet nodes - some of the packets get DetNet forwarding treatment and some get other forwarding treatment, so in a loaded network, significant reordering is likely, and the receiver is probably not going to be happy with the resulting packet arrival order.

If this all makes sense, that "clever" mixing of DetNet and non-DetNet DSCPs in a single flow (hence mixing DetNet and non-DetNet forwarding behavior in a single flow) deserves a "SHOULD NOT" statement in the draft, with a warning about the bizarre forwarding behavior at DetNet nodes that is a likely result, which seems like a better thing to do than my original suggestion for some sort of prohibition on mixing DetNet and non-DetNet packets in the same 5-tuple.

Thanks, --David

From: Balázs Varga A [mailto:[email-masked]]
Sent: Sunday, November 4, 2018 11:13 PM
To: Black, David; [email-masked]
Subject: RE: IP Solution problem: Use of DSCP and ECN fields in IP headers for detnet flow identification

Hi David,

Many thanks for your comments and suggestions. This is definitely something we must fix.

- ECN was not considered so far to be useful for DetNet flows. DetNet flows expect zero congestion loss.

- Masking for flow identification was considered as a general rule for the "tuples" (i.e. not DSCP specific).

C, New DiffServ PHB for DetNet: Agree in principle, let's discuss the details

Thanks

Bala'zs

From: detnet <[email-masked]> On Behalf Of Black, David
Sent: Sunday, November 4, 2018 11:27 PM
To: [email-masked]
Subject: [Detnet] IP Solution problem: Use of DSCP and ECN fields in IP headers for detnet flow identification

   behaviors.

Thanks, --David

---

From: Lou Berger <[email-masked]>
Date: Wed, 7 Nov 2018 20:07:00 +0700

Hi,

     David and I took advantage of being together this week and came up 
with a proposed resolution to this.  The proposal is as follows:

- ECN bits will *not* be mentioned as being used as part of flow 
identifications
- DSCP configuration will use value lists, not masks
- For the same 5-tuple value sender SHOULD NOT mix DSCP values used for 
DetNet with non-DetNet (DSCP) values

Please let me/us know what you think,

Lou

Thanks to David for the discussion.

On 11/6/2018 10:48 AM, Black, David wrote:
>
> Hi Varga,
>
> I’m pleased that we mostly agree, and I’ll be happy to discuss the PHB 
> further.
>
> This discussion may also provide insight into my concern about 5-tuple 
> vs. 6-tuple flow identification – it looks like we have two different 
> sorts of “flow identification” under discussion:
>
> At the moment, we’re using 5-tuples (src & dst addresses, protocol, 
> src & dst ports) for network flow identification and 6-tuples (5-tuple 
> + DSCP) for Detnet flow identification. In other words, we’re 
> effectively saying that a network flow (defined by a 5-tuple), becomes 
> a DetNet flow when a DetNet DSCP is added to request DetNet forwarding 
> by the network. There’s a level of indirection here caused by the 
> Diffserv architecture – a DetNet DSCP designates a DetNet PHB, and any 
> a packet marked with a DetNet DSCP receives DetNet forwarding at a 
> DetNet node.
>
> That said, my concern is that the 6-tuple term for a DetNet flow seems 
> counter-intuitive to me, even though the DetNet forwarding plane in a 
> DetNet node may well be matching on all 6 elements involved in order 
> to decide whether to provide DetNet forwarding to each arriving packet.
>
> IMHO, a better description would be that a network flow (5-tuple) is 
> being uniformly marked with a DetNet DSCP to request DetNet treatment 
> from the network (and presumably DetNet router hardware is matching on 
> DetNet DSCPs to dispatch inbound packets to DetNet forwarding logic 
> instead of conventional forwarding logic).   The counter-intuitivity 
> of the 6-tuple term arises in the error case when someone uses a 
> “clever” marking of a single 5-tuple flow with a mixture of DetNet and 
> non-DetNet DSCPs.  I believe that the result of that “cleverness” is 
> not multiple flows (which would be implied by the “6-tuple” term), 
> even though it may look like that in a DetNet node implementation.  I 
> think the result is better described as a single not-entirely-DetNet 
> flow (defined by a 5-tuple) that receives bizarre forwarding treatment 
> at DetNet nodes - some of the packets get DetNet forwarding treatment 
> and some get other forwarding treatment, so in a loaded network, 
> significant reordering is likely, and the receiver is probably not 
> going to be happy with the resulting packet arrival order.
>
> If this all makes sense, that “clever” mixing of DetNet and non-DetNet 
> DSCPs in a single flow (hence mixing DetNet and non-DetNet forwarding 
> behavior in a single flow) deserves a “SHOULD NOT” statement in the 
> draft, with a warning about the bizarre forwarding behavior at DetNet 
> nodes that is a likely result, which seems like a better thing to do 
> than my original suggestion for some sort of prohibition on mixing 
> DetNet and non-DetNet packets in the same 5-tuple.
>
> Thanks, --David
>
> *From:* Balázs Varga A [mailto:[email-masked]]
> *Sent:* Sunday, November 4, 2018 11:13 PM
> *To:* Black, David; [email-masked]
> *Subject:* RE: IP Solution problem: Use of DSCP and ECN fields in IP 
> - Masking for flow identification was considered as a general rule for 
> the “tuples” (i.e. not DSCP specific).
>
> *From:* detnet <[email-masked]> *On Behalf Of *Black, David
> *Sent:* Sunday, November 4, 2018 11:27 PM
> *To:* [email-masked]
> *Subject:* [Detnet] IP Solution problem: Use of DSCP and ECN fields in 
> IP headers for detnet flow identification
>
> We have a problem here …
>
> Starting with ECN (RFC 3168) – the 2-bit ECN field is intended to 
> enable ECN functionality to be
>
> the IP solutions draft has managed to violate all three “MUST” 
> requirements in that RFC 2474
>
> a)Abandon use of the ECN field for detnet flow identification.
>
> b)For the DSCP field, change from a bitmask approach to a list of DSCPs.
>
> a.I would note that a carefully chosen DSCP list can be implemented 
> via a bitmask.
>
> c)Define one or more Diffserv PHBs that realize DetNet behavior.
>
> a.I suspect that much of the content needed for this already exists in the
> +1 (774) 350-9323 *New *   Mobile: +1 (978) 394-7754
>
> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 8 Nov 2018 11:58:15 +0700

Lou,

I just talked this over with David and I agree with the proosal.

Thanks,
Andy

---

From: Lou Berger <[email-masked]>
Date: Thu, 08 Nov 2018 12:05:44 +0700

Excellent!

One thing I forgot to add here, and on today's slide is that this 
version/iteration of the IP will say that all routers along the path are 
presumed/required to support the DetNet ip solution.

Lou

----------
On November 8, 2018 11:59:00 AM "Andrew G. Malis" <[email-masked]> wrote:

> ----------

---

From: Greg Mirsky <[email-masked]>
Date: Thu, 8 Nov 2018 13:22:56 +0700

Hi Lou, et al.,
one clarification question:

   - is it expected that DetNet flows will use several DSCP values or all
   DetNet services will use one?

Regards,
Greg

---

From: Lou Berger <[email-masked]>
Date: Thu, 08 Nov 2018 13:46:32 +0700

----------

>    DetNet services will use one?

DetNet may use different/multiple values and there is (not yet at least) a 
DetNet dscps/phb.

Lou