Subject: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-03.txt

From: [email-masked]
Date: Mon, 02 Nov 2020 15:36:04 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Transport Area Working Group WG of the IETF.

        Title           : A Non-Queue-Building Per-Hop Behavior (NQB PHB) for Differentiated Services
        Authors         : Greg White
                          Thomas Fossati
	Filename        : draft-ietf-tsvwg-nqb-03.txt
	Pages           : 16
	Date            : 2020-11-02

Abstract:
   This document specifies properties and characteristics of a Non-
   Queue-Building Per-Hop Behavior (NQB PHB).  The purpose of this NQB
   PHB is to provide a separate queue that enables low latency and, when
   possible, low loss for application-limited traffic flows that would
   ordinarily share a queue with capacity-seeking traffic.  This PHB is
   implemented without prioritization and without rate policing, making
   it suitable for environments where the use of either these features
   may be restricted.  The NQB PHB has been developed primarily for use
   by access network segments, where queuing delays and queuing loss
   caused by Queue-Building protocols are manifested, but its use is not
   limited to such segments.  In particular, applications to cable
   broadband links and mobile network radio and core segments are
   discussed.  This document defines a standard Differentiated Services
   Code Point (DSCP) to identify Non-Queue-Building flows.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-tsvwg-nqb/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-tsvwg-nqb-03
https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-nqb-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-tsvwg-nqb-03

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Greg White <[email-masked]>
Date: Tue, 3 Nov 2020 00:14:48 +0000

All,

This update to the NQB draft adds a section (4) discussing the relationship of the NQB PHB to other PHBs in the DiffServ architecture, as well as a section (5.2) that discusses aggregation of NQB traffic with other DiffServ service classes.  In addition, it talks about interconnection & tunneling, and clarifies that WiFi equipment can support the NQB PHB requirements (in addition to the existing discussion of interoperating with legacy WiFi gear). 

Please see the diff linked below if you just want to review the updates.  I'd appreciate a more thorough review though, as IMO this is getting close to WGLC.  

Best Regards,
Greg

On 11/2/20, 4:36 PM, "tsvwg on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

    Abstract:

---

From: <[email-masked]>
Date: Tue, 3 Nov 2020 10:17:49 +0000

Hi Greg,

My comments on contents marked [RG0].

Section 4 contains:

"The intent of the NQB DSCP is that it signals verifiable behavior as opposed to wants and needs." And further down. "... simple verification of behavior." 

[RG0] Please provide text or a reference how to verify NQB behavior. Please also be clear, which verification mechanism is applicable in which part of an end to end path. As this is supposed to be "simple", that shouldn't be an issue.

####

Section 4 further contains:

"Also, the NQB traffic is to be given a separate queue with equal priority as default traffic, and given no reserved bandwidth other than the bandwidth that it shares with default traffic.  As a result, the NQB PHB does not aim to meet specific application performance requirements, nor does it aim to provide a differentiated service class as defined in [RFC4594].  Instead the goal of the NQB PHB is to provide statistically better loss, latency, and jitter performance for traffic that is not itself the cause of those degradations."

[RG0] First, I appreciate PHBs and DSCPs aggregating traffic rather than being assigned to applications.  

[RG0] "The NQB traffic is to be given a separate queue with equal priority as default traffic, and given no reserved bandwidth other than the bandwidth that it shares with default traffic."
[RG0] Please provide a configuration example for a commodity backbone router like, e.g.,  a Cisco ASR9k or a Juniper MX960. What you can configure
- a rate queue
- one bandwidth per queue
- Drop profiles, typically two per queue. Config options are low loss probability & high delay or high loss probability & low delay
Or
- Two priority queues - which share a joint bandwidth. If one is supposed to produce lower loss and lower delay than the other, while sharing the same capacity, it has to be prioritized over the other (and may starve it, unless it is rate limited).
[RG0] I don't understand how I would configure two separate queues with same priority and same bandwidth, one of which offers lower loss and delay than the other with existing gear. If that is impossible, please provide a reference to an implementation of an NQB scheduler in this section. 

[RG0] What the draft describes by a few words here to me sounds like a PHB group (see Usage of PHB Group, https://www.rfc-editor.org/rfc/rfc3260.txt ) of NQB and default Queue. Separate Queue for each PHB, same bandwidth shared between both PHBs, different loss probability and queue depth. From text in the tunneling section of your draft I'd conclude that NQB may be reordered against competing default traffic. Is that true? If yes, that's a part of the PHB  group definition in my mind. If NQB is not setting up a PHP group with default queueing, please provide a stand alone specification of NQB.

#####

Section 5.2 contains

"Nodes that support the NQB PHB may choose to aggregate other service classes into the NQB queue.  Candidate service classes for this aggregation would include those that carry inelastic traffic that has low to very-low tolerance for loss, latency and/or jitter as discussed in [RFC4594].  These could include Network Control, Telephony, Signaling, Real-Time Interactive and Broadcast Video."

[RG0] Unless you provide a " simple verification of [NQB] behavior" with provable functionality, I'd suggest not to recommend aggregating NQB or any other traffic, whose behavior and/ or access to resources can't be limited by the operator with "Network Control". RFC5127 clearly puts aside Network Control as a separate Treatment Aggregate. RFC8100 doesn't do so - the Network Control aggregate simply isn't accessible for external traffic, no matter how it is marked.

I'm stopping here.  No more remarks don't signal that these aren't there. 

Regards

Rüdiger

-----Ursprüngliche Nachricht-----
Von: tsvwg <[email-masked]> Im Auftrag von Greg White
Gesendet: Dienstag, 3. November 2020 01:15
An: [email-masked]
Betreff: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-03.txt

All,

Best Regards,
Greg

    Abstract:

---

From: Greg White <[email-masked]>
Date: Wed, 4 Nov 2020 23:35:17 +0000

Hi Rüdiger,

Thank you for reviewing the updated draft.  My responses to your comments are below, marked [GW].

-Greg

From: "[email-masked]" <[email-masked]>
Date: Tuesday, November 3, 2020 at 3:17 AM
To: Greg White <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: AW: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-03.txt

Hi Greg,

Section 4 contains:

[GW] To be clear, this was meaning that in an end-to-end context, verification of behavior is simple compared to verification of wants and needs.  It seems to me to be rather infeasible (and certainly more complex) to observe the packets of a flow and determine from the arrival pattern whether that application is more important (or more latency sensitive, loss sensitive, jitter sensitive, etc.) than another, or whether it contains the type of traffic that it claims to contain.  On the other hand, we’ve published one algorithm (referenced in the draft) that can be used to objectively verify behavior.  There are certainly other algorithms that could be used.  Given that the NQB “service” is not a guaranteed one, absolute precision is probably not needed.  Another approach that has been considered and would perhaps be useful in some scenarios is to use a per-flow rate-based re-marker (e.g. two color).  This space could be (and should be) open to innovations.

####

- a rate queue

Or

[GW] As mentioned in the draft, the NQB PHB is primarily intended for access network links (and it was presumed that NQB-marked traffic would most likely be treated as Default in backbones and interconnects).  That said, it shouldn’t be precluded that it be supported in backbone routers.  I’m not an expert on backbone router configurations and capabilities, but to me it sounds like the “Two priority queues” option may work if it is possible to set the scheduling weight or priority equal (or nearly equal) between the two queues.  If only strict priority is supported, then this would not be a good option.

[GW] That is provided in Section 6, but perhaps you didn’t get that far in your review.  I can add a forward reference if that would help other readers.

[GW] I don’t think it exactly qualifies as a PHB Group under the RFC2475 definition (and RFC3260 clarification), since the Default PHB is still meaningful on its own.  I agree that the NQB PHB may not be as useful* if it is not associated with the Default PHB, but the converse isn’t true.  But, I could be interpreting the definition more narrowly than it was intended (or perhaps there’s room to widen the definition to include the NQB-Default pair).   What would be the implications of defining it in this way?

*the requirements around the association with the Default PHB are currently all SHOULDs, so I think there is room for alternative configurations/implementations

#####

Section 5.2 contains

[GW] The scenario that I was considering here is one in which (for example) there are only 2 PHBs supported:  Default and NQB.  In that case it could be advantageous to aggregate these other service classes with NQB marked traffic, rather than aggregating them with Default.   I need to make this more clear.

Regards

Rüdiger

-----Ursprüngliche Nachricht-----

Von: tsvwg  Im Auftrag von Greg White

An: [email-masked]

All,

Best Regards,

Greg

On 11/2/20, 4:36 PM, "tsvwg on behalf of [email-masked]"  wrote:

            Authors         : Greg White

                              Thomas Fossati

         Pages           : 16

         Date            : 2020-11-02

    Abstract:

---

From: <[email-masked]>
Date: Thu, 5 Nov 2020 11:52:04 +0000

Hi Greg,

My replies in line, [RGrep1]… Regards, Rüdiger

Hi Rüdiger,

-Greg

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, November 3, 2020 at 3:17 AM
To: Greg White <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: AW: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-03.txt

Hi Greg,

Section 4 contains:

[RGrep1] Sections for innovations should be put on experimental rather than standards track. [I-D.ietf-tsvwg-ecn-l4s-id] section 'Safe' Unresponsive Traffic offers a specification which it characterizes as “vague but useful”. As compared to the definition given there (and in RFC8085), the text proposed by your draft to me is more vague. If draft tsvwg-nqb stays like that, to me it should be informational track. I doubt that an application implementer understands whether his traffic qualifies.

One of the examples application qualifying for NQB is “gaming” (with no further qualification given). Cloud Gaming traffic is NQB then? As compared to an n*100 Gbit/s peering, a single Cloud Gaming flow offers a “relatively low data rate…that doesn’t contribute to queueing delay and loss.”

####

- a rate queue

Or

[RGrep1] The introduction says “(NQB PHB), which is intended to enable networks to provide…” without further specification of “networks”. Please reword the draft to clarify that NQB PHB is intended for access network elements throughout the document.

[RGrep1] Oops, I see, I mistook two queues for one. Your text is ok up to, but not including the section regarding the “traffic protection”.

Coupled queues are a new requirement. I’m not sure whether VoQ will dominate the general access router QoS architecture. If so, NQB will have to be functioning for it (essentially, the queues are coupled on node ingress interfaces then and per egress policies have to be replicated to the relevant ingress interfaces).

Please be more clear in how the traffic protection is supposed to work. From contacts with my access node colleagues I’ve learned that per flow policies are very limited in scale. To them, one flow is roughly a single QoS policy, if I recall correctly. A reference to an expired ID is insufficient for a standards track doc.

[RGrep1] you recommend traffic protection which requires a feature moving traffic between different queues (impacting metering and scheduling on the final queue) an re-ordering of flows may result. This couples NQB and the default queue and result in a PHB group, I think.

#####

Section 5.2 contains

[RGrep1] I only speak about the network that I’m aware of. Network Control traffic MUST NOT share a PHB with any other kind of traffic. I of course can’t tell, whether other network providers share that philosophy, but I take RFC5127 as evidence that that’s the case. I don’t think that a standards track draft should suggest to change that, unless there’s operational proof that well behaving Network Control traffic under no circumstances is denied resources on NQB PHB in case of congestion.

Please be more precise in your draft to discern examples from standard specifications.

Regards

Rüdiger

-----Ursprüngliche Nachricht-----

An: [email-masked]<mailto:[email-masked]>

All,

Best Regards,

Greg

On 11/2/20, 4:36 PM, "tsvwg on behalf of [email-masked]<mailto:[email-masked]>"  wrote:

            Authors         : Greg White

                              Thomas Fossati

         Pages           : 16

         Date            : 2020-11-02

    Abstract:

---

From: Greg White <[email-masked]>
Date: Sun, 15 Nov 2020 22:49:02 +0000

All,

Rüdiger, David Black and I had a bit of offline discussion related to the current draft, and I’d like to summarize my takeaways from that discussion for the list.

  1.  The draft refers to DSCPs as being “standardized”, which doesn’t align with RFC2474 (which “standardizes” PHBs and only “recommends” DSCPs).
  2.  The current section 5.1 has some wording that indicates that existing standardized [sic] DSCPs are not expected to be used end-to-end.  While I think it is agreed that, in practice, end-to-end usage of IETF DSCP recommendations is not guaranteed (or perhaps even common) today, that was not the intent of the DiffServ architecture, so we shouldn’t make it sound as if it was.
  3.  The draft should more explicitly state that networks that do not support the NQB PHB SHOULD aggregate NQB traffic with Default, and SHOULD preserve the NQB marking (and make the case for why).  In addition, the draft should more clearly delineate the networks where it is believed that PHB support is most important (i.e. the ones discussed in the Use Cases section), vs those where it may be perfectly fine to aggregate NQB with Default (e.g. backbone networks).
  4.  Some interconnections and backbone networks (particularly those that use MPLS) have constraints on the number of service classes that can be supported (see RFCs 5127 & 8100 for examples), and thus will need to aggregate NQB traffic into the Default service class, and some existing implementations would more readily support the above recommendation about preserving marking if NQB used a DSCP other than 42. The draft should discuss this, and can suggest a DSCP value 000xxx be used in those situations. In addition, the draft shouldn’t refer to RFC8100 networks as being a special case here.
  5.  In Section 5.2, the draft suggests that Network Control traffic could be aggregated with other traffic into the NQB PHB.  At a minimum, that discussion needs more context to describe when such an aggregation might make sense, and provide some text that outlines the risks involved.  Alternatively, remove NC from the list of example service classes for aggregation (and perhaps explain why).
  6.  The WiFi use case section should be re-organized to emphasize a bit more that full support for NQB is possible (and recommended!) in WiFi networks by enabling two queues in the BE access category, and then secondarily discuss interoperation with the existing WiFi networks that have a single queue per AC and/or that use the default DSCP mapping.  The recommendation for RFC8325 devices should similarly describe both possibilities (and more strongly recommend NQB PHB compliance, rather than just assume that it isn’t possible).
  7.  Consider whether it would be beneficial to describe the NQB PHB as part of a PHB Group along with Default (no consensus on this).

I’ll try to walk through these briefly during the TSVWG meeting a few hours from now.  David or Rüdiger, if I’ve missed (or mischaracterized) anything important from our discussion, let me know.

-Greg

From: tsvwg <[email-masked]> on behalf of Greg White <[email-masked]>
Date: Wednesday, November 4, 2020 at 4:35 PM
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [tsvwg] I-D Action: draft-ietf-tsvwg-nqb-03.txt

Hi Rüdiger,

-Greg

Hi Greg,

Section 4 contains:

####

- a rate queue

Or

[RG0] What the draft describes by a few words here to me sounds like a PHB group (see Usage of PHB Group, https://www.rfc-editor.org/rfc/rfc3260.txt<https://protection.greathorn.com/services/v2/lookupUrl/e4b5d1cc-ca4a-4169-acba-4cc8a28d03e2/327/9bdecc420d07e07caf40c0099edb9d76f7e9e62c?domain=www.rfc-editor.org&path=/rfc/rfc3260.txt> ) of NQB and default Queue. Separate Queue for each PHB, same bandwidth shared between both PHBs, different loss probability and queue depth. From text in the tunneling section of your draft I'd conclude that NQB may be reordered against competing default traffic. Is that true? If yes, that's a part of the PHB  group definition in my mind. If NQB is not setting up a PHP group with default queueing, please provide a stand alone specification of NQB.

#####

Section 5.2 contains

Regards

Rüdiger

-----Ursprüngliche Nachricht-----

An: [email-masked]

All,

Best Regards,

Greg

            Authors         : Greg White

                              Thomas Fossati

         Pages           : 16

         Date            : 2020-11-02

    Abstract:

---

From: Sebastian Moeller <[email-masked]>
Date: Mon, 16 Nov 2020 08:59:10 +0100

Dear Greg,

all good points except for:

"In cases where a network operator is delivering traffic into an
   unmanaged WiFi network outside of their control (e.g. a residential
   ISP delivering traffic to a customer's home network), the network
   operator should presume that the existing WiFi equipment does not
   support the safeguards that are provided by the NQB PHB requirements,
   and thus should take precautions to prevent issues.  In these
   situations, the operator SHOULD deploy a policing function on NQB
   marked traffic that minimizes the potential for starvation of traffic
   marked Default, for example by limiting the rate of such traffic to a
   set fraction of the customer's service rate.

   As an additional safeguard, and to prevent the inadvertent
   introduction of problematic traffic into unmanaged WiFi networks,
   network equipment that is intended to deliver traffic into unmanaged
   WiFi networks (e.g. an access network gateway for a residential ISP)
   MUST by default remap the NQB DSCP to Default.  Such equipment MUST
   support the ability to configure the remapping, so that (when
   appropriate safeguards are in place) traffic can be delivered as NQB-
   marked."

Which clearly is not going to work, unmanaged Wifi networks are quite likely to be attached to non-NQB aware ISPs and hence there is nobody who can be tasked with neutering the NQB codepoint.
I know that you keep ignoring that point and argue that you addressed it, but you offer no actual solution to the actual issue. Which is surprising because the obvious and safe solution is to make Wifi priority treatment of NQB opt-in instead of opt-out. 
If I wished to donate all my WifI airtime to my neighboing DOCSIS-ISP's APs I would do so myself, I see little rationale for the IETF to help in such an bold airtime grab.

Best Regards
	Sebastian

> On Nov 15, 2020, at 23:49, Greg White <[email-masked]> wrote:
> 
> All,
>  
> 	• The draft refers to DSCPs as being “standardized”, which doesn’t align with RFC2474 (which “standardizes” PHBs and only “recommends” DSCPs).
> 	• The current section 5.1 has some wording that indicates that existing standardized [sic] DSCPs are not expected to be used end-to-end.  While I think it is agreed that, in practice, end-to-end usage of IETF DSCP recommendations is not guaranteed (or perhaps even common) today, that was not the intent of the DiffServ architecture, so we shouldn’t make it sound as if it was.
> 	• The draft should more explicitly state that networks that do not support the NQB PHB SHOULD aggregate NQB traffic with Default, and SHOULD preserve the NQB marking (and make the case for why).  In addition, the draft should more clearly delineate the networks where it is believed that PHB support is most important (i.e. the ones discussed in the Use Cases section), vs those where it may be perfectly fine to aggregate NQB with Default (e.g. backbone networks).
> 	• Some interconnections and backbone networks (particularly those that use MPLS) have constraints on the number of service classes that can be supported (see RFCs 5127 & 8100 for examples), and thus will need to aggregate NQB traffic into the Default service class, and some existing implementations would more readily support the above recommendation about preserving marking if NQB used a DSCP other than 42. The draft should discuss this, and can suggest a DSCP value 000xxx be used in those situations. In addition, the draft shouldn’t refer to RFC8100 networks as being a special case here.
> 	• In Section 5.2, the draft suggests that Network Control traffic could be aggregated with other traffic into the NQB PHB.  At a minimum, that discussion needs more context to describe when such an aggregation might make sense, and provide some text that outlines the risks involved.  Alternatively, remove NC from the list of example service classes for aggregation (and perhaps explain why).
> 	• The WiFi use case section should be re-organized to emphasize a bit more that full support for NQB is possible (and recommended!) in WiFi networks by enabling two queues in the BE access category, and then secondarily discuss interoperation with the existing WiFi networks that have a single queue per AC and/or that use the default DSCP mapping.  The recommendation for RFC8325 devices should similarly describe both possibilities (and more strongly recommend NQB PHB compliance, rather than just assume that it isn’t possible).
> 	• Consider whether it would be beneficial to describe the NQB PHB as part of a PHB Group along with Default (no consensus on this).
>