Subject: [tsvwg] NQB - which DSCP to recommend?

From: "Black, David" <[email-masked]>
Date: Fri, 15 Nov 2019 00:58:05 +0000

Lurking in the lengthy discussion about NQB and existing WiFi is a topic that needs broader WG attention, please.   Sebastian wrote:

> IMHO the upshot of this should be to

> a)  propose a DSCP for the NQB PHB that maps into AC_BE

The general topic is which DSCP should be the recommended DSCP for the  NQB PHB.  The NQB draft proposes 0x2A, but the WG may choose to select a different DSCP for this purpose.

Ok, please discuss ...

Thanks, --David
----------------------------------------------------------------
David L. Black, Senior Distinguished Engineer
Dell EMC, 176 South St., Hopkinton, MA  01748
+1 (774) 350-9323 New    Mobile: +1 (978) 394-7754
[email-masked]<mailto:[email-masked]>
----------------------------------------------------------------

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 15 Nov 2019 09:35:16 +0100

Hi David,

> On Nov 15, 2019, at 01:58, Black, David <[email-masked]> wrote:
> 
> Ok, please discuss …

	[SM] I propose to use 0x6 (000110) as DSCP for the NQB PHB on un-charcaterized networks, and to use 0x6A (101110) if the receiving network is known to contain NQB compatible wifi elements (or that network operator requested the re-mapping) as re-mapping to 0x6A (101110) will by default get packets into the desired AC_VI. 

	According to https://datatracker.ietf.org/meeting/99/materials/slides-99-tsvwg-sessb-31measurements-concerning-the-dscp-for-a-le-phb-00 partial bleaching can re-map a few common PHBs into the same DSCP (occurrence of partial bleach reported ~10%). But out of the "common" DSCPs only AF13, AF23, AF33, AF43, and EF carry 110 in their low three bits, and since all of the denote high priority (which partially indicates a request for low latency) this re-mapping does not see catastrophic and it certainly does carry the same cause priority inversion concerns that were relevant in selecting the LE PHB DSCP (https://tools.ietf.org/html/rfc8622). 

	I believe this to be a viable path forward that a) protects unsuspecting wifi networks from unintended side-effects of inceasing the use of AC_VI, yet b) allows ISPs to roll-out APs/wifi-routers equipped to handle NQB in the desired fashion. (By using a construct like hostapd's qos_map NQB-aware APs should even be able to instruct stations to map 0x6 into AC_VI if that should be desired).

Best Regards
	Sebastian

> [email-masked]

---

From: <[email-masked]>
Date: Fri, 15 Nov 2019 09:26:37 +0000

Hi Sebastian, hi David

I agree with your analysis. I don't object your proposed DSCP. Should there be a desire to circumvent the situation you describe, 000111 (or another spare 000dd1 DSCP) might be an option. As said, proposed only for the case that your suggestion doesn't reach consensus.

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: tsvwg <[email-masked]> Im Auftrag von Sebastian Moeller
Gesendet: Freitag, 15. November 2019 09:35
An: Black, David <[email-masked]>
Cc: [email-masked]
Betreff: Re: [tsvwg] NQB - which DSCP to recommend?

Hi David,

Best Regards
	Sebastian

> David L. Black, Senior Distinguished Engineer Dell EMC, 176 South St., 
> Hopkinton, MA  01748

---

From: "Jerome Henry (jerhenry)" <[email-masked]>
Date: Sat, 16 Nov 2019 00:44:02 +0000

Hi Sebastian, hi David,

I note that Sebastian's proposal clarifies that it " allows ISPs to roll-out APs/wifi-routers equipped to handle NQB", and as such targets residential WiFi routers. In this context (and if it is clarified in the draft), I have no objection.
I however note that in an enterprise environment, RFC 8325 support is expected, including all 8 TIDs. In this environment, 011110 would be a better choice imho.

Best

Jerome 

On 11/14/19, 11:27 PM, "tsvwg on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

    Hi Sebastian, hi David

    Regards,
    
    Ruediger
    
    -----Ursprüngliche Nachricht-----
    Von: tsvwg <[email-masked]> Im Auftrag von Sebastian Moeller
    Gesendet: Freitag, 15. November 2019 09:35
    An: Black, David <[email-masked]>
    Cc: [email-masked]
    Betreff: Re: [tsvwg] NQB - which DSCP to recommend?
    
    Hi David,

    

    Best Regards
    	Sebastian

---

From: Greg White <[email-masked]>
Date: Sat, 16 Nov 2019 07:25:27 +0000

TSVWG-ers,

We are proposing this PHB and a standardized DSCP that maps to AC_VI because we believe that it will provide a real benefit to residential broadband end-users worldwide (and our experimental usage to date has shown this to be true).   I believe the risk of unsuspecting users suffering as a result is entirely overblown.  I think the only thing that we agree on in that regard is that if the WiFi link is the bottleneck and if massive amounts of NQB marked traffic are delivered to the home network, it is possible to impact Default traffic.  But to have this occur in practice assumes that ISPs don't care about their customers, and are perfectly happy creating situations that result in trouble calls and/or truck rolls.  No ISP that I am aware of is interested in making their customers angry and adding to their operations costs.

Regardless of which DSCP the IETF selects, an ISP uninterested in providing the benefits of NQB to their customers (or one that is oblivious to NQB) can simply leave existing bleaching in place. 

If Sebastian's proposal is adopted, let's examine how this might play out.

An ISP interested in providing the benefits of NQB to their customers would presumably want to map 0x06 to AC_VI or AC_VO, so that the benefits are not lost when the traffic crosses a WiFi link. They could do this in ISP-provided gateways (either implementing a DSCP remapping, or using QOS_MAP), the upshot being that users of non-ISP-provided gateways fail to get the benefit of NQB.  Yes, they would be protected from the remote possibility of harm, but at a cost.   Another possible outcome is that an ISP remaps all NQB traffic to (e.g.) 0x2A in their core network, and this 0x06 value is only used across interconnects.  Since work would be needed on PE routers to turn off ingress bleaching of the NQB codepoint anyway, that seems like it might be the easiest place to map it to 0x2A.  Another possible outcome is that NQB applications ignore the IETF DSCP recommendation, and end up using a de facto DSCP in its place (possibly 0x2A), one that gets the benefit of segregated queuing provided by AC_VI (or AC_VO) across a broader user base, without creating remapping requirements for ISPs.  This would almost certainly be the case for uplink NQB traffic.

The latter two possibilities would result in NQB traffic arriving in all of the ISP's customers' home WiFi networks such that it maps to AC_VI or AC_VO anyway.  In cable access equipment, per-user bleaching of NQB traffic is straightforward, whereas per-user per-DSCP re-mapping isn't, so this approach would allow the ISP the ability to selectively enable/disable NQB marking on a per-user basis if need be.

The first option strikes me as being less likely anyway, since it involves new feature development on home routers.  Many ISPs do not write their own home router implementations.  

In all cases the ISP needs to ensure that supporting NQB on downlink traffic benefits their customers in the end.  This involves some type of control/policing of the NQB marking. This could be via traffic protection implemented on PHB compliant links, or it could be via other policies implemented in other network elements.

It seems to me that Sebastian's proposal simply creates a barrier to deployment and/or a degradation of the relevance of the IETF recommended DSCP, but no real protections.   

In my view, the sought after protections come instead from giving ISPs guidance on avoiding the potential for harm in WiFi networks that use existing equipment, as well as recommendations for upgrades to WiFi equipment to enable full PHB support (including traffic protection).  I am comfortable with David's suggestion to mandate that NQB PHB equipment that provides residential broadband service be capable of bleaching NQB to Default.  

I stand by the original proposal to assign 0x2A.

Greg

On 11/15/19, 2:27 AM, "tsvwg on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

    Hi Sebastian, hi David

    Regards,
    
    Ruediger
    
    -----Ursprüngliche Nachricht-----
    Von: tsvwg <[email-masked]> Im Auftrag von Sebastian Moeller
    Gesendet: Freitag, 15. November 2019 09:35
    An: Black, David <[email-masked]>
    Cc: [email-masked]
    Betreff: Re: [tsvwg] NQB - which DSCP to recommend?
    
    Hi David,

    

    Best Regards
    	Sebastian

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 16 Nov 2019 11:39:03 +0100

Dear All,

I believe that in a competition of proposals each side should be given the chance to make their arguments showing the merits of their position. I hence tried to objectively state arguments for 000101/101110.
But it seems that Greg intends to turn this into a debate instead. Suits me fine, please see more below in-line.

> On Nov 16, 2019, at 08:25, Greg White <[email-masked]> wrote:
> 
> TSVWG-ers,
> 
> We are proposing this PHB and a standardized DSCP that maps to AC_VI because we believe that it will provide a real benefit to residential broadband end-users worldwide (and our experimental usage to date has shown this to be true).

>   I believe the risk of unsuspecting users suffering as a result is entirely overblown.  I think the only thing that we agree on in that regard is that if the WiFi link is the bottleneck and if massive amounts of NQB marked traffic are delivered to the home network, it is possible to impact Default traffic.  

	[SM] No Greg, in that case there is not the "possibility" but the CERTAINTY that AC_BE traffic will be affected, both by increased latency and decreased "bandwidth", there is zero uncertainty in this situation. This is not a question of opinion that is a fact. Not accepting it is your prerogative but it does somewhat weaken your arguments about potential side-effects of your proposal on the existing wifi networks.

> But to have this occur in practice assumes that ISPs don't care about their customers,

	[SM] With NQB intended to pass the internet unchanged even for non-NQB aware ISPs, there will be loads of situations where there is no ISP that is even aware that there is the need could benevolently control NQB rates. (And as explained before for a shared medium like wifi the whole RF neighborhood will suffer from just a single ISPs/networks failure to properly care).

> and are perfectly happy creating situations that result in trouble calls and/or truck rolls.  No ISP that I am aware of is interested in making their customers angry and adding to their operations costs.

	[SM] Greg, please explain in detail how affected end-users are supposed to detect the issue in the first place and assign the root-cause to NQB-induced use of ACs > BE? A truck-roll will be initiated on an accepted support call, but do docsis ISPS send out a truck if a user complains about sluggish wifi, I don't think so.. As far as I know wifi packets carry no sign of the AC in the data structures, only the sending entity knows which EDCA parameter it used to get airtime, do you really expect end-users and ISP starting to query transmit statistics from all wifi devices in a home to figure out of observed traffic slow-down/starvation might be caused by AC_V[I|O] abuse do to NQB?
	This is a real engineering question, and I would like to hear an engineering answer instead of more spin, please.

> Regardless of which DSCP the IETF selects, an ISP uninterested in providing the benefits of NQB to their customers (or one that is oblivious to NQB) can simply leave existing bleaching in place.  

	[SM] Except there is no requirement for an uninterested ISP to bleach DSCPs in the first place, relaying on "accidental" bleaching as a safety mechanism strikes my a tad optimistic. Opt-in is by default safer than opt-out, or d=o you contest that?

> An ISP interested in providing the benefits of NQB to their customers would presumably want to map 0x06 to AC_VI or AC_VO,

	[SM] Any ISP mapping NQB to AV_VO is not going to provide an unqualified benefit to its users, on 802.11n AC_VO does not aggregate, so using it will noticeably reduce the attainable goodput at any MCS. Please get your facts straight before starting to go into the details.

> also that the benefits are not lost when the traffic crosses a WiFi link.

	[SM] IMHO this is a policy decision the administrator of that wifi link needs to make, if that is the ISP fine, but if the AP is operated by the an end-user I want to see that end-user in control.

> They could do this in ISP-provided gateways (either implementing a DSCP remapping, or using QOS_MAP), the upshot being that users of non-ISP-provided gateways fail to get the benefit of NQB.  

	[SM] But at the same time they will not have to suffer the negative side-effects of NQB as AC_VI as well, and these are the users whose networks presumably are not NQB aware. This is a trade-off and hence a policy decision that needs to be placed at the hands of the party responsible to deal with potential side effects and for non-ISP-provided APs (whether the AP serves as gateway or not is irrelevant) the responsible party is not the ISP not is it the IETF.

> Yes, they would be protected from the remote possibility of harm, but at a cost.  

	[SM] They would be just as fine as before, calling a foregone potential gain a cost seems a bit far fetched to me.

> Another possible outcome is that an ISP remaps all NQB traffic to (e.g.) 0x2A in their core network, and this 0x06 value is only used across interconnects. Since work would be needed on PE routers to turn off ingress bleaching of the NQB codepoint anyway, that seems like it might be the easiest place to map it to 0x2A.  

	[SM] That would be against my proposal to only set the CS5 upper bits, if the receiving leaf network is positively KNOWN to be NQB-aware. 

> Another possible outcome is that NQB applications ignore the IETF DSCP recommendation, and end up using a de facto DSCP in its place (possibly 0x2A), one that gets the benefit of segregated queuing provided by AC_VI (or AC_VO) across a broader user base, without creating remapping requirements for ISPs.
>  This would almost certainly be the case for uplink NQB traffic.

	[SM] Well, the uplink marking is at least in theory under the user's control, and most OSs allow to override an applications marking decision (although this is not straight forward, and probably not in the tool-kit of most end-users), but on ingress that is simply not an option for most users. 

> 
> The latter two possibilities would result in NQB traffic arriving in all of the ISP's customers' home WiFi networks such that it maps to AC_VI or AC_VO anyway.  

	[SM] But that would be counter the RFC, assuming that my DSCP proposal for NQB is accepted. That is all an RFC can do, propose a reasonable safe way to implement NQB in the existing internet, it can not force implementation, but that is a rather weak argument against well-reasoned and safe requirements in an RFC. That is akin to noting that since crime happens, penal law should be abolished, since it obviously does not extinguish all crime.

> In cable access equipment, per-user bleaching of NQB traffic is straightforward, whereas per-user per-DSCP re-mapping isn't,

	[SM] Well, this is why I mentioned QOS_MAP as that allows to instruct an AP to assign 000101 into AC_VI, no re-mapping needed, and an ISP will have sufficient control over its supply line to make sure the qos_map feature is available and properly configured. This seems like a viable technical solution to this specific short coming of cable access equipment.

> so this approach would allow the ISP the ability to selectively enable/disable NQB marking on a per-user basis if need be.
> 

	[SM] I assume that these ISPs will have a list of requirements/features already they use to select their wifi route models? Increasing that list by one-bullet point "implement/configure qos_map" does not strike me as overly onerous on an ISP, especially since this requirement only exists if the ISP wants to support NQB (my proposal is bey default safe for those who do not act). Also, if you argue that this is unlikely/hard for ISPs, it will effectively impossible for end-users. IMHO you can not have it both ways.

> It seems to me that Sebastian's proposal simply creates a barrier to deployment and/or a degradation of the relevance of the IETF recommended DSCP, but no real protections.   

	[SM] Care to elaborate? In my proposal the negative side-effects are minimized by design (by prioritizing NQB over wifi only if it is highly likely to be safe to do so), while in your proposal you simply hope that ISP will do the right thing in their quest that their users will benefit? One of our position strikes me as overly optimistic...

> In my view, the sought after protections come instead from giving ISPs guidance on avoiding the potential for harm in WiFi networks that use existing equipment, as well as recommendations for upgra…

	[SM] If NQB-over-wifi-prioritization offers real advantages to ISPs and end-users, than surely an opt-in roll-out that requires user/ISP action will not be an insurmountable obstacle. And if that energy can not be mustered, excuse me, it would indicate a lack of merit on the NQB idea.

> I stand by the original proposal to assign 0x2A.

	[SM] And I stand be my assessment that the consequences this will have on the existing non-NQB-aware wifi networks are not sufficiently taken into account. I am wil lingto change my position if shown real data that reliably demonstrates that I am overly pessimistic and that there is no realistic danger for NQB abuse. So far, Greg has not shared the experimental data he bases his safety assessments on, I am beginning to wonder whether such data exists at all.

Sebastian

P.S.: Greg I am also waiting on the data supporting that the "EDCA manipulation" proposed in your draft a) gives NQB the desired isolation from AC_BE traffic and b) is otherwise side-effect free on the existing wifi APs out in the field as this is a modification that certainly has not been tested by the OEMs. I do not believe it to be inadequate or rude to expect that RFC proposed remedies have had even a modicum of real-world testing.

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 16 Nov 2019 16:27:18 +0100

Dear All,

to cut though too many words and detailed arguments, the choice before us seems ultimately between:

A) a DSCP that will give NQB precedence on most existing wifi networks out there, like Greg's0x2A, on the hope that this will have no/little negative side-effects, but maximizing the number of users/networks that potentially can enjoy NQB's promise of low latency over wifi networks.

B) a DSCP that will map NQB into AC_BE and hence will not give NQB precedence over "normal traffic" by default, like my proposed 0x6, knowing that there will be no side-effects, but limiting the number of users seeing the "full potential" of the NQB PHB over wifi.

So it is effectively a decision between potential gain and risk. 
IMHO the risk is substantial and easily confirmed in the lab (run flent's rrul_cs8 test through an WMM AP and see the higher AC's dominate AC_BE) but hard to diagnose in the field, while the gain is mostly based on lab experiments and not guaranteed to to actually work as designed out in the real world (at least I have not seen any published data). 
	So I vote for B), it does not necessarily be 0x6, as long as it by default stays in AC_BE. So Jeromes proposal of placing NQB "into CS3" is just as acceptable to me. 

Best Regards
	Sebastian

---

From: Jonathan Morton <[email-masked]>
Date: Sun, 17 Nov 2019 09:51:27 +0800

> On 16 Nov, 2019, at 11:27 pm, Sebastian Moeller <[email-masked]> wrote:
> 

I agree with Sebastian here.

To put a finer point on the argument, note that the negative effects on wifi performance are most likely to be noticeable to close *neighbours* of each subscriber using NQB traffic, if the latter falls into AC_VI.  This is simply due to the way that the wifi MAC contends for the shared RF medium.  This is technically known as an "externalised risk", and is something the IETF should be very concerned with avoiding.

 - Jonathan Morton

---

From: Pete Heist <[email-masked]>
Date: Sun, 17 Nov 2019 16:22:21 +0800

> On Nov 17, 2019, at 9:51 AM, Jonathan Morton <[email-masked]> wrote:
> 

Hi Greg, while on the ‘ietf’ WiFi network, I tested traffic with fping and irtt as follows:

fping -O 0 -c 10 <router ipv4 addr> # all round trips passed
fping -O 0x20 -c 10 <router ipv4 addr> # CS1, all round trips passed
fping -O 0x80 -c 10 <router ipv4 addr> # CS4, no round trips passed
fping -O 0xa0 -c 10 <router ipv4 addr> # CS5, no round trips passed
fping -O 0xa8 -c 10 <router ipv4 addr> # EF, no round trips passed

irtt client -d 10s --dscp=0 <irtt server ipv6 addr> # all round trips passed
irtt client -d 10s --dscp=0x20 <irtt server ipv6 addr> # CS1, all round trips passed
irtt client -d 10s --dscp=0x80 <irtt server ipv6 addr> # CS4, only first round trip passed
irtt client -d 10s --dscp=0xa0 <irtt server ipv6 addr> # CS5, only first round trip passed
irtt client -d 10s --dscp=0xa8 <irtt server ipv6 addr> # EF, only first round trip passed

Note that the -O flag for fping (on Mac) and the (poorly named) --dscp flag for irtt both specify the _entire_ DS (former TOS) field.

The results suggest that, regardless of which DSCP value is chosen, applications may need to be prepared to fall back to DSCP 0. Perhaps worth an addition to the draft?

---

From: <[email-masked]>
Date: Mon, 18 Nov 2019 07:51:40 +0000

Greg,

the situation I'm in is:
- DiffServ is deployed since more than a decade and multiple DSCP are used
- DiffServ compliant interconnection works with re-marking as an operational commodity. RFC 8100 is deployed.
- The Layer 2 and Layer 3 access Wholesale market over here is regulated and that partially includes specification 
  of access network DSCPs and corresponding Ethernet priorities. 
- I suggest to use a 000xxx DSCP, if a DSCP identifying a specified PHB not requiring an SLA at interconnection 
  should arrive a customer access with an unchanged DSCP. 

At an Interconnection point of the backbone that I'm working with, a DSCP 101xxx is interpreted as Telephony traffic. If there's an SLA in place, the EF PHB is honored and suitable charges apply. If there's no SLA in place, this traffic continues being transported by the default PHB and will be marked appropriately.

Standardised E2E PHBs are a good idea, but DiffServ isn't greenfield. To me, changes in the DiffServ architecture (like standard E2E DSCPs) must respect operational deployments at interconnection. I don't think that's simple, but it may be possible, if all parties involved can agree on a joint way forward.

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: Greg White <[email-masked]> 
Gesendet: Samstag, 16. November 2019 08:25
An: Geib, Rüdiger <[email-masked]>; [email-masked]; [email-masked]
Cc: [email-masked]
Betreff: Re: [tsvwg] NQB - which DSCP to recommend?

TSVWG-ers,

Greg

    Hi Sebastian, hi David

    Regards,
    
    Ruediger
    
    -----Ursprüngliche Nachricht-----
    Von: tsvwg <[email-masked]> Im Auftrag von Sebastian Moeller
    Gesendet: Freitag, 15. November 2019 09:35
    An: Black, David <[email-masked]>
    Cc: [email-masked]
    Betreff: Re: [tsvwg] NQB - which DSCP to recommend?
    
    Hi David,

    

    Best Regards
    	Sebastian

---

From: Markku Kojo <[email-masked]>
Date: Mon, 18 Nov 2019 20:04:54 +0200 (EET)

Hi Greg,

I'm not commenting about which DSCP to select but would like to understand 
a somewhat related issue. I well understand that the NQB DSCP is 
intended for non-congestion-controlled traffic. However, what is the 
intended treatment if a NQB packet also has ECT(0) or ECT(1) set for some 
reason. The draft is quite clear what happens if the traffic turns out 
being QB, but how about if it does not? Does it happily get the same AQM 
treatment as NQB traffic with not-ECT set but also with the same ECN 
marking behavior as BE traffic with ECT(1) set?

Thanks,

/Markku