Subject: [arch-d] Fwd: A Public Option for the Core

From: John Day <[email-masked]>
Date: Sun, 16 Aug 2020 06:18:41 -0400

> Begin forwarded message:
> 
> From: John Day <[email-masked]>
> Subject: Re: [arch-d] A Public Option for the Core
> Date: August 16, 2020 at 06:18:04 EDT
> To: John Grant <[email-masked]>
> 
> No, that is not true.  Your first mistake is that there is a control plane.
> 
> John
> 
>> On Aug 16, 2020, at 06:02, John Grant <[email-masked]> wrote:
>> 
>> On 16/08/2020 02:48, Joseph Touch wrote:
>>> *I* want apps to be able to get different service from the network and to pay differently for them if needed, but *never* to have the network infer or enforce that mapping.
>> Of course, if you're going to do that -- have the application negotiate a particular kind of service rather than have the network "inspect" the packets and guess what service is needed -- you need a control plane protocol (let's call it "signalling") to do the negotiation, and you no longer have the connectionless/stateless paradigm on which IP is founded.
>> 
>> -- 
>> John Grant
>> Nine Tiles, Cambridge, England
>> +44 1223 862599 and +44 1223 511455
>> http://www.ninetiles.com
>> 
>> _______________________________________________
>> Architecture-discuss mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/architecture-discuss
>

---

From: Joe Touch <[email-masked]>
Date: Sun, 16 Aug 2020 08:01:19 -0700

And the second is that service level implies connection. It only requires the network understands a label *I* control, rather than trying to infer one from the packet contents. 

Joe

> On Aug 16, 2020, at 3:19 AM, John Day <[email-masked]> wrote:
> 
> 
> 
>> Begin forwarded message:
>>

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 17 Aug 2020 08:24:06 +1200

> And the second is that service level implies connection. It only requires the network understands a label *I* control, rather than trying to infer one from the packet contents.

However, it does need some kind of flow state (whether flows are individual or aggregated) in the forwarding queues. Not that this is a new argument; we had it in the diffserv WG many years ago and it continues to this day in TSVWG.

I don't think this really affects the basic arguments for a "public core". It just indicates that transit networks don't need to be strictly best effort and stateless in order to be neutral.

   Brian
 
>>> *From: *John Day <[email-masked] <mailto:[email-masked]>>
>>> *Subject: **Re: [arch-d] A Public Option for the Core*
>>> *Date: *August 16, 2020 at 06:18:04 EDT
>>> *To: *John Grant <[email-masked] <mailto:[email-masked]>>
>>>
>>>> On Aug 16, 2020, at 06:02, John Grant <[email-masked] <mailto:[email-masked]>> wrote:
>>>>

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Aug 2020 09:46:37 +0200

John, Joe, Brian:

All great discussion points, but i am not sure if JohnD's starting
point of "no deductive adaptation of behavior" is really mandatory
and/or sufficient for *I* net neutrality. Aka: Do we even have
an approved and complete *I* "net neutrality" definition ? If so, where ?

Btw, IMHO: Having an application tell the network exactly what it
wants from the network is quite a challenging task for applications.
When i worked with enterprise application developers, they wouldn't
even want to bother to understand what differences in service from the
network would best help the application.

Hence we thought of a framework where the application would instead
attribute traffic as to what it is, e.g.: application FOO, wihthin FOO floX,
e.g.: X = audio/video/...  Aka: Whatever the application developer was able/
willing to tell.

Then one can have a trusted layer that would map this information to
actual network service requirements/advisory information.

In classical enterprise environments, this trusted mapping layer
could typically be a nework device (given how applications in enterprise
are subject to a lot of control by the enterprise operator anyhow).
In typical residential environment, the trusted mapping would likely
best be the browser, given how that performs all type of control
operations against applications anyhow.

Cheers
    Toerless

On Mon, Aug 17, 2020 at 08:24:06AM +1200, Brian E Carpenter wrote:
> >> ???

-- 
---
[email-masked]

---

From: John Day <[email-masked]>
Date: Mon, 17 Aug 2020 07:42:36 -0400

> On Aug 17, 2020, at 03:46, Toerless Eckert <[email-masked]> wrote:
> 
> John, Joe, Brian:
> 

Really? I said that?  Interesting. I wonder, when and even more, why?
> operations against applications anyhow.

Interesting conjecture. O, well.

John

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Aug 2020 15:23:31 +0200

Didn't say you said. Was referring to this, from Joe:

>>>> *I* want apps to be able to get different service from the network and to pay differently for them if needed, but       
+*never* to have the network infer or enforce that mapping.   

> Interesting conjecture. O, well.

That second part was just my view of how beyond best effort service differentiation
would likely best be embodied into prevailing architectures in different  market
segments. Opinions welcome.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Andrew Campling <[email-masked]>
Date: Mon, 17 Aug 2020 13:55:13 +0000

> and/or sufficient for *I* net neutrality. Aka: Do we even have an 
> approved and complete *I* "net neutrality" definition ? If so, where ?

You seem to be assuming that "net neutrality" is always a good thing?  My personal preference is for networks to be transparent in the way that they manage traffic (if at all) so that I can choose my network based on what they do.  A network without, for example, any form of QoS might perform very poorly for at least some applications.  

> network would best help the application.

This point I agree with.  In general, the level of knowledge of networks by applications developers can be very low (and vice versa), they exist in very different worlds.  
 
> attribute traffic as to what it is, e.g.: application FOO, wihthin FOO 
> floX, e.g.: X = audio/video/...  Aka: Whatever the application developer was 
> able/ willing to tell.
> 
> In classical enterprise environments, this trusted mapping layer could 
> typically be a nework device (given how applications in enterprise are 
> subject to a lot of control by the enterprise operator anyhow).

Whilst I believe that I understand your point, I'm not at all convinced that, in residential environments, delegation of trusted mapping to browsers makes a lot of sense.  That seems to present a fine attack service for future exploitation.  No doubt others will have different views though.  

Andrew

-----Original Message-----
From: Toerless Eckert <[email-masked]> 
Sent: 17 August 2020 08:47
To: Brian E Carpenter <[email-masked]>
Cc: Joe Touch <[email-masked]>; John Day <[email-masked]>; [email-masked]
Subject: Re: [arch-d] Fwd: A Public Option for the Core

John, Joe, Brian:

All great discussion points, but i am not sure if JohnD's starting point of "no deductive adaptation of behavior" is really mandatory and/or sufficient for *I* net neutrality. Aka: Do we even have an approved and complete *I* "net neutrality" definition ? If so, where ?

Btw, IMHO: Having an application tell the network exactly what it wants from the network is quite a challenging task for applications.
When i worked with enterprise application developers, they wouldn't even want to bother to understand what differences in service from the network would best help the application.

Hence we thought of a framework where the application would instead attribute traffic as to what it is, e.g.: application FOO, wihthin FOO floX,
e.g.: X = audio/video/...  Aka: Whatever the application developer was able/ willing to tell.

Then one can have a trusted layer that would map this information to actual network service requirements/advisory information.

In classical enterprise environments, this trusted mapping layer could typically be a nework device (given how applications in enterprise are subject to a lot of control by the enterprise operator anyhow).
In typical residential environment, the trusted mapping would likely best be the browser, given how that performs all type of control operations against applications anyhow.

Cheers
    Toerless

On Mon, Aug 17, 2020 at 08:24:06AM +1200, Brian E Carpenter wrote:
> >>>> Architecture-discuss mailing list [email-masked] 

--
---
[email-masked]

---

From: Joseph Touch <[email-masked]>
Date: Mon, 17 Aug 2020 07:47:04 -0700

Hi, Toerless,

> On Aug 17, 2020, at 12:46 AM, Toerless Eckert <[email-masked]> wrote:
> 
> ...

That’s an implementation detail inside the endpoint. The *network* should not be doing that inference:

a) because it is typically made to favor the provider, not the user
b) because it’s often incorrect, either because the needed info isn’t available (encryption) or deliberately obscured (running DNS over ports other than 53)

My broader concern is the latter part of the second one: I don’t like the impact of anti-inference strategies on network architecture. It tends to force designers to reinvent the Internet at other layers (inside HTTP, e.g.).

Ultimately, there are only a few things the network can do with a packet: forward it, queue it, or drop it. Having classes of service that don’t map directly to those behaviors is an invitation for unnecessary state and complexity anyway.

Back to neutrality, though, the point is that the *network* should be neutral to the user and use of a service, not that there should be only one service.

Joe

---

From: Joseph Touch <[email-masked]>
Date: Mon, 17 Aug 2020 08:03:59 -0700

> On Aug 17, 2020, at 6:55 AM, Andrew Campling <[email-masked]> wrote:
> 
>> ...  My personal preference is for networks to be transparent in the way that they manage traffic (if at all) so that I can choose my network based on what they do.

Hmm. How many *choices* do you have?

The last time I looked, over 1/3 of the US had only one ISP, i.e., no choice. I’m in the metro area of the second largest US city and have 2 “choices” for high-capacity connectivity (i.e., not cellular), one of which is nearly bankrupt.

And let’s say I get both of those - it’s not like I can run apps connected to different WIFI access points at the same time from a single host.

Joe

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Aug 2020 17:40:44 +0200

> You seem to be assuming that "net neutrality" is always a good thing?  My personal preference is for networks to be transparent in the way that they manage traffic (if at all) so that I can choose…

I was only asking for definitions of net neutrality.

I fully agree with your statement. I hope there is no definition of
net neutrality that would prohibit such differentiation of service
level objectives of different traffic (which hopefully is a broder
and more accurate term than QoS).

> This point I agree with.  In general, the level of knowledge of networks by applications developers can be very low (and vice versa), they exist in very different worlds.  

Most advanced SLO (service level objective) knowledge IMHO can be found
in applications resulting from vertically integrated development. Aka:
Applications written on behalf of an organization that is capable and
happy to provision network services to support SLO that makes the
application perform best: enterprises, self-owned data-centers, defense,
space, IoT (transportation, oil&gas, building integration,...).

service provider networks only came into play when all those mission
centric networks where outsourced or had to be merged into common
infrastructures. Forgetting this history is one of the key reason
why we are struggling of driving QoS/SLO support forward.

> Whilst I believe that I understand your point, I'm not at all convinced that, in residential environments, delegation of trusted mapping to browsers makes a lot of sense.  That seems to present a f…

Browsers do provide  a  well constrained  environment for applications,
and there is  an active SDO (w3c) defining those standards. Even with
some help from IETF (RTCweb for example). Thats why purely from the
perspective of an SDO, the chance to define standards is better for this
option.

If you take a random app
and tried for the OS to do the mapping, there is no standard.For example,
microsoft had a great mapping scheme as early as windows XP (or before,
i only started to use it with XP) to trigger diffserv/intserv. configured
for different apps by the windows administrator (e.g.: across an enterprise). 

I have seen enterprises well utilizing this windows infrastructure.
Alas, AFAIK, this never trickled through across other OSs because 
obviously Microsoft tried to leverage this as a competitive differentiator,
and the more application traffic became Internet centric, the less
relevant this became.

Likewise, all DPI/QoS mapping in routers is of course also vendor
proprietary, even though there is of course a lot of commonality across many
vendors.

Aka: From the perspective of an SDO the question is firt what the
architectural APIs are to be defined. The instances through which the
mapping could be done (browser,OS, router, other?) could then even be
proprietary adaptations (ideally standardized too).

Cheers 
    Toerless

-- 
---
[email-masked]

---

From: Christian Huitema <[email-masked]>
Date: Mon, 17 Aug 2020 08:19:59 -0700

On 8/17/2020 7:47 AM, Joseph Touch wrote:
> Hi, Toerless,
>
>> On Aug 17, 2020, at 12:46 AM, Toerless Eckert <[email-masked]
>> <mailto:[email-masked]>> wrote:
>>
> b) because it’s often incorrect, either because the needed info isn’t
> available (encryption) or deliberately obscured (running DNS over
> ports other than 53)

At the bottom are issues of knowledge and trust. As Toerless points out,
knowledge itself is hard. Application developers have a hard time
understanding the network service that they actually want. Even examples
that appear trivial, like video, are not: the video refresh rate and the
desired definition depends on whether the video is displayed as a post
stamp, as a square in a grid, or full screen, not to mention adaptive
coding to actual images. Asking applications to label their traffic
leads to compromises.

From the provider point of view, trusting the labels set by applications
is fraught with a different kind of compromise. What if every
application pretended to be a real time service requiring super high
quality of service? What prevents the application from lying? Would that
lead to collapse? Those fears are very much there, which is why network
providers tend to rely on DPI, and not use application labels much.

In theory, labels could be trusted if the users were paying for the
extra service. In practice, paying for extra service happens very
rarely. First, there is the knowledge issue, why would applications
incur extra costs without knowing why? Then, there is the
interconnection issue. Most network paths involve several network
providers. The one with a business relation with the user might be paid
for the extra service, but what about the others? That would require an
agreed framework allowing for example traffic providers to charge the
ISP for the labelled traffic that they emit. Good luck with that.

>
> My broader concern is the latter part of the second one: I don’t like
> the impact of anti-inference strategies on network architecture. It
> tends to force designers to reinvent the Internet at other layers
> (inside HTTP, e.g.).

Or to encrypt everything, which is very much the current trend. And to
add padding and cover traffic to try defeat fingerprinting. And possibly
to deploy end-to-end FEC.

> Ultimately, there are only a few things the network can do with a
> packet: forward it, queue it, or drop it. Having classes of service
> that don’t map directly to those behaviors is an invitation for
> unnecessary state and complexity anyway.
>
> Back to neutrality, though, the point is that the *network* should be
> neutral to the user and use of a service, not that there should be
> only one service.

I don't know that. Given the knowledge and trust issue,
one-size-fits-all looks actually very practical. That does mean a
compromise, because the network should allow those applications that
want low latency to get it, but should not limit the bandwidth that
applications can use if it is available. That probably requires more
research and development in active queue management, and in congestion
control algorithms that can cooperate with active queue management and
deliver the profile that fits the application.

-- Christian Huitema

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Aug 2020 18:17:38 +0200

On Mon, Aug 17, 2020 at 07:47:04AM -0700, Joseph Touch wrote:
> That???s an implementation detail inside the endpoint.

See my prior answeer to andrew. Just because something may end
up as an implementation inside an endpoint doesn't mean it couldn't
be good work to standardize the abstract API.

Also, what is a host. A bog with a bunch of containers orchestrated
together via linux bridges ? ;-)

Scott McNealy: The Network is the Computer.

Aka: Just a matter of how we model things.

I know, for a lot of old-time IETF'er, looking inside a physical
box s heresy, but hey, we are even getting TAPS interface RFCs
as standards track documents (horray!).

> The *network* should not be doing that inference:
> 

And folks in ISOC may argue that in other countries it will
favour governments. And i think we understand these challenges
and can build solutions accordingly.

> b) because it???s often incorrect, either because the needed info isn???t available (encryption) or deliberately obscured (running DNS over ports other than 53)

Sure, IMHO just a technical challenge we can solve. And remember
that in vertically integrated networks (see BrianC limited domains)
visibility by the network is desired:

In a vertial network solution, the application LOVEs to expose the RTP
header so that jitter,latency,loss,reordering per flow can be monitored
hop-by-hop by the network, problems quickly be isolated and fixed.

If an OOT application does not want that service, it can encrypt
everything. Why not.. I am sure the OOT will do exactly the same
unencypted headers within its own DC for troubleshooting, diagnostics.

Aka: the only problem we have in IETF is that we hear mostly from
only one side of the spectrum of relevant options, and thats the
OTT that do want to prohibit anything more than cheapest cost best
effort service, because that type of network service is the basis
of their business model and anything better wold be competitive to
their business.

> My broader concern is the latter part of the second one: I don???t like the impact of anti-inference strategies on network architecture. It tends to force designers to reinvent the Internet at other layers (inside HTTP, e.g.).

Yes. But even worse  is as i said above the influence mongering
to ensure that no alternative technologies are allowed to be worked
on in the IETF - by making the sham argument about what is or is not
good for "the Internet". Such as when SPUD/PLUS where acted against.

To make a funny analogy: There are use cases that rightfully so go
even much further in their anti-inference/perpass strategy than the OTT,
but they do not go running around the IETF trying to argue their way
is the only way the IETF should go.

Aka: If you want to see the (crrent) final word on protection against
perpass: draft-ietf-ipsecme-iptfs ;-))

> Ultimately, there are only a few things the network can do with a packet: forward it, queue it, or drop it. Having classes of service that don???t map directly to those behaviors is an invitation for unnecessary state and complexity anyway.

You forgot FEC insert/delete , duplication&merge, or even more general
network coding). And of course there is a huge variety of differences
in service level objective outcomes by how you forward (QoS routing),
and/or how you queue.

Having worked for decades on per-flow behavior in routers i do think
that the most interesting area to explore is one which does not require
per-flow state but allows to distinguish SLO on a per-packet level.
This would allow us to introduce INtServ like functionality with even
better than DiffServ scalability.

> Back to neutrality, though, the point is that the *network* should be neutral to the user and use of a service, not that there should be only one service.

Well... I would love to say "exactly right", but:

a) Is there actually any accepted definition of net neutrality
that says exactly this ?

b) Even that definition goes way beyond differentiation in any
other type of market. E.g.: If you are a tier1 OTT like e.g.:
netflix, amazom prime, your ability to get your cache into
pops of access SPs or peering bandwidth etc. is just going to be
a lot better than a tier2 OTT will get. Aka: We already today
have the same type of "volume of the deal" negotiations.

But we have this in any other market as well. German example of course
being: oh, you want to buy 50,000 cars for your employees...
of yourse you get better pricing and/or get them earlier or
get specials we don't offer to other customers.

Aka: How do we argue that selling of packets should be different than
selling of cars ?

Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 17 Aug 2020 18:42:56 +0200

On Mon, Aug 17, 2020 at 08:19:59AM -0700, Christian Huitema wrote:
> Given the knowledge and trust issue,

Thats again the look for networks we currently call "the Internet",
and quite valid there. But certainly not the only type of networks
IETF should be interested in.

When you look at managing traffic flows in e.g.: networks of an airplane,
you will see every individual flow being designed, and planned for before
the plane is built. And those parameters ar then embodied into the network
during plane building. I would call this the other end of the spectrum.

Instead of just talking about some of the tools (CC, ACM), it would be
nice to try to set goals first, to see if we could get agreement there:

-> I would like traffic to get the same throughput under congestion
   whatever its RTT is.

-> I would like for traffic to get bandwidth share indpendent of the
   number of 5 tuple flows it utilizes (no gaming the system). But
   rather fair per subscriber (weighted by how much the subscriber pays).

-> I would like to be able for traffic to indicate preferences for
   lower latency or higher throughput... If you ask for lower latency,
   you get less throughput.

-> Goodput always > 95%

That would be a start for the most simple "one-size-fits" all i could
imagine. But there is a lot more things i would like to factor in beyond
that.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Christian Huitema <[email-masked]>
Date: Mon, 17 Aug 2020 10:24:43 -0700

> -> I would like traffic to get the same throughput under congestion

I agree, but there is a fundamental issue. Flows with short RTT will
react faster to changing conditions than flows with shorter RTT, so
short term averages will always show some benefit to short delays.

>    rather fair per subscriber (weighted by how much the subscriber pays).

Yes. Basic fairness means all subscribers should be treated the same. At
the same time, there are some interesting questions. Assume that the
same function could be implemented as an app on my PC and or as a self
contained IOT device. Should the IOT device that has its own IP address
get more bandwidth than the app that shares the IP address with other
apps on the PC? I suppose that this is only a problem for local area
networks, since after that all traffic gets to share the subscribed
bandwidth, but I could see some needs here.

>    lower latency or higher throughput... If you ask for lower latency,

No. That's where we differ. We have tried designing this sort of
two-classes system in the past, modelling that as a game between the
application and the network, and trying to define strategies that were
robust against player's misbehavior. There is no sign that applications
would be ready for that.

On the other hand, there is plenty of research and deployment experience
for CC algorithms that deliver low latency if the network cooperates:
LEDBAT, BBR and L4S come to mind. There is also evidence of algorithms
that can lead to large delays especially in presence of buffer bloat:
Reno and Cubic. Let's assume that delay-sensitive applications use a
delay-minimizing CC algorithm. If the network "isolates" each
application in its own queue and serves them fairly, then the LEDBAT
users get low delay and the Cubic users get high bandwidth, and
everybody is happy.

> -> Goodput always > 95%
I don't know how to measure that. But yes of course, we don't want
congestion collapse.
> imagine. But there is a lot more things i would like to factor in beyond

Sure. But I like your general approach.

-- Christian Huitema

---

From: Joseph Touch <[email-masked]>
Date: Mon, 17 Aug 2020 11:52:06 -0700

> On Aug 17, 2020, at 8:19 AM, Christian Huitema <[email-masked]> wrote:
> 
>>> hen one can have a trusted layer that would map this information to
> At the bottom are issues of knowledge and trust.
> 

The mapping is hard, but inferring it is only harder (vs. letting the user/host decide).

As to trust, it only matters that an intermediate hasn’t forged a new header. You don’t need to trust the user - if they want to map their email to high-priority and pay for that usage, that’s their business (literally). If that’s more service than they paid for, then drop packets.

Or do you think UPS/DHL/etc should open every 2-day delivery to make sure it actually had to be there in 2 days?

Joe

---

From: Joseph Touch <[email-masked]>
Date: Mon, 17 Aug 2020 11:59:09 -0700

> On Aug 17, 2020, at 9:42 AM, Toerless Eckert <[email-masked]> wrote:
> 

Yes, as long as the issue isn’t that the ISP does something differently based on RTT>

If they run feedback loops whose recovery depends on RTT, then it’s their fault if they don’t get a fair share. It’s not the ISPs job to compensate for that.

>   rather fair per subscriber (weighted by how much the subscriber pays).

That’s 1 class of service. There can be more than one (or are you just treating that as ‘different subscribers’?).

>   lower latency or higher throughput... If you ask for lower latency,

Three network decisions:
	- go now
	- take up space and go later
	- drop

It seems like there ought to be three service classes (at least):
	- low latency and low loss (go now)
	- low loss with increased latency (go later)
	- everything else (drop if you can’t handle it)

That roughly maps to a few of the DSCPs. The rest are there largely to ensure that “some pigs are more equal than others”, i.e., that network management/control protocols can bump others. That sort of priority - with a single ‘reserved’ level - may be important to operations, but shouldn’t be sold out to users.

> -> Goodput always > 95%

I have no idea what goodput is. Nothing ensures that a packet that ‘wins’ on a link now won’t be dropped downstream when competing for other resources. If you want goodput to be high, then load has to be low (vs. capacity), i.e., under 50-60%-ish.

Joe

---

From: "John Levine" <[email-masked]>
Date: 17 Aug 2020 16:53:40 -0400

In article <[email-masked]> you write:
>As to trust, it only matters that an intermediate hasn’t forged a new header. You don’t need to trust the user - if
>they want to map their email to high-priority and pay for that usage, that’s their business (literally). If that’s more
>service than they paid for, then drop packets.

The problem is when you have networks connected A-B-C, network A sets
the header for deluxe service, then complains to B that C isn't
providing the service they're paying for.  What service? says C.

I realize that in the real world there's tons of VPNs and overlays
that provide various sorts of guarantees but they're a direct
arrangement between a customer and the network providing the service.

R's,
John

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 18 Aug 2020 00:50:26 +0200

On Mon, Aug 17, 2020 at 11:59:09AM -0700, Joseph Touch wrote:
> Yes, as long as the issue isn???t that the ISP does something differently based on RTT>
> 
> If they run feedback loops whose recovery depends on RTT, then it???s their fault if they don???t get a fair share. It???s not the ISPs job to compensate for that.

Thats a matter of opinion. many transportation services have elements
of equalization in them. postal charges within countries for example
(for most parts equal cost even when you live far away). Financial
market information needs to get more and more latency equalized for
market fairness.

> That???s 1 class of service. There can be more than one (or are you just treating that as ???different subscribers????).

Yes, i was talking about granularity of fairness.

> 	- take up space and go later

Yes. There is IMHO the not well explored opportunity to take up space
in already provisioned but unused buffers - for example for equalization
of latency where that is desirable.

But when you want lower latency you might want to get into one of the
fewer buffers served faster. And there would be fewer of those than
when you have more time to hang around. What impact would that have
on relative throughput ?

> 	- everything else (drop if you can???t handle it)

We understand how to manage classical diffserv/intserv solutions.
And how managing them is limited by scale and cost of management vs.
incremental sales revenue. The interesting question is what different
services you can reasonably multiplex so different apps get different
benefits but it doesn't end up simply every app wanting to game the
system by picking the one "winner" service.

For example, theoretically something like scavenger sounds like a
nice option. But applications would only use it if the non-scavenger
traffic was more expensive. Eg.: go beyond your monthly volume limit
with scavenger. Likewise the use of best-effort vs. low latency would
self regulate if the throughput loss of low-latency was making it
only worth for traffic that really needed lower latency.

> That roughly maps to a few of the DSCPs. The rest are there largely to ensure that ???some pigs are more equal than others???, i.e., that network management/control protocols can bump others. That sort of priority - with a single ???reserved??? level - may be important to operations, but shouldn???t be sold out to users.
> 
> > -> Goodput always > 95%
> 
> I have no idea what goodput is. Nothing ensures that a packet that ???wins??? on a link now won???t be dropped downstream when competing for other resources. If you want goodput to be high, then load has to be low (vs. capacity), i.e., under 50-60%-ish.

True, goodput is ambiguous as fairness. I was primarily thinking
of retransmissions to be below 5% (of course for non-low-latency traffic
only).

Cheers
   Toerless

-- 
---
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 18 Aug 2020 11:54:34 +1200

On 18-Aug-20 04:42, Toerless Eckert wrote:
...
 
>    rather fair per subscriber (weighted by how much the subscriber pays).

You just broke GridFTP, used in Big Science to move terabyte datasets
around the world efficiently. It's not gaming, it's achieving throughput
despite defects in TCP. But the topic is very much alive there, since
GridFTP is now unsupported.

For further reading:
https://twiki.cern.ch/twiki/bin/view/LCG/ThirdPartyCopy

    Brian

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 18 Aug 2020 07:26:41 +0200

Multiple parallel TCP connections to overcome TCP issues with high
capacity high loss paths even in the absence of congestion was always
a bad workaround. Digital Fountain was already selling software 15 years
ago with scatter storage and network coding gather retrieval. Still
network coding researchers  seem to claim this stuff is new today.

A lot of of the network side problems of this are the result of the
traditional, uncontrolled transit SP paths, aka: the classical Internet
model. 

Cheers
    Toerless

---

From: Joseph Touch <[email-masked]>
Date: Tue, 18 Aug 2020 12:11:30 -0700

> On Aug 17, 2020, at 1:53 PM, John Levine <[email-masked]> wrote:
> 

That’s not a problem. That’s just a contract and (potentially) a lawsuit, depending on which network claims that traversal would support deluxe service. If you can’t trust your ISP - from whom you buy deluxe service - to contract the correct service out to parties that honor the tags, then you’ve chosen poorly.

I.e., that isn’t a protocol problem. 

> arrangement between a customer and the network providing the service.

Direct or subcontract - at some point, if you don’t have that relationship with a peer, you can’t offer that as a transit.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Aug 2020 09:23:31 +1200

On 18-Aug-20 17:26, Toerless Eckert wrote:
> a bad workaround. 

It's certainly a workaround, but why was it particularly bad? The
context where GridFTP was developed and used was the world of very
high capacity links between major Big Data sites; it's actually a
good example for RFC8799 that we overlooked.

If you really want maximum throughput on such links, which also have
very low bit error rates, you can use a non-windowing, rate-controlled
transport protocol with simplified error handling. But that's not an
IETF problem, really.

> Digital Fountain was already selling software 15 years

The network coding literature goes back to at least 2000**.
The network coding people I know are aiming at a different problem
area today: poor quality wireless and/or overloaded satellite links.
Exactly the opposite of the Big Data scenario. (However, I am not
tracking the NWCRG. I am rather amazed by the lack of references
to the extensive literature in RFC8406, but draft-irtf-nwcrg-nwc-ccn-reqs
does better.)
 
> traditional, uncontrolled transit SP paths, aka: the classical Internet

Indeed. 

Regards
    Brian

** R. Ahlswede, N. Cai, S.-Y.R. Li and R.W. Yeung, “Network
Information Flow”, IEEE-IT, vol. 46, pp. 1204-1216, 2000.

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 18 Aug 2020 14:30:41 -0700

Just a note: while in nwcrg we have not addressed coding and storage there
is indeed a large amount of work in the area.

Marie-José Montpetit, Ph.D.
[email-masked]

On August 18, 2020 at 5:23:50 PM, Brian E Carpenter (
[email-masked]) wrote:

to the extensive literature insurance, but draft-irtf-nwcrg-nwc-ccn-reqs
does better.)

Indeed.

Regards
Brian

---

From: Andrew Campling <[email-masked]>
Date: Tue, 18 Aug 2020 21:52:22 +0000

On Aug 17, 2020, at 16:64, Joseph Touch <[email-masked]> wrote:

> On Aug 17, 2020, at 6:55 AM, Andrew Campling <[email-masked]<mailto:[email-masked]>> wrote:
>>
>> ...  My personal preference is for networks to be transparent in the way that they manage traffic (if at all) so that
>> I can choose my network based on what they do.
>
> The last time I looked, over 1/3 of the US had only one ISP, i.e., no choice. I’m in the metro area of
> the second largest US city and have 2 “choices” for high-capacity connectivity (i.e., not cellular), one
> of which is nearly bankrupt.
>
> And let’s say I get both of those - it’s not like I can run apps connected to different WIFI access
> points at the same time from a single host.

Because the UK mandates competition between ISPs at the retail level with wholesale network access, there is quite good choice from a range of different ISPs including both mass market and more niche providers.  I do note though that the situation in the US is very different.

This does of course helpfully underline the importance of not designing schemes based on the assumption that all markets are the same as the one with which we are most familiar.

Andrew

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 19 Aug 2020 05:02:39 +0200

On Wed, Aug 19, 2020 at 09:23:31AM +1200, Brian E Carpenter wrote:
> It's certainly a workaround, but why was it particularly bad?

Ships in the night (no coupled CC) between the multiple parallel TCP sessions
across the same path. Given how DoE HEP money was repsonsible for a lot
of US/EU transcontinental capacity even in the 80th/90th, there must have
been enough money around to look at this problem before 2000, but alas
progress was limited by having to muck around in BSD kernel for the most
part.

> The context where GridFTP was developed and used was the world of very

No worries. Everything exccept the paths used to grow amazon, facebook and
google was and is is limited domains ;-))

> transport protocol with simplified error handling.

I think one of the issues was that in the 90th a lot of the paths used
by researchers where still controlled/limited-domains, and when they
started to buy more into general purpose internet capacity, the available
paths became even worse due to bufferbloat, which maybe explaining
more focus on hose really bad high capacity, high loss paths later on.

> But that's not an IETF problem, really.

Why not. AFAIK it was, is and should be. Even had an FECFRAME WG for a while.

> The network coding literature goes back to at least 2000**.

Yes, but at that time the more exciting new usse of it was multicast,
only a few yers later did the technology became a mayor player to
solve the supposedly easer problem of bad high capacity unicast paths
(a least from the one off stories i was told). 

> area today: poor quality wireless and/or overloaded satellite links.

Sure. research wise they probably think the other areas are done, but
that still doesn't mean we have a good ubiquitously availale NC based
unicast reliable transport standardized. IPR played a big roole IMHO
in throttling wide ranging adoption/standardization.

Cheers
    Toerless

> 
> Regards
>     Brian
> 
> ** R. Ahlswede, N. Cai, S.-Y.R. Li and R.W. Yeung, ???Network
> Information Flow???, IEEE-IT, vol. 46, pp. 1204-1216, 2000.
> > 
> > Cheers
> >     Toerless
> > 

-- 
---
[email-masked]