Subject: [re-ECN] Fwd: Pls bash: Congestion Exposure (re-ECN) BoF in Hiroshima?

From: Bob Briscoe <[email-masked]>
Date: Mon, 07 Sep 2009 11:19:19 +0100

Folks,

Attached is my attempt so far. I started again - I'm happy with it so 
far, but it needs the specifics added at the end, where indicated.

I'm sending in case I don't get good connectivity while travelling. 
Once I'm done, I'll send a complete copy. But this gives something 
for you to push back on or for you to propose alternative text.

Apologies for sending an attachment (in a hurry).

Bob

>Date: Sat, 05 Sep 2009 13:31:14 +0100
>To: "Woundy, Richard" <[email-masked]>, 
>"COURCOUBETIS, Costas" <[email-masked]>, Steven BLAKE 
><[email-masked]>, Marcelo BAGNULO BRAUN 
><[email-masked]>, "MONCASTER, Toby" <[email-masked]>, 
>"Agarwal, Anil" <[email-masked]>, Tom Taylor 
><[email-masked]>, Ken Carlberg <[email-masked]>
>From: Bob Briscoe <[email-masked]>
>Subject: RE: [re-ECN] Fwd: Pls bash: Congestion Exposure (re-ECN) 
>BoF inHiroshima?
>Cc: re-ECN unIETF list <[email-masked]>

[snip]

>I'm off to a wedding for the rest of the day. I'll get back to this 
>first-thing (UK time) Sunday.

>Here's a suggested proposal outline:
>I'm aiming for something as brief as possible (e.g. 1-2pp).
>
>1. Intro
>   1 para top level motivation: Accountability for Congestion
>   1 para ambitious, so we have to bite off smallest self-contained chunk
>   1 para which particular bites to take (using an expt approach like LISP):
>     a) (INF) recording motivation(s)
>     b) (EXP) base congestion exposure protocol
>     c) (STD) process pre-requisites to do (b)
>     d) (INF) reports on experiments
>   1 para where other stuff is getting done, e.g. ICCRG
>
>2. A little more on each proposed working-group activity
>2.1 Motivation
>     Accountability for Congestion
>     Good fences make good neighbours
>     - IETF not been good at doing this (NATs, firewalls)
>     - this is a chance to do it well
>     Vision
>     - ECN gives all traffic tiny jitter & loss
>     - congestion accountability handles other QoS dimension; b/w allocation
>     - that's QoS sorted :)
>2.2 Protocol work
>        prob re-ECN, but open to suggestions
>        IPv4, IPv6 & TCP as example transport (for now)
>2.3 IETF Process
>     Depends on protocol encoding chosen
>     Current view:
>       need bit 48 in IPv4 hdr & IPv6 extension hdr + clash with ECN nonce
>     Planned assignment of required field(s) as experimental
>     Guidelines on how to confine experimental values (in space & time)
>2.4 Reports on Experiments
>     This w-g NOT designed to standardise uses of the protocol
>     - e.g. policers, new congestion controls, simpler QoS,
>       inter-domain metering, traffic engineering, DDoS miitigation
>     But w-g will act as a focus for expts & trials in using its protocol
>     Will produce reports on role of congestion exposure in trials, issues,
>     recommendations, re-thinks, etc.
>     Informs any future move from experimental to stds track
>2.5 (Optional) Focused work on deployment?
>     This is more than the minimum work that the w-g needs to bite off
>     But it's the most important gating factor
>     Therefore, it could form a focused piece of work in its own right
>     Survey of middleboxes that will break ECN, re-ECN etc.
>     Permanent partial deployment (user & net choice to expose congestion)
>     Incremental deployment outline & incentives
>
>3. Proposed BoF Agenda
>    Motivations (which main motivation?)
>    Demo (what demo?)
>    Misconceptions
>     - congestion (with ECN) != impairment
>     - uncongested path != good (a symptom of broken transport protocols)
>     - exposing congestion != operator privacy concerns
>    Brief protocol outline
>    Relationship to other w-gs
>    Community - who's doing what; who's planning what
>    Questions to put to a vote
>
>
>Bob

Bob

________________________________________________________________
Bob Briscoe,               Networks Research Centre, BT Research  

IETF BoF Proposal: Congestion Exposure (re-ECN)

The Internet is all about sharing capacity between multiple users -
that's what packet multiplexing is all about. But the IETF (and the
wider industry) is realising we really don't understand how best to
share out capacity.

In Nov 2008, the Transport Area of the IETF asked the IRTF Congestion
Control Research Group (ICCRG) to set direction on this challenge and
good progress is being made. In parallel, this BoF proposal brings
together those who want to form a working group to experiment with one
of the more promising approaches. The aim is not to pre-judge the
outcome of the ICCRG's deliberations, but to investigate the practical
issues that will arise in defining and using one of the more promising
options: congestion exposure.

The relationship between the LISP w-g in Int Area and the IRTF Routing
Research Group provides a close analogy for the proposed process.

Congestion Exposure?

The premise is that we are having trouble sharing capacity because the
information needed to share capacity properly isn't made visible at the
internetwork layer. Specifically, the idea is to make expected
congestion on the rest of the path visible in the IP header of each
packet as it enters the internetwork. A protocol called re-ECN
(re-inserted explicit congestion notification) has been proposed to do
this. Re-ECN is the strongest candidate for adoption, but of course the
proposed working group might want to redesign it, or adopt an
alternative if one surfaces.

Whatever protocol is adopted, it should be possible for a network
operator to count the volume of congestion about to be caused by an
aggregate of traffic, as easily as it can count the volume of bytes
entering its network today - it just counts only the volume of those
packets that carry a congestion marking. This can be done for each user,
or for whole attached networks.

There is no intent to change the existing approach, where congestion is
detected and responded to by transports on endpoints (e.g. congestion
control in TCP, RTP/RTCP). The only difference is that network operators
will be able to discourage the one factor that causes grief to their
other customers: the volume of congestion.

The proposed working group will NOT define how exposed congestion should
be used. It will confine itself to the focused task of defining the
protocol for exposing congestion. However, it will definitely encourage
informational descriptions of how exposed congestion was used in various
experiments, to assess whether the way the protocol exposes congestion
is fit for purpose - where different people will have different
purposes.

Congestion exposure seems a rather unexpected direction to take, but a
growing community is realising just how powerful this approach can be;
and it sheds light on why we have misunderstood capacity sharing in the
past.

The work of the LEDBAT working-group (low extra delay background
transport) is a good example of the direction we ought to be taking, and
it also highlights that congestion exposure will be needed. LEDBAT is a
congestion control intended for background traffic. It yields far more
than TCP does to competing traffic. It senses congestion earlier than
other congestion controls (by noticing an increase in round trip time),
so it uses a smaller share of capacity when other traffic is present.

LEDBAT illustrates two points:
1. It is not appropriate (`fair') for all flows to get the same rate as
TCP would through a bottleneck (TCP-friendliness);
2. It would not be fair for an ISP to count the volume of bytes
transferred in order to limit heavy users. A certain volume of LEDBAT
bytes causes significantly less harm to other uses of the Internet than
the same volume of bytes transferred using TCP (and an unresponsive UDP
would be even worse).

In a Transport Area plenary straw poll at the March '09 IETF, there was
zero support for TCP-friendliness as a way forward for the IETF. But we
also need to recognise that LEDBAT is only a first step, not the whole
answer. For instance, a 100kB flow will want 10MB flows to yield into
the background, but similarly, the 10MB flow will want a 10GB flow to
yield further into the background. And operators need a metric to ensure
transports have the incentive to push back or yield as appropriate.
Getting this right will allow the smaller flows to complete much sooner,
but then free up capacity earlier so that the larger flows can still
complete nearly as quickly.

TBA: Vision. Relax transport constraints. Congestion != impairment.

TBA: Proposed working group deliverables

TBA: Proposed BoF Agenda.

---

From: <[email-masked]>
Date: Mon, 7 Sep 2009 11:36:06 +0100

Immediate top-level comment - drop the re-ECN from the title. This is a
BoF where we are trying to get the IETF to agree there is a need to
introduce congestion transparency. Re-ECN is a specific protocol for
doing that but there may be others so we shouldn't put it in the title.

I really fear the overall order of things is wrong as well. The bulk of
the first 3 paragraphs is just about IETF processes and the IRTF... The
first paragraph is fine but you need to expand on that and get quickly
towards a summary of the problem (the IETF hasn't provided a proper
system on which to build network accountability so ISPs have started to
bodge their own, with dire consequences for the future of the network).

I think we need to re-phrase quite a bit of the detailed stuff as well,
but that is a matter of editing rather than complete change of meaning
so I will leave it for now...

Final thing - this is already starting to get too long. The MPTCP BoF
description was ~600 words in total, TANA 9pre-cursor to LEDBAT) was
~450 total). You are already at 750 and you have 3 major bullets with no
text! In other words we need to cut by about 50%...

Toby

> -----Original Message-----
> From: Briscoe,RJ,Bob,XVR9 BRISCORJ R
> Sent: 07 September 2009 11:19
> To: Woundy, Richard; COURCOUBETIS, Costas; Steven BLAKE; Marcelo
> BAGNULO BRAUN; Moncaster,T,Toby,DER3 R; Agarwal, Anil; Tom Taylor; Ken
> Carlberg; Leslie Daigle; BOWMAN Don
> Cc: re-ECN unIETF list
> Subject: Fwd: [re-ECN] Pls bash: Congestion Exposure (re-ECN) BoF in
> Hiroshima?
> 
> Folks,
> 
> Once I'm done, I'll send a complete copy. But this gives something for
> you to push back on or for you to propose alternative text.
> 
> >Subject: RE: [re-ECN] Fwd: Pls bash: Congestion Exposure (re-ECN) BoF
> >inHiroshima?

---

From: <[email-masked]>
Date: Mon, 7 Sep 2009 11:52:59 +0100

Note for non-British people - Bodge = "A clumsy or inelegant job,
usually a temporary repair") http://en.wiktionary.org/wiki/bodge - they
might also have added usually involving the use of copious amounts of
gaffer tape!

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: 07 September 2009 11:36
> To: Briscoe,RJ,Bob,XVR9 BRISCORJ R; [email-masked];
> [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked];
[email-masked];
> [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: Re: [re-ECN] Pls bash: Congestion Exposure (re-ECN) BoF
> inHiroshima?
> 
> for
> BoF
> > >   1 para which particular bites to take (using an expt approach
> like
> > LISP):
> > >       need bit 48 in IPv4 hdr & IPv6 extension hdr + clash with
ECN
> > nonce
> _______________________________________________
> re-ECN mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/re-ecn

---

From: João Taveira Araújo <[email-masked]>
Date: Mon, 07 Sep 2009 12:13:50 +0100

Agree with most of Toby's comments, and would add that the last 3 
paragraphs seem too long to basically convey:

- LEDBAT shows that there is a need for more flexible CC paradigms than 
TCP-friendliness
- LEDBAT itself is only a solution to a subset of problems, and we need 
an overarching framework to:
  - expose congestion so operators are no longer prone to information 
assymetry.
  - allow different CC algorithms to evolve independently within less 
restrictive design space than TCP-friendliness.

Joao

[email-masked] wrote:
>>>   1 para ambitious, so we have to bite off smallest self-contained
>>>       
>> chunk
>>     
>>>   1 para which particular bites to take (using an expt approach like
>>>       
>> LISP):
>>     
>>>     - congestion accountability handles other QoS dimension; b/w
>>>       
>> allocation
>>     
>>>     - that's QoS sorted :)
>>> 2.2 Protocol work
>>>       need bit 48 in IPv4 hdr & IPv6 extension hdr + clash with ECN
>>>       
>> nonce
>>     
>>>     Guidelines on how to confine experimental values (in space &
>>>       
>> time)
>>     
>>>     But w-g will act as a focus for expts & trials in using its
>>>       
>> protocol
>>     
>>>     Will produce reports on role of congestion exposure in trials,
>>>       
>> issues,
>>     
>>>     This is more than the minimum work that the w-g needs to bite
>>>       
> off
>   
>>>     Therefore, it could form a focused piece of work in its own
>>>       
> right
>   
>>>     Permanent partial deployment (user & net choice to expose
>>>       
>> congestion)
>>     
>>>     - uncongested path != good (a symptom of broken transport
>>>       
>> protocols)
>>

---

From: "Woundy, Richard" <[email-masked]>
Date: Mon, 7 Sep 2009 09:06:00 -0400

I have to agree with Toby's comments.
 
The charter should be a declarative document, e.g. "this is what we will do and how we will do it". We don't have to review the history of the process that led to the BoF/WG being set up. No one will care about that two years from now.
 
If you look at the first paragraph of the LEDBAT WG charter, it is a simple statement of the goal of the WG: "The LEDBAT WG is chartered to standardize a congestion control mechanism that should saturate the bottleneck, maintain low delay, and yield to standard TCP."
 
If we want something similar, we could say: "This BoF xxx is focused on exposing information needed to share capacity among many users on the Internet. Specifically, the protocols to be considered by this BoF will expose the expected congestion on the rest of the end-to-end path visible in the IP header of each packet." (stealing Bob's words of course)
 
The charter should say briefly (one sentence) how other stakeholders benefit from re-ECN et al, particularly end users and application providers.
 
As for an alternative BoF title, why not use 'Congestion Transparency' or 'Congestion Exposure' for now? We can come up with a 'cute acronym' like contran or conexp later in the process.
 
-- Rich, heading back to my Labor Day holiday, http://en.wikipedia.org/wiki/Labor_day for those in the UK

________________________________

From: [email-masked] [mailto:[email-masked]]
Sent: Mon 9/7/2009 6:36 AM
To: [email-masked]; Woundy, Richard; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [re-ECN] Pls bash: Congestion Exposure (re-ECN) BoF in Hiroshima?

Toby

off
right

---

From: <[email-masked]>
Date: Mon, 7 Sep 2009 15:52:11 +0100

All, 

I've just talked to Bob whose fears about getting into connectivity
problems have unfortunately proved very much founded. He's now hoping to
get back to the list around 19:00 UTC.  

Arnaud  

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Bob Briscoe
Sent: 07 September 2009 11:19
To: Woundy, Richard; COURCOUBETIS, Costas; Steven BLAKE; Marcelo BAGNULO
BRAUN; Moncaster,T,Toby,DER3 R; Agarwal, Anil; Tom Taylor; Ken Carlberg;
Leslie Daigle; BOWMAN Don
Cc: re-ECN unIETF list
Subject: [re-ECN] Fwd: Pls bash: Congestion Exposure (re-ECN) BoF in
Hiroshima?

Folks,

Bob

[snip]

chunk
allocation
nonce
protocol
issues,
congestion)
protocols)

Bob

---

From: <[email-masked]>
Date: Mon, 7 Sep 2009 17:03:25 +0100

OK here is my attempt at something a bit better. Feel free to bash it
hard:

-----------------
Congestion Transparency

It is increasingly clear that the current approach to sharing capacity
between users of networks isn't working. As a consequence ISPs feel they
have to police "heavy users" to free up resources for the bulk of their
customers. This involves making assumptions about the wishes of their
customers which in turn is fuelling tension between ISPs, customers,
content providers and application writers.

Other work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is looking at new
approaches to controlling bulk data transfer rates. But these new
approaches will only work with the cooperation of the operators. What is
lacking is a mechanism to build trust between operators and end-users
and between different tiers of operators. In short the Internet lacks a
system for accountability.

This Congestion Transparency BoF focuses on exposing the congestion
information that end systems use to determine their transmission. By
sharing this information in an open manner among the many users on the
Internet it can provide the foundations for trust and accountability
throughout the network. Specifically, the protocols to be considered by
this BoF will expose the expected rest-of-path congestion in the IP
header of each packet such that nodes downstream of the source can see
the impact that will be caused by any packet they forward.

Currently a protocol called re-ECN (re-inserted explicit congestion
notification) has been proposed to do this. Re-ECN is the strongest
candidate for adoption, but of course the proposed working group is
expected to redesign and thoroughly test alongside any alternative if
one surfaces. Whatever protocol is adopted, it should be possible for a
network operator to count the volume of congestion that is attributable
to an aggregate of traffic, as easily as it can count the volume of
bytes today. This can be done for each user, or for whole attached
networks. The defined protocol should work with minimal changes to the
existing network, in particular it should work with unmodified routers,
but in the long run it is envisaged these small changes will make a big
impact to how the limited resource of the network are shared.

This is not about changing the existing approach to congestion control,
where congestion is detected and responded to by transports on endpoints
(e.g. congestion control in TCP, RTP/RTCP). The only difference is that
network operators will be able, if they choose, to monitor and control
the one factor that causes grief to their other customers: the volume of
congestion caused. In turn they can be monitored and controlled by their
providers. The proposed working group will NOT mandate how exposed
congestion should be used. It will confine itself to the focused task of
defining the protocol for exposing congestion. However, it will strongly
encourage experiments into how this information can be used and will
ultimately produce guidance on the dos and don'ts of using this
information. 

The proposed work items for this group are:
1)	An informational document giving the motivations for congestion
transparency and specifying the requirements for any protocol
2)	An experimental protocol for congestion transparency. This will
probably be based upon the proposed re-ECN protocol as significant
research work has already been done to understand and test this
protocol.
3)	One or more informational documents reporting the results of
experiments on applications of the congestion transparency protocol.
Specifically these experiments are likely to look at means to force
honest disclosure of congestion information and the impact of rationing
the congestion volume a given user or network can cause.

The initial goal is that all the output of the working group will be
informational or experimental, but once the experiments have been
completed then they will be used to determine if this approach should be
standardised within the IETF.
------------

I purposely omitted anything about congestion != impairment although I
did write the following paragraph. It just didn't seem to fit in with
the rest though...

-----
There is a common misconception that congestion is a bad thing.
Congestion != impairment... congestion is a necessary side-effect of
using a shared resource efficiently. This is implicitly recognised in
TCPs classic congestion avoidance strategy which is constantly probing
the network to find the current congestion level, with the aim of
running at a sufficient rate to efficiently utilise the available
bandwidth. With that in mind we simply seek to make end hosts openly
declare this information to the rest of the network so that everyone has
full visibility of all the information that matters.
-----

I was also wondering about making explicit mention of the causes of
this, but chose not to as it is likely to be contentious. For the same
reason I didn't explicitly mention Net Neutrality

Enjoy the rest of your Labor Day if you're in the States... 

Toby

> From: Woundy, Richard [mailto:[email-masked]]
> Sent: 07 September 2009 14:06
> To: Moncaster,T,Toby,DER3 R; Briscoe,RJ,Bob,XVR9 BRISCORJ R;
[email-masked];
> The charter should be a declarative document, e.g. "this is what we
> will do and how we will do it". We don't have to review the history of
> the process that led to the BoF/WG being set up. No one will care
about
> that two years from now.
> 
> If you look at the first paragraph of the LEDBAT WG charter, it is a
> simple statement of the goal of the WG: "The LEDBAT WG is chartered to
> standardize a congestion control mechanism that should saturate the
> bottleneck, maintain low delay, and yield to standard TCP."
> 
> If we want something similar, we could say: "This BoF xxx is focused
on
> exposing information needed to share capacity among many users on the
> Internet. Specifically, the protocols to be considered by this BoF
will
> expose the expected congestion on the rest of the end-to-end path
> visible in the IP header of each packet." (stealing Bob's words of
> course)
> 
> The charter should say briefly (one sentence) how other stakeholders
> benefit from re-ECN et al, particularly end users and application
> providers.
> 
> As for an alternative BoF title, why not use 'Congestion Transparency'
> or 'Congestion Exposure' for now? We can come up with a 'cute acronym'
> like contran or conexp later in the process.
> 
> -- Rich, heading back to my Labor Day holiday,
> http://en.wikipedia.org/wiki/Labor_day for those in the UK
> 
> To: [email-masked]; Woundy, Richard; [email-masked];
> [email-masked]; [email-masked];
[email-masked];
ECN

---

From: John Leslie <[email-masked]>
Date: Mon, 7 Sep 2009 12:41:07 -0400

> OK here is my attempt at something a bit better. Feel free to bash it

   This doesn't feel like the correct title: we don't want congestion
to be invisible, which is the direction this seems to aim. I'm happier
with "Congestion Exposure".

> content providers and application writers.

   Basically good. I wonder if there's a way to say something more
moderate than "fuelling tension". I'm partial to the term "tussle"...

> approaches will only work with the cooperation of the operators.

   This is not quite true: what we mean is that they will fail if the
operators continue to punish congestion-free transport.

> What is lacking is a mechanism to build trust between operators and
> end-users and between different tiers of operators. In short the
> Internet lacks a system for accountability.

   Good.

> sharing this information in an open manner among the many users on the
  ^^^^^^^
   Not quite right... I think we mean, "Openly propagating congestion
information can provide..."

> the impact that will be caused by any packet they forward.

   Perhaps clearer to say "expose both actual congestion and expected
rest-of-path congestion"...

> one surfaces.

   Good, IMHO; but I'd end the paragraph there.

> operator to count the volume of congestion that is attributable

   I'd go for something more like, "The WG will publish a protocol which
enables network operators to count the volume of congestion for any
chosen aggregate of traffic, whether per customer, per peer, or any
other aggregate."

   And add, "The protocol should work with the substantial majority of
existing routers needing no modification whatsoever."

> ultimately produce guidance on the dos and don'ts of using this

   Long-winded, but I'll let someone else edit it for length.

>    research work has already been done to understand and test this

   I'd say "likely" rather than "probably"; otherwise good.

>    experiments on applications of the congestion transparency

   Good...

>    Specifically these experiments are likely to look at means to
>    force honest disclosure of congestion information

   I'm not thrilled with "means to force"... It implies ongoing war.
I'd rather talk about "means to recognize misleading usage of
congestion-exposure bits", and that as part of Security Considerations.

>    and the impact of rationing the congestion volume a given user or
>    network can cause.

   I don't quite get the point here.

> completed then they will be used to determine if this approach should
> be standardised within the IETF.

   Looks basically good! It might help to make reference to the discussion
that has proceeded on this list...

--
John Leslie <[email-masked]>

---

From: <[email-masked]>
Date: Mon, 7 Sep 2009 18:29:33 +0100

Inline...

> From: John Leslie [mailto:[email-masked]]
> Sent: 07 September 2009 17:41
> To: Moncaster,T,Toby,DER3 R
> Cc: [email-masked]; Briscoe,RJ,Bob,XVR9 BRISCORJ R;
[email-masked];
> [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [re-ECN] Pls bash: Congestion Exposure (re-ECN) BoF in
> Hiroshima?
> 

I guess I agree - I accept transparency is open to misinterpretation by
some people but it is technically a correct description - we are making
the level of congestion visible - in other words we are being
transparent about it and not seeking to hide it...

> moderate than "fuelling tension". I'm partial to the term "tussle"...

Yep, would be very happy with tussle, but we need to be clear that this
is not a healthy tussle (like Clark's Design of Tussle) - this is a
tussle that cannot be won

> information can provide..."

Yes

> rest-of-path congestion"...

agreed

>    Good, IMHO; but I'd end the paragraph there.

I wasn't quite sure where to break the paragraphs - I kept changing my
mind!

> chosen aggregate of traffic, whether per customer, per peer, or any

Sounds OK to me

> existing routers needing no modification whatsoever."

Yes

> > the one factor that causes grief to their other customers: the
volume
> of
> > congestion should be used. It will confine itself to the focused
task
> of

Agree it needs to be reduced. May get round to it after my supper...

>    I'd say "likely" rather than "probably"; otherwise good.

OK

> congestion-exposure bits", and that as part of Security
Considerations.

Indeed, but I was unsure how to put this - what the actual aim might be
be is to reduce the traffic to the equivalence of what it should be for
the declared congestion - in other words remove any gain from gaming the
system - so perhaps something like "means to prevent users gaining from
cheating or gaming the system"?

>    I don't quite get the point here.

OK, I was driving at what happens if you ration congestion on a per-user
basis. We have results that suggest this leads to heavy users having to
back of meaning lighter users that share a bottleneck will benefit from
being able to go faster (and that was with unmodified TCP CC)

> that has proceeded on this list...

Yes, that is the missing bit- something that gives a feel for the size
of this community (which seems to be getting quite large)

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 07 Sep 2009 23:29:56 +0100

Guys,

OK, I've just got to your comments saying it 
needs to be a lot more punchy and shorter - will 
have a go in the minimal time left.

I've just realised I've got 90mins, not 30 (I was accounting for summer time).

I'll have big carve up with the scissors and add lashings of snake-oil :)

Bob

At 12:11 07/09/2009, João Taveira Araújo wrote:
>Agree with most of Toby's comments, and would 
>add that the last 3 paragraphs seem too long to basically convey:
>
>- LEDBAT shows that there is a need for more 
>flexible CC paradigms than TCP-friendliness
>- LEDBAT itself is only a solution to a subset 
>of problems, and we need an overarching framework to:
>   - expose congestion so operators are no 
> longer prone to information assymetry.
>   - allow different CC algorithms to evolve 
> independently within less restrictive design space than TCP-friendliness.
>
>Joao
>

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 08 Sep 2009 01:35:52 +0100

Toby, John,

I picked up some of the ideas in this in time. But I didn't get to 
the final parts of the thread until now (after I've posted for the deadline).

inline...

At 18:29 07/09/2009, [email-masked] wrote:
> > > between users of networks isn't working.

I kept my opening stating "The Internet is all about sharing 
capacity...". I always think it's worth spelling out what makes the 
Internet what it is - people get complacent and forget the basics are 
important.

Also, rather than making accusations that "it isn't working", I 
preferred to say we're " realising we really don't understand." This 
is lass accusatorial.

>As a consequence ISPs feel
> > they

I liked this opening. I put a much briefer sentence alluding to this 
at the start, then returned to it a bit later.

>tussle that cannot be won

I called it an arms race.

> > > Internet lacks a system for accountability.

Bu;;er - I missed this para of your mail - it's good. Hold it for 
another time, Toby!

> > information can provide..."

I wasn't sure about this. I prefer to avoid sentences like these that 
muddy who is the subject of the verb. We should be clear that this is 
hosts exposing congestion to networks. Truth in advertising.

> > rest-of-path congestion"...

Again, if I'd have noticed this, I'd have included it - sorry.

I did manage to make the sentence I had much clearer tho.

> > chosen aggregate of traffic, whether per customer, per peer, or any

Again, I'd have used this, but what's there isn't so bad anyway.

> > existing routers needing no modification whatsoever."

I say "without changing forwarding on routers" (true if they already 
implement RFC3168, which they should!)

This nicely excludes the additions an ISP would add at trust 
boundaries for policing etc.

>Agree it needs to be reduced. May get round to it after my supper...

I made it much shorter.

> > >    research work has already been done to understand and test this

I avoided saying re-ECN again, as the main point had been made earlier.

I added that we would define the protocol for v4, v6 & TCP

Given we've already defined re-ECN for all these, I didn't think that 
was committing us overly. (We haven't implemented for v6 tho - I 
thought that would be a nice exercise for someone else).

>cheating or gaming the system"?

I don't think it harms not to mention it's ungamable. If any passing 
traveller turns up at the mic and says "can't they lie," it's a 
useful excuse to introduce this whole area.

>of this community (which seems to be getting quite large)

I added a community size estimate in the covering note to the ADs, 
rather than in the BoF proposal itself (where it's not so appropriate).

Sorry again, for not getting to this thread before the deadline. But 
I think some of it got in.

Bob

---

From: <[email-masked]>
Date: Tue, 8 Sep 2009 09:52:30 +0100

Having got a BoF proposal in (albeit a rather longer one than I
expected) we can now turn our attention to the stuff that is needed to
make the BoF a success:

1) Decide what questions we will ask at the BoF in order to get clear
hums that result in a WG being forms.
2) Define EXACTLY the problem we are addressing (by writing a problem
statement document). And keep this concise and focussed (e.g. only
concentrate on a single aspect).
3) Write a BRIEF email summarising the key points from the BoF proposal
which can be used to publicise the BoF on a bunch of other lists.
4) Find a couple of experienced people to chair this - ideally both
having chaired successful BoFs and at least 1 having some experience in
this space.
5) Work out an agenda - I envisage at most 3 presentations - the
problem, re-ECN as a potential solution (but without ANY protocol
detail) and a summary of what the WG would do. Then work out who
presents these.

I'm sure there are other things I'm missing. Now we've got things moving
let's keep the momentum going and hopefully we can have a successful BoF
in November.

Toby

> Sent: 08 September 2009 01:36
in
> > > > OK here is my attempt at something a bit better. Feel free to
> bash
> >it
> > > > hard:
> > > >
> > > > -----------------
> > > > Congestion Transparency
> > >
> > > > have to police "heavy users" to free up resources for the bulk
of
> > > their
> > > > Other work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is
looking
> at
> > > new
> > > > information that end systems use to determine their
transmission.
> By
> > > > sharing this information in an open manner among the many users
> on
> > > the
> > >   ^^^^^^^
> > > > throughout the network. Specifically, the protocols to be
> considered
> > > by
> > > > header of each packet such that nodes downstream of the source
> can
> > > see
> > > > expected to redesign and thoroughly test alongside any
> alternative
> >if
> > > > one surfaces.
> > >
> > > > networks. The defined protocol should work with minimal changes
> to
> > > the
> > > > but in the long run it is envisaged these small changes will
make
> a
> > > big
> > >    And add, "The protocol should work with the substantial
majority
> > > > (e.g. congestion control in TCP, RTP/RTCP). The only difference
> is
> > > that
> > > > congestion caused. In turn they can be monitored and controlled
> by
> > > their
> > > >    and the impact of rationing the congestion volume a given
user
> or
> > > > The initial goal is that all the output of the working group
will
> be

---

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 7 Sep 2009 20:21:54 +0200

Hi,

I really like this version. I think this approach doesn't focus too early on 
re-ECN and gives a more general motivation to the problem. 
I think its good to bring accountability into the game even though it probably 
would go beyond the scope of this draft to explain what this actually means 
and how it helps for the whole situation...

One (useful) comment though: I would go for Congestion Exposure as the name, 
too. Because with re-ECN you can't really reveal the current congestion 
situation. It's more or less an estimation of the expected congestion given 
by the information collected in the last RTT. So this feels to me it is more  
exposing to have congestion at all (in a certain range) than actually showing 
up the exact level of current congestion.

I hope this helps a little,
Mirja

On Monday 07 September 2009 18:03:25 [email-masked] wrote:
> > Immediate top-level comment - drop the re-ECN from the title. This is
>
> a
>
> > doing that but there may be others so we shouldn't put it in the
>
> title.
>
> > I really fear the overall order of things is wrong as well. The bulk
>
> of
>
> > the first 3 paragraphs is just about IETF processes and the IRTF...
>
> The
>
> > system on which to build network accountability so ISPs have started
>
> to
>
> > bodge their own, with dire consequences for the future of the
>
> network).
>
> > I think we need to re-phrase quite a bit of the detailed stuff as
>
> well,
>
> > > BAGNULO BRAUN; Moncaster,T,Toby,DER3 R; Agarwal, Anil; Tom Taylor;
> >
> > Ken
> >
> > > Attached is my attempt so far. I started again - I'm happy with it
>
> so
>

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

web: www.ikr.uni-stuttgart.de
email: [email-masked]
tel: +49(0)711/685-67973
-------------------------------------------------------------------

---

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 7 Sep 2009 21:33:14 +0200

Hi again,

one more remark: Maybe you want to point out more explicit that it doesn't 
make sense to count the volume of bytes as an user can blow as much traffic 
as he wants into the internet as long as he doesn't bother other users.

Mirja

---

From: ken carlberg <[email-masked]>
Date: Tue, 8 Sep 2009 06:12:25 -0400

> concentrate on a single aspect).

one other thing that is related and helpful in this initial process is  
to bring up a discussion of what is and is not in scope for the BoF.

-ken

---

From: <[email-masked]>
Date: Tue, 8 Sep 2009 11:19:05 +0100

That's a much clearer way of phrasing what I was trying to say :-)

I am working on a draft problem statement and should have an
introduction available for people to comment on in a few hours...

Toby

> From: ken carlberg [mailto:[email-masked]]
> Sent: 08 September 2009 11:12
> Cc: Briscoe,RJ,Bob,XVR9 BRISCORJ R; [email-masked]

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 08 Sep 2009 11:39:54 +0100

Toby,

Good list.
inline...

Bob

---

From: <[email-masked]>
Date: Tue, 8 Sep 2009 16:55:33 +0100

OK, as promised here is the opening page of a problem statement about
congestion exposure. Before going any further with this I would be glad
to get feedback from people on the list, especially those that have
experience in this area. I am not really happy with the last paragraph
and am not sure if we even want to go there. I am contemplating instead
referencing a new version of draft-briscoe-tsvwg-relax-fairness-01.txt
(expired, but it should be available somewhere, bobbriscoe.net if
nowhere else):

A problem statement motivating the need for congestion exposure

The Internet has been a phenomenal success. But for how much longer can
that success last? As access networks get faster and faster, the nature
of the network is changing. New applications appear every year, each in
turn making greater and greater demands on the infrastructure. It is
increasingly clear that the current approach to sharing capacity between
users of networks isn't working. As a consequence ISPs feel they have to
police "heavy users" to free up resources for the bulk of their
customers. This involves making assumptions about the wishes of their
customers which in turn is eroding trust between ISPs, customers,
content providers and application writers.

Their motives for behaving like this are perfectly understandable - they
want the majority of their customers to receive a good service. The
problem is that in trying to control these so-called heavy users, ISPs
are seeking to control the wrong thing. It isn't the volume of data a
user sends that causes problems; it is trying to send too much data when
everyone else is that is the problem - it is a gross simplification to
assume that a user sending lots of data must be causing lots of
congestion. By "punishing" traffic regardless of the impact it is
having, these ISPs are second-guessing their customers priorities and
not allowing them the freedom to prioritise their own traffic.

Current work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is looking at
new approaches to controlling bulk data transfer rates. But these new
approaches will only work if the operators stop limiting traffic purely
based on application type. What is lacking is a mechanism to build trust
between operators and end-users and between different tiers of
operators. In short the Internet lacks a system for accountability.

So what is the solution? Well the first stage is to be more open about
the information end-hosts are using to determine their sending rate -
congestion along the path. Currently this information is visible to the
end-points but is concealed from the rest of the network. It is our
thesis that this information should be made visible at the IP layer -
the waist of the hourglass. Specifically, the protocols to be considered
by this BoF will expose the actual congestion and expected rest-of-path
congestion in the IP header of each packet such that nodes downstream of
the source can see the impact that will be caused by any packet they
forward. 

Once the information is there, it is then possible to use it to measure
the true impact of this traffic on the network. Rather than counting
volume, a node in the network now counts congestion volume - the
remaining congestion along the path multiplied by the size of the
packet. This reflects the excess data that packet is trying to push
through the network and make this a measure of the marginal cost of
forwarding that packet.

> Sent: 08 September 2009 11:19
> To: [email-masked]
> Cc: [email-masked]

---

From: João Taveira Araújo <[email-masked]>
Date: Tue, 08 Sep 2009 18:01:15 +0100

Feedback inline.

> content providers and application writers.

My issue with the first half of the paragraph is it could hold true at 
any point
in time over the past 20 years at the very least. Before going into how ISPs
react I think it would be more useful to directly state why throwing 
bandwidth
at the problem is no longer an effective way of dealing with capacity 
sharing
*at this point in time*, and why it will only get worse.

> not allowing them the freedom to prioritise their own traffic.

Then this paragraph can give an overview of how ISPs retaliate, and 
explain how
each fails to address the problem scalably/elegantly/efficiently. So I 
think the last
two sentences of the first paragraph actually belong somewhere here.

> operators. In short the Internet lacks a system for accountability.

This bridges nicely into proposed solution. Might want to add that this 
simply
subdivides traffic into best-effort and lower-than-best effort, so while 
it is
an improvement, it is not flexible enough to span the full range of 
application
requirements.

> the source can see the impact that will be caused by any packet they

Not sure it's perfectly clear how exposing congestion helps here, namely 
that it
corrects information asymmetry. Maybe something along the lines

"This lack of accountability derives from an information asymmetry which
deprives the network from viewing path metrics only available to
end-hosts. It is our thesis .."?

I'm not sure what you mean by saying the protocol will expose actual 
congestion
and expected rest-of-path congestion. AFAIK, only the latter is 
revealed, which
can be taken as a decent approximation of the former.

> through the network and make this a measure of the marginal cost of

Not sure how deep we have to go into actually explaining what the 
network does
with this information, but then again i'm not entirely familiar with 
proposal
statements. Is it not enough to explain that networks can leverage this
information in the tussle with end-hosts, or is that too much of an
architectural vision?

Cheers,
Joao.

---

From: <[email-masked]>
Date: Tue, 08 Sep 2009 14:37:44 -0400

Toby,

I think the basic message here is fine, but phrasing it in a set of
questions rather than declarative statements does not work, IMHO.  Please
consider the included text as an alternative.

> content providers and application writers.

Replace with:

Currently, the Internet shares capacity and avoids congestion collapse
using distributed algorithms executed on its end-hosts.  While the
resulting allocation of capacity is fair in a certain mathematical sense,
it is not necessarily fair in certain economic or philosophical senses. 
Further, end-hosts have means to "game the system" to increase their access
to capacity, and to progress communications even in the presence of
persistent congestion.

> not allowing them the freedom to prioritise their own traffic.

Replace with:

ISPs have a variety of measures available to them to reduce congestion, or
to skew access to capacity amongst their users, ranging from the complex
(per-user fair queueing, volume-based billing) to the crude
(per-application prioritization).  A problem with such mechanisms is that
they control the wrong quantity: it is not the total traffic of a
particular user or application that impedes service to other users, but it
is the congestion-causing excess traffic.

> operators. In short the Internet lacks a system for accountability.

Replace with:

Current work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is looking at new
approaches to controlling bulk data transfer rates.  But ISPs have no means
for enforcing their use, and hence still have incentives to deploy the
tools at their disposal to manage capacity and reduce congestion.  What
ISPs lack is a way to measure the congestion-causing excess traffic of a
user on an end-to-end basis.  Such a measure could be used to build trust
between operators and end-users and between different tiers of operators.
In short the Internet lacks a system for accountability.

> the source can see the impact that will be caused by any packet they

Replace with:

Currently, congestion information is visible only at end-hosts, and is
concealed from the rest of the network.  We propose that this information
should be made visible at the IP layer - the waist of the hourglass.
Specifically, the protocols to be considered by this BoF will expose the
actual congestion and expected rest-of-path congestion in the IP header of
each packet such that nodes downstream of the source can see the impact
that will be caused by any packet they forward. 

> through the network and make this a measure of the marginal cost of

Replace with:

Once congestion information is exposed in this way, it is then possible to
use it to measure the true impact of this traffic on the network. Rather
than counting total traffic volume, a node in the network may now count
congestion volume.

Regards,

// Steve

---

From: <[email-masked]>
Date: Wed, 9 Sep 2009 10:01:52 +0100

Hi Steve, João,

Thanks for the very prompt and extensive comments. I have accepted pretty much all of Steve's new text and have tried to add in those suggestion from João that are still relevant to the new text. Here is the revised version. One thing I wasn't sure about was whether to mention PCN (or more specifically Bob's re-PCN) as being a similar approach that works for inelastic realtime traffic... On balance I think not.

-----------------

One of the strengths of the Internet is its ability to share capacity and avoid congestion collapse by using distributed algorithms executed on its end-hosts.  While the resulting allocation of capacity is fair in a certain mathematical sense, it is not necessarily fair in other economic or philosophical senses. 

Further, end-hosts have means to "game the system" to increase their access to capacity, and to progress communications even in the presence of persistent congestion. As a consequence ISPs feel they have to police "heavy users" to free up resources for the bulk of their customers. This involves making assumptions about the wishes of their customers which in turn is eroding trust between ISPs, customers, content providers and application writers.

ISPs have a variety of measures available to them to reduce congestion, or to skew access to capacity amongst their users, ranging from the complex (per-user fair queuing, volume-based billing) to the crude (per-application prioritization or rate capping).  A problem with such mechanisms is that they control the wrong quantity: it is not the total volume or absolute rate of traffic of a particular user or application that impedes service to other users; rather, it is the excess traffic that causes others to experience congestion that is the real problem.

Current work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is looking at new approaches for controlling bulk data transfer rates.  But ISPs have no means for enforcing their use and indeed may even be hindering their deployment through their crude attempts at traffic control. This means the ISPs still have an incentive to deploy the flawed tools at their disposal to manage capacity and reduce congestion.  What ISPs lack is a way to measure or even see the congestion caused by a user’s excess traffic on an end-to-end basis.  Such a measure could be used to build trust between operators and end-users and between different tiers of operators. In short the Internet lacks a system for accountability.

Currently, congestion information is visible only at end-hosts, and is concealed from the rest of the network.  We propose that this information should be made visible at the IP layer - the waist of the hourglass.

Specifically, the protocols to be considered by this BoF will expose the actual upstream and expected downstream congestion in the IP header of each packet such that nodes downstream of the source can see the impact that will be caused by any packet they forward. This will serve to correct the current issues of information asymmetry [http://www.soton.ac.uk/~ram2/papers/ccm3.pdf]. 

Once congestion information is exposed in this way, it is then possible to use it to measure the true impact of this traffic on the network. Rather than counting total traffic volume, a node in the network may now count congestion volume.

---------------------

I am quite sure it still needs more bashing and I am not at all precious about what I write - my view is that we need to get something in reasonable shape in the next 3 weeks. By the beginning of next week I am intending to put out a preliminary I-D and would love to have some volunteers as co-authors. Before that stage I also need to settle on a rough ToC for the remainder of the document. I imagine it would roughly follow the 3 main sections in the current document - issues with excessive congestion, ISPs attempts to control this, our proposal to do this better. In the latter section I will probably use a strawman protocol where the precise congestion level (e.g. p in the famous Padhye TCP throughput equations) is somehow encoded into the IP header.

Toby

> From: [email-masked] [mailto:[email-masked]]
> Sent: 08 September 2009 19:38
> Cc: [email-masked]; [email-masked]
> operators.

---

From: Steven Blake <[email-masked]>
Date: Wed, 09 Sep 2009 08:08:46 -0400

On Wed, 2009-09-09 at 10:01 +0100, [email-masked] wrote:

> Hi Steve, João,
> 
> Thanks for the very prompt and extensive comments. I have accepted
> pretty much all of Steve's new text and have tried to add in those
> suggestion from João that are still relevant to the new text. Here is
> the revised version. One thing I wasn't sure about was whether to
> mention PCN (or more specifically Bob's re-PCN) as being a similar
> approach that works for inelastic realtime traffic... On balance I
> think not.

[snip revised text]

I like your revised text.

Regards,

// Steve

---

From: ken carlberg <[email-masked]>
Date: Wed, 9 Sep 2009 08:27:23 -0400

Toby,

here are some comments/questions as food for thought...

> One of the strengths of the Internet is its ability to share  
> capacity and avoid congestion collapse by using distributed  
> algorithms executed on its end-hosts.  While the resulting  
> allocation of capacity is fair in a certain mathematical sense, it  
> is not necessarily fair in other economic or philosophical senses.

is "fairness" also determined by temporal qualities?...instantaneous  
condition versus some period of time, or accumulated accounting.

> Further, end-hosts have means to "game the system" to increase their  
> access to capacity, and to progress communications even in the  
> presence of persistent congestion. As a consequence ISPs feel they  
> have to police "heavy users" to free up resources for the bulk of  
> their customers. This involves making assumptions about the wishes  
> of their customers which in turn is eroding trust between ISPs,  
> customers, content providers and application writers.
>
> ISPs have a variety of measures available to them to reduce  
> congestion, or to skew access to capacity amongst their users,  
> ranging from the complex (per-user fair queuing, volume-based  
> billing) to the crude (per-application prioritization or rate  
> capping).  A problem with such mechanisms is that they control the  
> wrong quantity: it is not the total volume or absolute rate of  
> traffic of a particular user or application that impedes service to  
> other users; rather, it is the excess traffic that causes others to  
> experience congestion that is the real problem.

at the risk of being picky, what is considered "excess traffic"?  Is  
it traffic beyond that negotiated or configured per aggregate or  
application, or some standard deviation above previous traffic levels  
of a flow?  As a reviewer, I could easily find myself asking "excess  
in relation to what?"

> Current work at the IETF [LEDBAT, ALTO] and IRTF [ICCRG] is looking  
> at new approaches for controlling bulk data transfer rates.  But  
> ISPs have no means for enforcing their use and indeed may even be  
> hindering their deployment through their crude attempts at traffic  
> control. This means the ISPs still have an incentive to deploy the  
> flawed tools at their disposal to manage capacity and reduce  
> congestion.  What ISPs lack is a way to measure or even see the  
> congestion caused by a user’s excess traffic on an end-to-end  
> basis.  Such a measure could be used to build trust between  
> operators and end-users and between different tiers of operators. In  
> short the Internet lacks a system for accountability

if you are making an argument for "trust" between operators and end- 
users, don't you have to advocate complimentary efforts like  
integrity?  Or is Security (other than recognizing its importance)  
outside the scope of the BoF?

Finally, I'd also shy away from bringing up PCN at this point.  There  
is no dependency on it, and I think its best not to increase the  
probability of confusing reviewers that have just a few cycles to read  
and decide what is being proposed.  But I like what has been written  
so far.

-ken

---

From: Steven Blake <[email-masked]>
Date: Tue, 8 Sep 2009 12:01:23 -0400

FYI - you can stop cc'ing me; I'm on the re-ecn list.

Thanks,

/////////////////////////////////////////////
Steven Blake       [email-masked]
Extreme Networks              +1 919-884-3211

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 10 Sep 2009 05:00:21 +0100

Toby,

>customers, content providers and application writers.

I don't like the implication here that end-hosts game the system. 
It's mostly innocent use of the tools provided (e.g. TCP) that 
developers just assume sort out fair usage.

Bob

---

From: <[email-masked]>
Date: Thu, 10 Sep 2009 10:38:46 +0100

Perhaps what needs to be got across is that some ISPs fear that there is
"gaming of the system" happening. And whilst the user is usually
innocently pursuing their self-interest, the application writers are
definitely trying it on...

However there may well be a way to get the idea across in a less
controversial phrase. I'll keep working on this.

Toby

> Sent: 10 September 2009 05:00

---

From: João Taveira Araújo <[email-masked]>
Date: Thu, 10 Sep 2009 11:42:00 +0100

Hi,

[email-masked] wrote
> Further, end-hosts have means to "game the system" to increase their access to capacity, and to progress communications even in the presence of persistent congestion. As a consequence ISPs feel the…

I'm not entirely sure people unfamiliar with the flow fairness debate 
will understand what "certain mathematical sense" means, or why it's 
suddenly an issue. Suggestion:

"One of the strengths of the Internet is its ability to share capacity 
and avoid congestion collapse by using distributed algorithms executed 
on its end-hosts. The resulting capacity allocation has reflected a 
single definition of fairness however, and has remained inflexible in 
adapting to differing needs as the Internet has scaled to include new 
and often conflicting stakeholders."

I'm fine with the second paragraph, but if you want to avoid the "game 
the system" bit here's my suggestion. Not sure it's clearer though:

"Further, current capacity allocation is prone to self-interest and 
malice, and has resulted in a system where few can obtain more at the 
expense of many. Faced with such vicarious punishment for the bulk of 
their customers, ISPs resort to policing and throttling "heavy users", 
which in turn is eroding trust between ISPs, customers, content 
providers and application writers."

Joao.

---

From: <[email-masked]>
Date: Thu, 10 Sep 2009 12:49:22 +0100

> From: João Taveira Araújo [mailto:[email-masked]]
> Sent: 10 September 2009 11:42
> > One of the strengths of the Internet is its ability to share capacity
> on its end-hosts.  While the resulting allocation of capacity is fair
> in a certain mathematical sense, it is not necessarily fair in other
> economic or philosophical senses.
> >
> access to capacity, and to progress communications even in the presence
> of persistent congestion. As a consequence ISPs feel they have to

How about something like 

"One of the strengths of the Internet is its ability to share capacity and avoid congestion collapse through the use of distributed algorithms such as TCP running on its endhosts. The resulting capacity allocation is fair in one narrow sense, but is proving poor at adapting to the differing needs of conflicting stakeholders in the ever-more commercialised Internet ["design for tussle"]. "

> providers and application writers."

Still trying to work out how to best to phrase this... getting distracted by lots of other things 

Toby