Subject: [Iccrg] Update to draft-irtf-iccrg-welzl-congestion-control-open-research
From: [email-masked] (Michael Welzl)
Date: Thu Apr 24 18:27:36 2008

> dear welzl,

Hi Mascolo   :-))

> referring to the section '3.2 Challenge 2: Corruption Loss' of the draft,
i
> think that an important point is missing here, which is the basic idea
> introduced at Mobiocm 2001 in the TCP Westwood paper.
>
> The key concept of westwood tcp is the following:

That's all fine, but of course there are a million ideas for improving
TCP in wireless links that we didn't put there - what we don't want
to miss are the open research issues, not all their possible solutions...

Cheers,
Michael

From: [email-masked] (Saverio Mascolo)
Date: Thu Apr 24 18:28:31 2008
Subject: [Iccrg] Update to  draft-irtf-iccrg-welzl-congestion-control-open-research
Message-ID: <009c01c8a631$40f3b1a0$723bccc1@HPSM>

dear welzl,

referring to the section '3.2 Challenge 2: Corruption Loss' of the draft, i think that an important point is missing here, which is the basic idea introduced at Mobiocm 2001 in the TCP Westwood paper. 

The key concept of westwood tcp is the following:

- when a packet loss is detected through 3 dupack (or timeout) set the congestion window equal to 
the measured bandwidth at time of congestion. this measurement is obtained by counting and filtering the ack rate. this setting  provides 
a significant goodput improvement in lossy channels, i.e. when packet are lost not due to congestion.

The reason why this setting works very well in lossy channel is that, since losses are  not due to congestion, 
the "blind" by half window reduction 
of New Reno is avoided, i.e. the window is not over-shrinked.

We have run thousands  of tests over real networks using New Reno,  Cubic and Westwood and we have found that all 
basically provide similar performance except in the case of lossy links where tcp westwood provides a neat improvment due to control window shrinking 
based on measured available bandwidth. 

On the other hand it is well-known that using croos-layer signalling is not a working solution due to inter operability, indeed it violates the fundamental design principle of layering.

best,
saverio 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080424/18ae28d9/attachment.html
>From [email-masked]  Thu Apr 24 19:34:26 2008
From: [email-masked] (Saverio Mascolo)
Date: Thu Apr 24 19:23:13 2008
Subject: [Iccrg] (no subject)
Message-ID: <00b601c8a639$d382f1e0$723bccc1@HPSM>

On Thu, Apr 24, 2008 at 7:40 PM, Michael Welzl <[email-masked]> wrote:

  > dear welzl,

  Hi Mascolo   :-))

  > referring to the section '3.2 Challenge 2: Corruption Loss' of the draft,
  i
  > think that an important point is missing here, which is the basic idea
  > introduced at Mobiocm 2001 in the TCP Westwood paper.
  >
  > The key concept of westwood tcp is the following:

  That's all fine, but of course there are a million ideas for improving
  TCP in wireless links that we didn't put there - what we don't want
  to miss are the open research issues, not all their possible solutions...

dear Michael ,

i do not think it is one of million ideas, since it is   unique and  relevant under the assumption of lossy link. 
another relevant piece of work that you may want  cite  to improve  this draft is the icnp08 paper:
David A. Eckhardt, Peter Steenkiste: 
Improving Wireless LAN Performance via Adaptive Local Error Control. 327-338

which very clearly explains that you have to design a good wireless link so that losses not due to congestion should not be relevant (indeed this is the case in current 3G  systems obtained via ARQ). this is old stuff today, already documented in RFC 3819 - Advice for Internet Subnetwork Designers, by P. Karn.

best
saverio

  Cheers,
  Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080424/1122432d/attachment.html
>From [email-masked]  Fri Apr 25 09:33:01 2008
From: [email-masked] (Michael Welzl)
Date: Fri Apr 25 09:21:19 2008
Subject: [Iccrg] Re:
In-Reply-To: <00b601c8a639$d382f1e0$723bccc1@HPSM>
References: <00b601c8a639$d382f1e0$723bccc1@HPSM>
Message-ID: <[email-masked]>

Hi,

> i do not think it is one of million ideas, since it is   unique and
> relevant under the assumption of lossy link. 

Fine - don't get me wrong, I have no problem with including it.
After all we do mention a number of solutions, and TCP Westwood
is quite well known anyway, I agree it deserves being mentioned.

(hm - I wonder if this kind of thinking turns the world of
scientific citation into a scale-free graph? I guess so  :)   )

> another relevant piece of work that you may want  cite  to improve
> this draft is the icnp08 paper:
> David A. Eckhardt, Peter Steenkiste: 
> Improving Wireless LAN Performance via Adaptive Local Error Control.
> 327-338

Thanks!

> which very clearly explains that you have to design a good wireless
> link so that losses not due to congestion should not be relevant
> (indeed this is the case in current 3G  systems obtained via ARQ).
> this is old stuff today, already documented in RFC 3819 - Advice for
> Internet Subnetwork Designers, by P. Karn.

In fact I'm surprised now that I really forgot to include RFC 3891 -
I really planned to refer to this one in this section, just forgot
about that. Thanks a lot!

Cheers,
Michael

From: [email-masked] (Michael Welzl)
Date: Thu Apr 24 16:21:47 2008
Subject: [Iccrg] Update to draft-irtf-iccrg-welzl-congestion-control-open-research
Message-ID: <[email-masked]>

Dear all,

(Apparently upon telling you about the update of the other
draft), Wes noticed that we forgot to tell you that we also
recently updated the open issues draft. Thanks Wes!

As usual, the main editorial work was done by Dimitri, so
it's quite undeserved that the document is called
"draft-...-welzl-something".

The update is available from:
http://www.welzl.at/research/publications/draft-irtf-iccrg-welzl-congestion-control-open-research-01.txt

Please comment! It would be great if we could let this
document converge soon, too - there was hardly any feedback
in Manchester, which I take as an indication that nobody
has a major problem with it. If I'm wrong, this would be
a good time to tell us!

Cheers,
Michael

From: [email-masked] (Eddy, Wesley M. (GRC-RCN0)[VZ])
Date: Thu Apr 24 14:22:41 2008
Subject: [Iccrg] update to draft-irtf-iccrg-cc-rfcs
Message-ID: <[email-masked]>

An update to the congestion control RFC survey is at:
http://www.ietf.org/internet-drafts/draft-irtf-iccrg-cc-rfcs-04.txt

The only changes from the last version are the addition of
entries for RFC 5033 and RFC 5166, that were recently published.

In Philadelphia, there were no objections to having a "last-call"
on this, so unless there are major issues raised, we would like to
send this up to IRSG review after May 9 (in 2 weeks).  We would
appreciate any further comments on the draft before then.

From: [email-masked] (Lars Eggert)
Date: Thu Apr 17 22:10:25 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Hi,

On 2008-4-16, at 13:19, ext Eddy, Wesley M. (GRC-RCN0)[VZ] wrote:
>> -----Original Message-----
>> From: [email-masked]
>> [mailto:[email-masked]] On Behalf Of Lars Eggert
>> Sent: Wednesday, April 16, 2008 1:04 PM
>>
>> I wouldn't go quite so far and call the current status quo of TCP-
>> friendliness dogmatic; it's mostly what VJ congestion control left us
>> with, and it's sort of a better-than-nothing style of fairness. I do
>> encourage folks to revisit that design choice and discuss
>> whether this
>> type of fairness is what the current and especially future Internet
>> should strive to adhere to, or whether there are alternatives or
>> generalizations that offer more flexibility or "stronger" notions of
>> fairness.
>
> I will just point out that fairness is not a goal of VJ CC; avoiding
> packet loss is.

I didn't say it was a goal, I said some degree of fairness is an  
effect of VJ CC. (If end systems don't cheat.)

Sure, nothing prevents all end systems form all of a sudden turning  
greedy, but the current state we're in - where most end systems do use  
VJ CC for the majority of their traffic - is not completely useless.  
It may not be sufficient going into the future (which is where Matt  
and Bob are coming from, or maybe they're even saying it's not  
sufficient at present), but the state we're currently in is not  
complete anarchy.

> The inputs to VJ CC are "some bytes were not lost" (an ACK) and "some
> bytes were probably lost" (an RTO or 3-dupack).  They are not "someone
> is sending faster than me" or "I'm sending faster than someone else",
> or any other metric or inference useful for acheiving any form of
> fairness, because that is not its goal.

Lars

From: [email-masked] (Lars Eggert)
Date: Thu Apr 17 22:13:03 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Hi,

On 2008-4-16, at 12:58, ext S. Keshav wrote:
> On Apr 16, 2008, at 1:03 PM, Lars Eggert wrote:
>> I wouldn't go quite so far and call the current status quo of TCP- 
>> friendliness dogmatic; it's mostly what VJ congestion control left  
>> us with, and it's sort of a better-than-nothing style of fairness.  
>> I do encourage folks to revisit that design choice and discuss  
>> whether this type of fairness is what the current and especially  
>> future Internet should strive to adhere to, or whether there are  
>> alternatives or generalizations that offer more flexibility or  
>> "stronger" notions of fairness.
>>
> 	I think two points are obvious:
> 		1. VJ congestion control, or any end-point congestion control for  
> that matter, cannot by itself provide fairness of any sort. Fairness  
> (whatever the definition) must come from inside the network, simply  
> because the edges are uncontrolled. We can argue all we want what  
> fairness means, but this fact remains. So the status quo is broken,  
> a fact that has been known for 20 years.

I completely agree, *if* the end systems cheat, and didn't mean to  
imply that a purely edge based control system can deal with this.

But in many cases the edges don't cheat and VJ congestion control does  
leave us with a situation where flows end up sharing a path. They may  
not end up sharing it according to a number of stringent definitions  
of fairness, but typically, all flows make progress. That's better  
than nothing.

(I'm open to the argument that it gets easier and/or more attractive  
for the edges to cheat than it used to be, and that that's why we need  
a network-based mechanism..)

> 	2. Even if we put into place mechanisms for 'fairness', it is not  
> clear that anyone cares about fairness in the first place. Two  
> endpoint owners can compare notes about what service they receive,  
> but the reason why they get different service could be  
> misconfiguration, different RTTs, bugs in their software, line noise  
> (or RF noise), cross traffic or just the phase of the moon.  As far  
> as I can observe since 1988, the need for fairness is always  
> assumed, rarely verified, and never necessary.

I think your definition of fairness is a lot more stringent than mine.

> 	My conclusion: fairness can't be achieved by endpoint control and  
> in any case is a non-goal :-) Time to move on to something else (I  
> stopped working in this area in 1991 for exactly this reason).

Again, I didn't mean to say that we can rely on end-system based CC,  
and Matt and Bob have made arguments similar to yours (i.e., that we  
need something in the network). My point was that what we currently  
have (due to VJ) isn't completely useless (yet.)

Lars

From: [email-masked] (Dirceu Cavendish)
Date: Thu Apr 17 18:33:22 2008
Subject: [Iccrg] Heresy recapped
Message-ID: <[email-masked]>

Eddy,

I will not dwell on VJ "intention" regarding fairness issues. However, from a control theoretical perspecive, it is a fact that a ON-OFF type of rate regulation embedded in the AIMD feature of TCP CA causes bottleneck bandwidth resharing, converging* to similar transmission rates.

*I should be careful about the term convergence, as theoreticcally an ON-OFF controller never converges, but in steady state oscillates around a "fair" point. The oscillation band can be related with teh session rtt...

Dirceu

----- Original Message ----
From: "Eddy, Wesley M. (GRC-RCN0)[VZ]" <[email-masked]>
To: Michael Welzl <[email-masked]>; iccrg <[email-masked]>
Sent: Thursday, April 17, 2008 6:11:10 AM
Subject: RE: [Iccrg] Heresy recapped

>-----Original Message-----
>From: [email-masked] 
>[mailto:[email-masked]] On Behalf Of Michael Welzl
>Sent: Thursday, April 17, 2008 1:38 AM
>
>>I will just point out that fairness is not a goal of VJ CC; avoiding
>>packet loss is.
>>
>>The inputs to VJ CC are "some bytes were not lost" (an ACK) and "some
>>bytes were probably lost" (an RTO or 3-dupack).  They are not "someone
>>is sending faster than me" or "I'm sending faster than someone else",
>>or any other metric or inference useful for acheiving any form of
>>fairness, because that is not its goal.
>
>Now that ain't true. The choice of AIMD in the Cong. Avoid & Control
>paper is based on the Chiu/Jain paper (this paper is cited in Van's),
>where AIMD was found to be the only control leading to efficiency
>*and fairness* among the four simple linear ones that they looked at.

The message at:
http://ee.lbl.gov/tcp.html#dynamic
only briefly mentions fairness, and actually labels it as something
for the router to decide, not the end hosts.

The from:
http://ee.lbl.gov/papers/congavoid.pdf
"While algorithms at the transport endpoints can insure the network
capacity isn't exceeded, they cannot insure fair sharing of that
capacity. Only in gateways, at the convergence of flows, is there enough
information to control sharing and fair allocation."

Both references describe the selection of AIMD without reference to
fairness, but specifically for stability.

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080417/2f3c7ea2/attachment.html
>From [email-masked]  Thu Apr 17 20:12:01 2008
From: [email-masked] (John Leslie)
Date: Thu Apr 17 20:01:29 2008
Subject: [Iccrg] A "Thought Experiment"
Message-ID: <20080417191201.GI7903@verdi>

   I would like to propose a thought experiment.

   Let us assume an "ideal" internet where a large number of routers,
managed by a largish number of Autonomous Systems are interconnected
pairwise with bidirectional connections such that there is a data path
with a known minimum latency and maximum bandwidth, separate from a
control path of similar latency and infinite bandwidth; and let us
assume each router has infinite memory and infinite processing power.

   (Obviously I don't propose to build such a network.)

   This would, IMHO, be a good basis to start designing an "ideal"
routing architecture. For one example:

1) assume all routers have a UTC clock synchronized to a microsecond.
2) assume all packets are injected with a UTC expiration time.
3) assume routers will drop packets that "cannot" be delivered by then.
4) assume that drop event can be reported back over the control path.

   We now have a much more useful measure of "congestion," in that the
sender can know, quite possibly before the packet would have been
delivered, that there's too much "congestion" for timely delivery.
S/he can then try to live with slower delivery or give up trying to
use this internet for those packets.

   A number of optimizations present themselves for consideration,
for example:

a) higher priority could be assigned to packets which will expire soon.
b) projected latency (not just minimum) could be reported back the path.
c) an ISP could (per contract) switch packets to a more expensive path.

   (We're even getting close to being able to route around congestion
_within_ the network.)

   Does this sound interesting? If so, we might actually think about
building something with "relaxed" values for "infinite"... ;^)

--
John Leslie <[email-masked]>

From: [email-masked] (Lachlan Andrew)
Date: Thu Apr 17 17:12:18 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <003301c8a04d$249994f0$0200a8c0@fun> <[email-masked]>
Message-ID: <[email-masked]>

Greetings Wes,

On 17/04/2008, Eddy, Wesley M. (GRC-RCN0)[VZ] <[email-masked]> wrote:
>
>  The from:
>  http://ee.lbl.gov/papers/congavoid.pdf
>  "While algorithms at the transport endpoints can insure the network
>  capacity isn't exceeded, they cannot insure fair sharing of that
>  capacity. Only in gateways, at the convergence of flows, is there enough
>  information to control sharing and fair allocation."
>
>  Both references describe the selection of AIMD without reference to
>  fairness, but specifically for stability.

I agree (now) that that wasn't the original intention.  However this
reminds me of US discussions of gun laws based on what the writers of
the constitution intended hundreds of years ago, when circumstances
(firepower, data rates) were vastly different.

Although I don't think it matters whether we get "exact fairness", I
do think that
1) congestion control has assumed the task of resource allocation on
*some* links, and
2) whatever allocates resources has a responsibility to ensure no user
is starved.

I agree users will not "compare notes", but a user who is being
starved (e.g., by excessively slow starting) will often be
dissatisfied, and as researchers we should try to find ways to avoid
that.  <shamelessPlug>For example, in
<http://netlab.caltech.edu/lachlan/abstract/dq_infocom99.pdf> we
looked at a scheme aiming to keep users satisfied at the expense of
short-term fairness.</shamelessPlug>

The current notions of fairness may be broken and/or overemphasised,
but I'm not convinced that it is a non-issue.

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (Eddy, Wesley M. (GRC-RCN0)[VZ])
Date: Thu Apr 17 14:00:17 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <003301c8a04d$249994f0$0200a8c0@fun>
References: <[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]> <003301c8a04d$249994f0$0200a8c0@fun>
Message-ID: <[email-masked]>

>-----Original Message-----
>From: [email-masked] 
>[mailto:[email-masked]] On Behalf Of Michael Welzl
>Sent: Thursday, April 17, 2008 1:38 AM
>
>>I will just point out that fairness is not a goal of VJ CC; avoiding
>>packet loss is.
>>
>>The inputs to VJ CC are "some bytes were not lost" (an ACK) and "some
>>bytes were probably lost" (an RTO or 3-dupack).  They are not "someone
>>is sending faster than me" or "I'm sending faster than someone else",
>>or any other metric or inference useful for acheiving any form of
>>fairness, because that is not its goal.
>
>Now that ain't true. The choice of AIMD in the Cong. Avoid & Control
>paper is based on the Chiu/Jain paper (this paper is cited in Van's),
>where AIMD was found to be the only control leading to efficiency
>*and fairness* among the four simple linear ones that they looked at.

The message at:
http://ee.lbl.gov/tcp.html#dynamic
only briefly mentions fairness, and actually labels it as something
for the router to decide, not the end hosts.

The from:
http://ee.lbl.gov/papers/congavoid.pdf
"While algorithms at the transport endpoints can insure the network
capacity isn't exceeded, they cannot insure fair sharing of that
capacity. Only in gateways, at the convergence of flows, is there enough
information to control sharing and fair allocation."

Both references describe the selection of AIMD without reference to
fairness, but specifically for stability.

From: [email-masked] (Mark Allman)
Date: Thu Apr 17 01:25:17 2008
Subject: [Iccrg] SSDT Scope - summary
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

> Personally, I'm not a fan of QuickStart/JumpStart, because it doesn't
> give other flows enough time to back off.  

I think this sentence lumps things together that should not be lumped
together.  QuickStart should not have the problem of interacting with
other flows because each router should be tracking the capacity being
consumed and so should not agree to a QS request that would in fact
cause interference with competing traffic.  (Certainly there are cases
that can cause mistakes---but, these are just that: mistakes.)

The consideration for JumpStart interfering with other traffic is
completely justified because JS is at the complete opposite end of the
spectrum from QS.  While QS tries to be quite careful about checking for
a proper rate, JS crudely simply starts at whatever rate the endpoint
feels like.  Clearly this latter approach is much more likely to
interfere with competing traffic.

allman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080416/59f89856/attachment.bin
>From [email-masked]  Thu Apr 17 06:37:35 2008
From: [email-masked] (Michael Welzl)
Date: Thu Apr 17 06:26:06 2008
Subject: [Iccrg] Heresy recapped
References: <[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]>
	<[email-masked]>
Message-ID: <003301c8a04d$249994f0$0200a8c0@fun>

>I will just point out that fairness is not a goal of VJ CC; avoiding
>packet loss is.
>
>The inputs to VJ CC are "some bytes were not lost" (an ACK) and "some
>bytes were probably lost" (an RTO or 3-dupack).  They are not "someone
>is sending faster than me" or "I'm sending faster than someone else",
>or any other metric or inference useful for acheiving any form of
>fairness, because that is not its goal.

Now that ain't true. The choice of AIMD in the Cong. Avoid & Control
paper is based on the Chiu/Jain paper (this paper is cited in Van's),
where AIMD was found to be the only control leading to efficiency
*and fairness* among the four simple linear ones that they looked at.

Cheers,
Michael

From: [email-masked] (Eddy, Wesley M. (GRC-RCN0)[VZ])
Date: Wed Apr 16 21:09:32 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]><[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

>-----Original Message-----
>From: [email-masked] 
>[mailto:[email-masked]] On Behalf Of Lars Eggert
>Sent: Wednesday, April 16, 2008 1:04 PM
>
>I wouldn't go quite so far and call the current status quo of TCP- 
>friendliness dogmatic; it's mostly what VJ congestion control left us  
>with, and it's sort of a better-than-nothing style of fairness. I do  
>encourage folks to revisit that design choice and discuss 
>whether this  
>type of fairness is what the current and especially future Internet  
>should strive to adhere to, or whether there are alternatives or  
>generalizations that offer more flexibility or "stronger" notions of  
>fairness.

I will just point out that fairness is not a goal of VJ CC; avoiding
packet loss is.

The inputs to VJ CC are "some bytes were not lost" (an ACK) and "some
bytes were probably lost" (an RTO or 3-dupack).  They are not "someone
is sending faster than me" or "I'm sending faster than someone else",
or any other metric or inference useful for acheiving any form of
fairness, because that is not its goal.

From: [email-masked] (S. Keshav)
Date: Wed Apr 16 20:47:33 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

On Apr 16, 2008, at 1:03 PM, Lars Eggert wrote:
>
> I wouldn't go quite so far and call the current status quo of TCP- 
> friendliness dogmatic; it's mostly what VJ congestion control left  
> us with, and it's sort of a better-than-nothing style of fairness. I  
> do encourage folks to revisit that design choice and discuss whether  
> this type of fairness is what the current and especially future  
> Internet should strive to adhere to, or whether there are  
> alternatives or generalizations that offer more flexibility or  
> "stronger" notions of fairness.
>
> Lars

Lars,
	I think two points are obvious:

		1. VJ congestion control, or any end-point congestion control for  
that matter, cannot by itself provide fairness of any sort. Fairness  
(whatever the definition) must come from inside the network, simply  
because the edges are uncontrolled. We can argue all we want what  
fairness means, but this fact remains. So the status quo is broken, a  
fact that has been known for 20 years.

	2. Even if we put into place mechanisms for 'fairness', it is not  
clear that anyone cares about fairness in the first place. Two  
endpoint owners can compare notes about what service they receive, but  
the reason why they get different service could be misconfiguration,  
different RTTs, bugs in their software, line noise (or RF noise),  
cross traffic or just the phase of the moon.  As far as I can observe  
since 1988, the need for fairness is always assumed, rarely verified,  
and never necessary.

	My conclusion: fairness can't be achieved by endpoint control and in  
any case is a non-goal :-) Time to move on to something else (I  
stopped working in this area in 1991 for exactly this reason).

keshav

From: [email-masked] (Lars Eggert)
Date: Wed Apr 16 18:02:21 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Hi,

On 2008-4-15, at 20:23, ext Matt Mathis wrote:
> On Tue, 15 Apr 2008, S. Keshav wrote:
>> Its already a done deal.
>
> Not in TCPM, TSVWG, etc, where dogmatic attachment to TCP-friendly  
> is probably hurting the IETF.  This is where we need to change minds  
> and some deeply entrenched documents.
>
> I think the ADs are probably listening - do they have any commemts?

you know us to well :-)

I wouldn't go quite so far and call the current status quo of TCP- 
friendliness dogmatic; it's mostly what VJ congestion control left us  
with, and it's sort of a better-than-nothing style of fairness. I do  
encourage folks to revisit that design choice and discuss whether this  
type of fairness is what the current and especially future Internet  
should strive to adhere to, or whether there are alternatives or  
generalizations that offer more flexibility or "stronger" notions of  
fairness.

Lars

From: [email-masked] (Dirceu Cavendish)
Date: Wed Apr 16 06:12:08 2008
Subject: [iccrg-slowstart 00007] Re: [Iccrg] SSDT contributions
Message-ID: <[email-masked]>

Michael,

Good charts. Question: how did you pick the specific 82Mbps and 5.1Mbps request values? Why didnt you request 9Mbps on a 10Mbps link?

Dirceu

----- Original Message ----
From: Michael Scharf <[email-masked]>
To: [email-masked]
Cc: iccrg IRTF list <[email-masked]>
Sent: Tuesday, April 15, 2008 12:33:04 PM
Subject: [iccrg-slowstart 00007] Re: [Iccrg] SSDT contributions

Dirceu,

in simple usage scenarios (e. g., HTTP/1.0-like communication) the
impact of Slow-Start is rather obvious, as well as the potential
speedup that faster algorithms could achieve.

The attached slides show some Linux measurement results that compare
the Slow-Start with Quick-Start (best case). There can be a
performance improvement of one second or more, which is typically
"user perceivable". The difference would be even larger if the TCP
stacks used delayed ACKs (Linux disables them sometimes). Of course,
the larger the RTT, the more significant is the effect.

I also have data for more realistic HTTP request patterns, but in this
case the Slow-Start impact is more difficult to quantify and depends a
lot on the application communication pattern.

Michael

On Fri, 11 Apr 2008 at 10:45:20, Dirceu Cavendish wrote:
> Michael,
> 
> I think the "user perceivable" impact of faster SS algorithms should be documented within ICCRG. Do you have data backing that up (for any application of your choice)?
> 
> I am attaching few slides on experimental results in reducing segment losses during SS, for discussion. As presented in Manchester, we reduce losses by both reducing the speed of cwnd increase (in a fractional rather than by 1 way), as well as NOT transmitting the entire cwnd on a back-to-back fashion - introducing "micro-pauses" during transmissions...
> 
> Dirceu

      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080415/0618cbe8/attachment.html
>From [email-masked]  Tue Apr 15 20:33:04 2008
From: [email-masked] (Michael Scharf)
Date: Wed Apr 16 08:06:53 2008
Subject: [iccrg-slowstart 00006] Re: [Iccrg] SSDT contributions
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Dirceu,

in simple usage scenarios (e. g., HTTP/1.0-like communication) the
impact of Slow-Start is rather obvious, as well as the potential
speedup that faster algorithms could achieve.

The attached slides show some Linux measurement results that compare
the Slow-Start with Quick-Start (best case). There can be a
performance improvement of one second or more, which is typically
"user perceivable". The difference would be even larger if the TCP
stacks used delayed ACKs (Linux disables them sometimes). Of course,
the larger the RTT, the more significant is the effect.

I also have data for more realistic HTTP request patterns, but in this
case the Slow-Start impact is more difficult to quantify and depends a
lot on the application communication pattern.

Michael

On Fri, 11 Apr 2008 at 10:45:20, Dirceu Cavendish wrote:
> Michael,
> 
> I think the "user perceivable" impact of faster SS algorithms should be documented within ICCRG. Do you have data backing that up (for any application of your choice)?
> 
> I am attaching few slides on experimental results in reducing segment losses during SS, for discussion. As presented in Manchester, we reduce losses by both reducing the speed of cwnd increase (in a fractional rather than by 1 way), as well as NOT transmitting the entire cwnd on a back-to-back fashion - introducing "micro-pauses" during transmissions...
> 
> Dirceu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ss_vs_qs.pdf
Type: application/pdf
Size: 37740 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080415/86b57083/ss_vs_qs-0001.pdf
>From [email-masked]  Wed Apr 16 08:32:40 2008
From: [email-masked] (Michael Scharf)
Date: Wed Apr 16 08:21:50 2008
Subject: [iccrg-slowstart 00008] Re: [Iccrg] SSDT contributions
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Dirceu,

the Quick-Start spec (RFC 4782, p. 10) allows only rather coarse
grained requests such as 5.12 Mbit/s, 10.24 Mbit/s, ..., 81.92 Mbit/s,
etc.

In the presented experiments, the admission control threshold has been
configured for a link capacity of 10 Mbit/s or 100 Mbit/s,
respectively.  Our Quick-Start admission control reduces or denies
larger requests, and it prevends overbooking. Whatever data rate is
asked for, 5.12 Mbit/s or 81.92 Mbit/s is the maximum request that
will get approved by the admission control.

But, even though the approved rate is smaller than the link capacity
of the setup, the Quick-Start performance is very close to the
theoretical optimum. The reason is that the sender continues to open
its window in Slow-Start after completing the Quick-Start phase. Thus,
it requires at most one additional RTT to send with the full speed.

A "perfect" scheme that exactly knows the optimal initial sending rate
would only slighly improve the application response time (it's very
simple to calculate this theoretical optimum).

Michael

On Tue, 15 Apr 2008 at 22:22:57, Dirceu Cavendish wrote:
> Michael,
> 
> Good charts. Question: how did you pick the specific 82Mbps and 5.1Mbps request values? Why didnt you request 9Mbps on a 10Mbps link?
> 
> Dirceu
> 
> 
> 
> ----- Original Message ----
> From: Michael Scharf <[email-masked]>
> To: [email-masked]
> Cc: iccrg IRTF list <[email-masked]>
> Sent: Tuesday, April 15, 2008 12:33:04 PM
> Subject: [iccrg-slowstart 00007] Re: [Iccrg] SSDT contributions
> 
> Dirceu,
> 
> in simple usage scenarios (e. g., HTTP/1.0-like communication) the
> impact of Slow-Start is rather obvious, as well as the potential
> speedup that faster algorithms could achieve.
> 
> The attached slides show some Linux measurement results that compare
> the Slow-Start with Quick-Start (best case). There can be a
> performance improvement of one second or more, which is typically
> "user perceivable". The difference would be even larger if the TCP
> stacks used delayed ACKs (Linux disables them sometimes). Of course,
> the larger the RTT, the more significant is the effect.
> 
> I also have data for more realistic HTTP request patterns, but in this
> case the Slow-Start impact is more difficult to quantify and depends a
> lot on the application communication pattern.
> 
> Michael
> 
> On Fri, 11 Apr 2008 at 10:45:20, Dirceu Cavendish wrote:
> > Michael,
> > 
> > I think the "user perceivable" impact of faster SS algorithms should be documented within ICCRG. Do you have data backing that up (for any application of your choice)?
> > 
> > I am attaching few slides on experimental results in reducing segment losses during SS, for discussion. As presented in Manchester, we reduce losses by both reducing the speed of cwnd increase (in a fractional rather than by 1 way), as well as NOT transmitting the entire cwnd on a back-to-back fashion - introducing "micro-pauses" during transmissions...
> > 
> > Dirceu
> 
> 
>       ____________________________________________________________________________________
> Be a better friend, newshound, and 
> know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
-- 
Dipl.-Ing. Michael Scharf
Institute of Communication Networks and Computer Engineering  (IKR)
Universit?t Stuttgart, Pfaffenwaldring 47, 70569 Stuttgart, Germany
Phone: +49 711 685-69006 Email: [email-masked]
Fax:   +49 711 685-67983 Web:   www.ikr.uni-stuttgart.de/en/~scharf

From: [email-masked] (Matt Mathis)
Date: Wed Apr 16 04:12:54 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

On Tue, 15 Apr 2008, S. Keshav wrote:

> Matt,
> 	The paradigm holds sway only in the minds of academia. I think (based 
> almost purely on cynicism), that in the real world TCP friendliness never had 
> a chance. There is a long history of TCP accelerators, multi-connection 
> applications, UDP-blasters, packet classifiers-and-delayers, and who knows 
> what else that have never cared about it. So, why do we need to phase it out?

I would tend to agree.  However, isn't this list supposed to represent 
academia?

> Its already a done deal.

Not in TCPM, TSVWG, etc, where dogmatic attachment to TCP-friendly is probably 
hurting the IETF.  This is where we need to change minds and some deeply 
entrenched documents.

I think the ADs are probably listening - do they have any commemts?

Thanks,
--MM--

From: [email-masked] (Lachlan Andrew)
Date: Tue Apr 15 19:16:01 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Keshav,

Perhaps Matt meant that we need to consider how we can get the IETF to
agree to standardise Reno-unfriendly mechanisms, rather than
continuing the current defacto free-for-all that you describe.

We might not need to phase TCP-friendliness out of the Internet, but
we may need to phase it out of the IETF expectations.  (FWIW, I don't
see the IETF as academia, even though there is overlap.)

Cheers,
Lachlan

On 15/04/2008, S. Keshav <[email-masked]> wrote:
> Matt,
>         The paradigm holds sway only in the minds of academia. I think
> (based almost purely on cynicism), that in the real world TCP friendliness
> never had a chance. There is a long history of TCP accelerators,
> multi-connection applications, UDP-blasters, packet
> classifiers-and-delayers, and who knows what else that have never cared
> about it. So, why do we need to phase it out? Its already a done deal.
>
>  regards
>  keshav
>
>  On Apr 14, 2008, at 12:16 AM, Matt Mathis wrote:
>
> > I would like to nudge the iccrg to go on to the next step:  What do we
> need to do to phase out the TCP-friendly paradigm?
> >
>
>
>  _______________________________________________
>  Iccrg mailing list
>  [email-masked]
>  http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg
>

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (S. Keshav)
Date: Tue Apr 15 18:49:34 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Matt,
	The paradigm holds sway only in the minds of academia. I think (based  
almost purely on cynicism), that in the real world TCP friendliness  
never had a chance. There is a long history of TCP accelerators, multi- 
connection applications, UDP-blasters, packet classifiers-and- 
delayers, and who knows what else that have never cared about it. So,  
why do we need to phase it out? Its already a done deal.

regards
keshav

On Apr 14, 2008, at 12:16 AM, Matt Mathis wrote:
> I would like to nudge the iccrg to go on to the next step:  What do  
> we need to do to phase out the TCP-friendly paradigm?

From: [email-masked] (Dirceu Cavendish)
Date: Sat Apr 12 10:33:50 2008
Subject: [iccrg-slowstart 00005] Re: [Iccrg] SSDT contributions
Message-ID: <[email-masked]>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: reducing_loss_slow_start.ppt
Type: application/vnd.ms-powerpoint
Size: 428544 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080411/1999a05a/reducing_loss_slow_start-0001.ppt
>From [email-masked]  Mon Apr 14 05:16:38 2008
From: [email-masked] (Matt Mathis)
Date: Mon Apr 14 05:06:09 2008
Subject: [Iccrg] Heresy recapped
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
	<[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

My "heresy" proved to be rather anticlimactic.  Although there was a spirited 
discussion about the nature of fairness*, there was no real discussion about 
my position about rescinding the TCP-friendly paradigm.  Does this mean there 
is agreement in this group?

I would like to nudge the iccrg to go on to the next step:  What do we need to 
do to phase out the TCP-friendly paradigm?

I had a separate conversation with the original authors of "TCP-Friendly", 
Jamshid Mahadavi and Sally Floyd [1], where we came to realize that there 
still needs to be some litmus test for acceptable protocol behavior.

There is a huge amount of work needed here: we do not even have a formal 
definition of congestion collapse, much less any methods to test if a given 
congestion control algorithm might be subject to collapse.  Is it sufficient 
for all users/protocols to merely optimize their own good put?  There is an 
entire arena of research here that has been waved away by "its TCP-friendly, 
so we don't need to worry about ...."  All past questions potentially need to 
be re-opened.

In the short term, we can always evaluate protocols and algorithms on a 
case-by-case basis.  In doing so we are likely to discover the underlying 
principles.  I guess the very short term question is what, if anything, should 
we say to tcpm about BIC and CTCP?

Note we probably need a warning labels on some protocols: "This protocol (or 
application) will not politely share the network with other users, unless 
network itself enforces appropriate sharing.  In environments without the 
required controls in the network it is likely to cause lockout or other 
extreme unfairness."

*) I specifically did not define fairness because I believe that my comments 
apply independent of the fairness model.  Fairness is very important to ISPs, 
and their conversations with both their customers and their equipment 
suppliers.  Except for algorithms where people want to allocate header bits or 
code points, I don't believe that these conversation need the participation of 
standards organizations because reasonable differences in implementation do 
not have any affect on interoperability.  Furthermore, it surprises me that 
some of the ideas are not held more closely, because this is an ideal arena 
for ISPs to have proprietary solutions that are better than their competitors.

[1] Jamshid Mahdavi, Sally Floyd, "TCP-Friendly Unicast Rate-Based Flow 
Control" http://www.psc.edu/networking/papers/tcp_friendly.html, January, 
1997.

Thanks for the feedback!
--MM--
-------------------------------------------
Matt Mathis     http://staff.psc.edu/mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

From: [email-masked] (Michael Scharf)
Date: Fri Apr 11 09:13:48 2008
Subject: [Iccrg] SSDT contributions
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

The "slowness" of slow-start is quite well-understood and has been
studied various times in the last decate (e. g., Heidemann et. al. in
1997). 

I have the impression that the slow-start delay is also one reason for
the widespread deployment of certain middleboxes (WAN optimizers), for
instance, in enterprise networks. But I am not aware of much published
work on this issue.

The potential benefit of new schemes such as quick-start or jump-start
has recently been demonstrated in a number of publications. Our own
experiments suggest that there can be a user-perceivable improvement
if the RTT is of the order of 200ms. But, of course, it also depends a
lot on the communication pattern of the application. 

So far, I've not studied in detail to which extent more aggressive
schemes increase the risk of buffer overshooting. In case of Linux
end-systems, the default packet buffers are rather large and can thus
accommodate some burstiness.

Michael

On Mon, 07 Apr 2008 at 11:56:34, Dirceu Cavendish wrote:
> I would like to solicit experimental/simulation evidence on the following topics.
> 
> - Transaction latencies taking too long to complete, in a large bandwidth delay product networks.
> - Buffer overshoot, causing multiple segment losses/retransmits, potentially causing RTOs
> - Session returning to SS and starting from scratch due to "application constrained" scenario
> 
> If you are working in any of these topics, and is willing to contribute, please come forward. Past results are good enough, to start a discussion on the threads...
> 
> We (KIT) can start with the second topic, buffer overshoot and multiple RTOs...
> I will prepare some charts with data...
> 
> Rgds to all,
> Dirceu

From: [email-masked] (Dirceu Cavendish)
Date: Tue Apr  8 21:32:47 2008
Subject: [Iccrg] SSDT Scope - summary
Message-ID: <[email-masked]>

Stefan,

Just confirming you are talking about storage space, not link spped, like Lachlan's previous msg.

This is the type of characterization we would like to see documented - and perhaps reflected in a document...

Dirceu

----- Original Message ----
From: Stefan Hirschmann <[email-masked]>
To: [email-masked]
Sent: Tuesday, April 8, 2008 12:55:20 PM
Subject: Re: [Iccrg] SSDT Scope - summary

Lachlan Andrew wrote:
Hi!
> But when we first start, we don't know if we're talking to a 300bps
> mote or a 100GbE supercomputer.  After a single timeout, we will
> generally (???) be using a "similar" network.

In times of QoS this assumption may be very wrong. E.g. You have two 
wires to the internet. One 1MB and one 100MB, but for the 100MB only 
HTTP is allowed all other protocols have to use the 1MB link. If youre 
first connection is an HTTP one, then the 1MB link will be overshot, the 
other way round you'll have to wait for a long time to satisfy the 100MB 
if you increase just linear.

The first mentioned problem, with 1MB link get 100 MB data could be 
solved in one RTT, but in this case 99MB data get dropped. This would be 
an overshot of the factor 100 and much worser than slowstart.

Cheers Stefan

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080408/4bcb4c5b/attachment.html
>From [email-masked]  Tue Apr  8 21:50:29 2008
From: [email-masked] (Lachlan Andrew)
Date: Tue Apr  8 21:40:53 2008
Subject: [Iccrg] SSDT Scope - summary
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

> Lachlan Andrew wrote:
> > But when we first start, we don't know if we're talking to a 300bps
> > mote or a 100GbE supercomputer.  After a single timeout, we will
> > generally (???) be using a "similar" network.
>
> In times of QoS this assumption may be very wrong. E.g. You have two
> wires to the internet. One 1MB and one 100MB, but for the 100MB only
> HTTP is allowed all other protocols have to use the 1MB link. If youre
> first connection is an HTTP one, then the 1MB link will be overshot, the
> other way round you'll have to wait for a long time to satisfy the 100MB
> if you increase just linear.
>
> The first mentioned problem, with 1MB link get 100 MB data could be
> solved in one RTT, but in this case 99MB data get dropped. This would be
> an overshot of the factor 100 and much worser than slowstart.

If only one is HTTP, then they're probably different TCP connections.

Even ignoring that:
1. going from 100Mbit/s to 1Mbit/s, you will overshoot in the first
RTT, when the rate is much less than 100Mbit/s, get a time-out, and
start a normal exponential slow-start.  I'm not suggesting going
straight   from the current rate to the final rate.

2. going from 1Mbit/s to 100Mbit/s, either way slow start will stop
when  ssthresh  is reached, when the source is sending at ~1Mbit/s.

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (Stefan Hirschmann)
Date: Tue Apr  8 20:45:39 2008
Subject: [Iccrg] SSDT Scope - summary
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Lachlan Andrew wrote:
Hi!
 > But when we first start, we don't know if we're talking to a 300bps
 > mote or a 100GbE supercomputer.  After a single timeout, we will
 > generally (???) be using a "similar" network.

In times of QoS this assumption may be very wrong. E.g. You have two 
wires to the internet. One 1MB and one 100MB, but for the 100MB only 
HTTP is allowed all other protocols have to use the 1MB link. If youre 
first connection is an HTTP one, then the 1MB link will be overshot, the 
other way round you'll have to wait for a long time to satisfy the 100MB 
if you increase just linear.

The first mentioned problem, with 1MB link get 100 MB data could be 
solved in one RTT, but in this case 99MB data get dropped. This would be 
an overshot of the factor 100 and much worser than slowstart.

Cheers Stefan

From: [email-masked] (Lachlan Andrew)
Date: Tue Apr  8 19:31:12 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Greetings Bob,

On 08/04/2008, Bob Briscoe <[email-masked]> wrote:
>  Just to be clear - we're not proposing charging for congestion like Kelly.
>  It's an important difference. Like the difference between volume charging
> and volume capping.

I didn't mean to misrepresent you, but to me, that's an IETF-level
"implementation detail", rather than an IRTF-level issue.  To be
maximally useful, it should be something which affects the
users'/applications' behaviour to avoid peak times.

The key is that the quantity you are both sharing out (however you do
it) is  *congestion*  rather than rate or volume.

>  Finally, I don't think 'we' have to use WFQ as an interim solution -
> fortunately 'we' don't make that choice - operators have protected a better
> form of fairness than TCP's on the Internet with WFQ for many years.

OK, but "we" have the choice of letting loose algorithms which will
increase pressure to deploy it where it isn't already enabled, don't
we?

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (Dirceu Cavendish)
Date: Tue Apr  8 00:42:55 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

OK, Bob. If you say we disagree, you know best :-)

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: [email-masked]
Sent: Monday, April 7, 2008 4:21:28 PM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceu,

If you say we're agreeing, then OK.

But I read a lot of tell-tale signs in your various postings that 
imply to me you are still thinking the way I believe has been a 
mistake in the past.

For instance, it's not at all hopeless if all sessions don't pass 
through a core router - they still have to pass through the point 
where the user's trust domain attaches to the rest of the Internet. 
That's always the case (otherwise the user isn't attached to the 
Internet). So there is no problem* seeing all the user's flows in one 
place (or a small number of places if multi-homing).

However, let's leave this one at that for now.

Bob

* The outstanding problem is that you can't see congestion ahead in 
the rest of the Internet from the attachment point (our re-feedback 
solution is one way to ensure trustworthy information about remote 
congestion is visble in packets crossing this attachment point).

At 23:55 07/04/2008, Dirceu Cavendish wrote:
>Let me state one thing clearly, first of all. I am NOT disagreeing 
>that flow rate equality as a fairness criterion has its problems. 
>So, you do not have to be defensive, as your last paragraph below.
>
>Second, about routing, I think you were the one who did not captured 
>what I wanted to say - communication is always a two way process, so 
>I take the blame as well. I understand that multiple sessions do not 
>require diverse routes in order to "confuse" routers, so as to 
>defeat some fairness scheme. However, one may imagine that, the same 
>way the router of application "attachment" is able to correlate 
>multiple session into a single application - as one of your papers 
>mention, a core router might be able to do it, provided that all 
>sessions follow the same path. When that is not the case, then the 
>situation is REALLY hopeless...That is what I meant.
>
>I believe we are in violent agreement, overall :-)...

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080407/0dd912aa/attachment.html
>From [email-masked]  Tue Apr  8 13:07:27 2008
From: [email-masked] (Bob Briscoe)
Date: Tue Apr  8 12:57:59 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked] m>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

Lachlan,

Just to be clear - we're not proposing charging for congestion like Kelly.

The reason we developed re-feedback was to be able to limit how much 
congestion each user causes.

It's an important difference. Like the difference between volume 
charging and volume capping.

We have also proposed a moving cap (using a single token bucket per 
user) that would never run out, just limit the congestion you cause 
to the fill rate if you exhausted it.

Finally, I don't think 'we' have to use WFQ as an interim solution - 
fortunately 'we' don't make that choice - operators have protected a 
better form of fairness than TCP's on the Internet with WFQ for many 
years. See previous posting.

Bob

At 19:12 04/04/2008, Lachlan Andrew wrote:
>Thanks for starting this thread, Matt.
>
>FWIW, I tend to agree with Bob that locking "bandwidth fairness" into
>the network is
>(a) not necessarily the sort of fairness that applications need
>and
>(b) hard to reverse when we finally do see what applications need.
>
>I agree that "TCP-friendliness" is overrated, but would be pushing the
>Kelly/Briscoe line of charging users for the congestion they cause,
>rather than arbitrarily limiting them to equality of a metric most
>users don't care about (instantaneous bandwidth).
>
>It may be sensible WFQ is used as an interim solution until we can get
>the network-wide agreement necessary for charging.  However, if we
>agree that charging is a better eventual solution, we should be
>careful not to lock in design decisions which will make it harder
>eventually.
>
>$0.02,
>Lachlan
>
>On 04/04/2008, Dirceu Cavendish <[email-masked]> wrote:
> >
> > Zeroing into the bandwidth metric, a per flow queueing would give routers
> > the power to share their link capacities however they find fit - WFQ,
> > max-min fairness, etc - independent of other factors, such as 
> sessions RTTs.
> > In this scenario, and aggressive congestion control would cause its own
> > traffic to be spilled, so "isolation" of sessions would be achieved.
>
>
>--
>Lachlan Andrew  Dept of Computer Science, Caltech
>1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
>Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
>http://netlab.caltech.edu/lachlan

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

From: [email-masked] (Dirceu Cavendish)
Date: Mon Apr  7 19:46:58 2008
Subject: [Iccrg] SSDT contributions
Message-ID: <[email-masked]>

I would like to solicit experimental/simulation evidence on the following topics.

- Transaction latencies taking too long to complete, in a large bandwidth delay product networks.
- Buffer overshoot, causing multiple segment losses/retransmits, potentially causing RTOs
- Session returning to SS and starting from scratch due to "application constrained" scenario

If you are working in any of these topics, and is willing to contribute, please come forward. Past results are good enough, to start a discussion on the threads...

We (KIT) can start with the second topic, buffer overshoot and multiple RTOs...
I will prepare some charts with data...

Rgds to all,
Dirceu

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080407/5c6e44c5/attachment.html
>From [email-masked]  Mon Apr  7 23:43:31 2008
From: [email-masked] (Bob Briscoe)
Date: Mon Apr  7 23:33:50 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080407/c38cf21c/attachment.html
>From [email-masked]  Mon Apr  7 23:55:07 2008
From: [email-masked] (Dirceu Cavendish)
Date: Mon Apr  7 23:45:24 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

Dear Bob,

Let me state one thing clearly, first of all. I am NOT disagreeing that flow rate equality as a fairness criterion has its problems. So, you do not have to be defensive, as your last paragraph below.

Second, about routing, I think you were the one who did not captured what I wanted to say - communication is always a two way process, so I take the blame as well. I understand that multiple sessions do not require diverse routes in order to "confuse" routers, so as to defeat some fairness scheme. However, one may imagine that, the same way the router of application "attachment" is able to correlate multiple session into a single application - as one of your papers mention, a core router might be able to do it, provided that all sessions follow the same path. When that is not the case, then the situation is REALLY hopeless...That is what I meant.

I believe we are in violent agreement, overall :-)...

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Monday, April 7, 2008 3:43:31 PM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceu,

I suggest everyone on this list reads John Nagle's RFC970 from 1985 - not the first half which is archaic now, but the second half on "
Fairness in Packet Switching Systems
", which introduced Fair Queuing. It's salutory reading. Fortunately descendents of that algorithm are in most consumer access servers (DSL, cable, cellular). Otherwise the Internet would be completely up the shoot. 

Admittedly FQ is also based on bogus science, but at least it's deployed to share backhaul links per-user, not per-flow. And at least many deployments use WFQ adapting its weights to be inversely proportional to the count of bytes seen in a flow. 
Fortunately, TCP is only left to arbitrate fairness where there's not too much contention (cores, campuses and enterprises). If fairness on those highly contended backhaul links had been left to TCP, the Internet would now be completely useless (as exemplified by our recent I-D draft-briscoe-tsvwg-relax-fairness-00.txt, which shows what the Internet would be like if left solely to TCP). This teaches us two things:
a) that flow rate equality isn't what governs fairness on the Internet
b) if it was, the Internet would now be utterly unbearable

responses inline...

At 17:07 05/04/2008, Dirceu Cavendish wrote:

Comments below...

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Saturday, April 5, 2008 3:42:24 AM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceau,

At 21:29 04/04/2008, Dirceu Cavendish wrote:
>Regarding place in the network, I mean everywhere - I guess all, as 
>per your question.

So then, a follow-on question is: if there are flows passing through 
router R1 from a user, how does R1 take account of flows from the 
same user not going through R1 at all?

IOW, I'm questioning whether the fairness question should be focused 
on sharing out each link (whether between flows or users) as it has 
been for the last n decades, or whether that was a distraction 
because we should have been looking at how much traffic is crossing 
trust boundaries between the principals (individuals, organisations, 
network operators), and how much of that traffic is causing 
congestion on /any/ other link. For instance, you have to be able to 
bring swarmcasting (e.g. BitTorrent) into your fairness framework.

<dc> The routing of TCP sessions is typically not under control of the application. Otherwise, I agree that it is difficult to enforce even the simple link level share of resources.

I don't think you've understood me. I'm not saying an app controls its routing. I'm saying an app controls which other endpoints it talks to (like BitTorrent).

I'm also saying that fairness should be the same whether an app or a user splits a flow into subflows or not. And it should be the same whether it splits its transfer activity across multiple endpoints.

I'm asserting that it is only practical to judge the fairness of a specific user using equipment at the point where that user attaches to another network, not at all the routers in the Internet through which that user's flows pass.

About BitTorrent type of applications, it would be nice to have some sort of fairness concept between these types of application, and across to other applications. But then, assuming there are N types of applications in the Internet (which I dont have to say N can be large), one would need to provide fairness definitions for N(N-1) cases.

Well, the metric (congestion volume) I gave in the paper I quoted applies for all applications, over all topologies, and over any timescale. And it's enforceable. And any form of fairness can be agreed around it among any sub-group of Internet users. And it relates correctly to fairness in real life.

 
So I guess the question is: where do we stop with the fairness issue. We can draw a parallel with security - how much paranoid are we about it :-)

Security paranoia isn't a parallel at all. The difference here is that there's been a huge mistake. A valid analogy would be if 'secure' had been defined to mean "only vulnerable when there's an 'r' in the month".

I'm not saying the prevailing view is merely different from mine, I'm saying flow-rate equality has been a whopping great massive 20 year long mistake.

What do you expect me to do if I've identified a mistake? Just shut up and continue to pretend I believe in the mistake? So we can continue to progress backwards, while I know which way is forwards?

Bob

Cheers,
Dirceu
</dc>

I think that's where Matt is going as well - moving the focus of 
fairness to the edges (the trust boundaries).

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked] >      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. 
____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080407/70f2459c/attachment-0001.html
>From [email-masked]  Tue Apr  8 00:21:28 2008
From: [email-masked] (Bob Briscoe)
Date: Tue Apr  8 00:11:36 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Dirceu,

If you say we're agreeing, then OK.

But I read a lot of tell-tale signs in your various postings that 
imply to me you are still thinking the way I believe has been a 
mistake in the past.

For instance, it's not at all hopeless if all sessions don't pass 
through a core router - they still have to pass through the point 
where the user's trust domain attaches to the rest of the Internet. 
That's always the case (otherwise the user isn't attached to the 
Internet). So there is no problem* seeing all the user's flows in one 
place (or a small number of places if multi-homing).

However, let's leave this one at that for now.

Bob

* The outstanding problem is that you can't see congestion ahead in 
the rest of the Internet from the attachment point (our re-feedback 
solution is one way to ensure trustworthy information about remote 
congestion is visble in packets crossing this attachment point).

At 23:55 07/04/2008, Dirceu Cavendish wrote:
>Let me state one thing clearly, first of all. I am NOT disagreeing 
>that flow rate equality as a fairness criterion has its problems. 
>So, you do not have to be defensive, as your last paragraph below.
>
>Second, about routing, I think you were the one who did not captured 
>what I wanted to say - communication is always a two way process, so 
>I take the blame as well. I understand that multiple sessions do not 
>require diverse routes in order to "confuse" routers, so as to 
>defeat some fairness scheme. However, one may imagine that, the same 
>way the router of application "attachment" is able to correlate 
>multiple session into a single application - as one of your papers 
>mention, a core router might be able to do it, provided that all 
>sessions follow the same path. When that is not the case, then the 
>situation is REALLY hopeless...That is what I meant.
>
>I believe we are in violent agreement, overall :-)...

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

From: [email-masked] (Douglas Leith)
Date: Mon Apr  7 14:51:24 2008
Subject: [Iccrg] updated h-tcp ID
References: <[email-masked]>
Message-ID: <[email-masked]>

I've posted an updated version of the h-tcp internet draft at http://
www.ietf.org/internet-drafts/draft-leith-tcp-htcp-06.txt.  The ID is
changed to be more streamlined (general discussion removed etc) and
so that the language conforms to RCF2119.  Comments welcome.

Although this ID predates the formation of the ICCRG and was
originally submitted direct to TCPM for consideration as a work item  
and experimental RFC,
the new process for moving  things on is now via ICCRG review and so  
I'd like to
ask the TCPM chairs to request consideration of the ID by ICCRG (as  
per the process here: http://www3.tools.ietf.org/group/irtf/trac/wiki/ 
ICCRG)

Doug
>>

From: [email-masked] (Michael Welzl)
Date: Mon Apr  7 13:48:57 2008
Subject: [Iccrg] updated h-tcp ID
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Thanks Doug!

I created a small website for it, as usual:
http://www3.tools.ietf.org/group/irtf/trac/wiki/ICCRG_htcp

Additional pointers are welcome, just let me know and I'll add them.

So ... do we have volunteer reviewers in the house?

Cheers,
Michael

On Mon, 2008-04-07 at 11:40 +0100, Douglas Leith wrote:
> I've posted an updated version of the h-tcp internet draft at http:// 
> www.ietf.org/internet-drafts/draft-leith-tcp-htcp-06.txt.  The ID is  
> changed to be more streamlined (general discussion removed etc) and  
> so that the language conforms to RCF2119.  Comments welcome.    
> Although this ID predates the formation of the ICCRG and was  
> originally submitted direct to TCPM,  the new process for moving  
> things on is now via ICCRG review and so I'd like to submit the ID  
> for consideration by ICCRG.
> 
> Doug
> 
> _______________________________________________
> Iccrg mailing list
> [email-masked]
> http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

From: [email-masked] (Douglas Leith)
Date: Mon Apr  7 11:29:28 2008
Subject: [Iccrg] updated h-tcp ID
Message-ID: <[email-masked]>

I've posted an updated version of the h-tcp internet draft at http:// 
www.ietf.org/internet-drafts/draft-leith-tcp-htcp-06.txt.  The ID is  
changed to be more streamlined (general discussion removed etc) and  
so that the language conforms to RCF2119.  Comments welcome.    
Although this ID predates the formation of the ICCRG and was  
originally submitted direct to TCPM,  the new process for moving  
things on is now via ICCRG review and so I'd like to submit the ID  
for consideration by ICCRG.

Doug

From: [email-masked] (Dirceu Cavendish)
Date: Sat Apr  5 16:58:25 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

Comments below...

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Saturday, April 5, 2008 3:42:24 AM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceau,

At 21:29 04/04/2008, Dirceu Cavendish wrote:
>Regarding place in the network, I mean everywhere - I guess all, as 
>per your question.

So then, a follow-on question is: if there are flows passing through 
router R1 from a user, how does R1 take account of flows from the 
same user not going through R1 at all?

IOW, I'm questioning whether the fairness question should be focused 
on sharing out each link (whether between flows or users) as it has 
been for the last n decades, or whether that was a distraction 
because we should have been looking at how much traffic is crossing 
trust boundaries between the principals (individuals, organisations, 
network operators), and how much of that traffic is causing 
congestion on /any/ other link. For instance, you have to be able to 
bring swarmcasting (e.g. BitTorrent) into your fairness framework.

<dc> The routing of TCP sessions is typically not under control of the application. Otherwise, I agree that it is difficult to enforce even the simple link level share of resources.

About BitTorrent type of applications, it would be nice to have some sort of fairness concept between these types of application, and across to other applications. But then, assuming there are N types of applications in the Internet (which I dont have to say N can be large), one would need to provide fairness definitions for N(N-1) cases.

So I guess the question is: where do we stop with the fairness issue. We can draw a parallel with security - how much paranoid are we about it :-)

Cheers,
Dirceu
</dc>

I think that's where Matt is going as well - moving the focus of 
fairness to the edges (the trust boundaries).

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080405/724e9305/attachment.html
>From [email-masked]  Sat Apr  5 19:10:32 2008
From: [email-masked] (Lachlan Andrew)
Date: Sat Apr  5 19:01:17 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On 05/04/2008, Dirceu Cavendish <[email-masked]> wrote:
>
> About BitTorrent type of applications, it would be nice to have some sort of fairness concept between these types of application, and across to other applications. But then, assuming there are N types of applications in the Internet (which I dont have to say N can be large), one would need to provide fairness definitions for N(N-1) cases.

We don't need N(N-1) cases.  Just as we use money as an abstraction to
avoid exchange rates for the k(k-1) goods which could be bartered, we
could have a "congestion currency".  That is Kelly's approach.  His
work was based on instantaneous rates etc, so would need to be
extended, but shows a way forward.

We might need a few, representing different QoS parameters;  Kevin
Tang has been working on extending that framework to networks with
multiple "congestion prices" at each link.

Another issue is how to deal with fairness which isn't just "dividing
a resource".  For example, consider one flow wanting high throughput,
which is achieved with queueing, and another sharing that bottleneck
wanting low jitter, which is achieved by keeping buffers empty.  What
is the "fair" behaviour?

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (Bob Briscoe)
Date: Fri Apr  4 19:55:48 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080404/54635637/attachment.html
>From [email-masked]  Fri Apr  4 20:18:41 2008
From: [email-masked] (Dirceu Cavendish)
Date: Fri Apr  4 20:09:38 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

Bob,

I agree that even if we tried hard some form of per-session fairness, a smart application could work around it. So, effectively, we would be pushing the problem up the protocol stack. And agree again that, in that case, "instantaneous" metrics tend to make less and less sense, once you are looking at it from an application perspective.

About routers, they have the power to serve packets however way they decide to (and can afford to) do. With deep packet inspection, one can imagine a router "sorting packets" on any protocol level, up to soap/http, etc. So, about your differentiation between "any" router from a special router - yes, a legacy router may not do unlimited deep packet processing - it will stop somewhere in the protocol stack. In this case, there is always the "trick" to work on one level up to forfeit whatever resource management strategy is being exercised down below...

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Friday, April 4, 2008 12:05:07 PM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceau,

Why do you want the comparison to be between /sessions/? Given sessions are defined by identifiers taken arbitrarily many times from a large number space.

And are all your metrics deliberately instantaneous metrics? Or do you think that fairness should actually be judged over time? 

And when you say that routers should have the power to share their link capacity, do you mean any router, or only a router able to see all the sessions of one user?

Bob

At 18:50 04/04/2008, Dirceu Cavendish wrote:

One more heteric coming out of the woods. However, my disclaimer is that I sympathize with the concept :-)

On top of all technical difficulties, the concept of "fairness" in itself is elusive. All things being equal, then one session should have the  same (pick one - bandwidth, transaction response time, packet loss) than the others. The utopy of it comes from the "all things being equal" part - in the Internet?

Zeroing into the bandwidth metric, a per flow queueing would give routers the power to share their link capacities however they find fit - WFQ, max-min fairness, etc - independent of other factors, such as sessions RTTs. In this scenario, and aggressive congestion control would cause its own traffic to be spilled, so "isolation" of sessions would be achieved.

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Matt Mathis <[email-masked]>
Cc: [email-masked]
Sent: Friday, April 4, 2008 6:53:43 AM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Matt,

>From one heretic to another...

0) Priority to shortest jobs
In your bullet list you missed a point that I believe is the most 
important one: about arrivals and departures of whole flows. If you 
have a mix of flow activity at a shared bottleneck, some continuously 
streaming and some intermittent (as we do on the Internet), you can 
make the intermittent flows go much much faster without hardly 
prolonging the completion time of the continuous flows. It's totally 
unfair (and very inefficient) for the intermittent flows to get the 
same bit rate as the continuous flows, because their behaviour is 
much more multiplexable. A picture may help: 
< http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0801cfp/shortestJobsPriority.png >

If you're intrigued by how we'd move to such a world, I recently 
posted a bit more on this here: 
< http://episteme.arstechnica.com/eve/forums?a=tpc&s=50009562&f=174096756&m=703000231931&r=160007441931#160007441931 >

Now thoughts on your main points:

1) Fair queuing
Many broadband deployments already do some form of per-flow FQ 
(usually WFQ) at the most likely bottleneck (the broadband remote 
access server, BRAS). I'm less sure what common practice is in 
broadband cellular networks. I believe WFQ is not such an obvious 
choice for radio access because of the unpredictable radio link rate. 
I think there are a range of schemes, some distributed at the base 
station and others centralised at the radio network controller (RNC).

Incidentally, back to the first point, I recently realised that most 
deployed per-flow FQ tends to help short duration flows, by giving 
higher priority at the start of a flow and reducing the longer each 
flow continues. Altho this helps a little bit today, it's also 
actually a huge potential problem for the future - same problem as 
TCP: it still converges to the incorrect goal of sharing out bit-rate 
equally for each flow at the bottleneck - tho at least at a BRAS it's 
done separately per each user.

My point about priority to shortest jobs (and your point about huge 
differences between no.s of flows per app) shows that flow rates need 
to be /very/ unequal to be fair. So per-flow FQ embedded in the 
network will be fighting what we really need to do in the transport 
in future (tho operators would obviously turn off per-flow FQ if they 
were trying to encourage the new world I'm talking about).

2) Edge bottlenecks protecting core
Although it would be nice, we can't mandate that the core be 
protected by bottlenecks at the edge. Technology limits & economics 
determine these things, not the IETF/IRTF:
* At present technology trends are moving bottlenecks gradually 
closer to the core as access rates increase, because the newest 
access is now using the same technology as the core (optics) and 
there's nothing faster on the horizon.
* However, economics always pushes the bottleneck location the other 
way - outwards. The cost of a bps of logical channel capacity follows 
about a square root (?) law wrt to the bandwidth of the physical pipe 
in which the logical channel sits. Ie. where you need many physical 
cables/fibres to cover a dispersed area, the cost per bps is much 
greater than where each bps can be provided within a few large pipes.

The net effect of these two opposing forces is pushing bottlenecks 
inwards at the moment - particularly onto border routers. The message 
of a talk I gave to a recent workshop (on photonics research and 
future Internet design) was that the challenge is to find ways to 
push complex trust-related stuff currently done at border routers 
outward to the edge, so we can have dumb all-optical interconnection 
without electronics: 
< http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0709ecoc-fid >

3) RTT fairness
I'd say this is only a small part of the problem, because it's 
relatively easy to solve in the transport alone - e.g. FAST TCP 
[Jin04:FAST_TCP] ensures its dynamics are slower for longer RTTs but, 
even tho it takes longer to get there, it ends up at the same rate as 
competing FAST TCPs with shorter RTTs.

Bob

[Jin04:FAST_TCP] Cheng Jin, David Wei and Steven Low "FAST TCP: 
Motivation, Architecture, Algorithms, Performance", In "Proc. IEEE 
Conference on Computer Communications (Infocomm'04)" (March, 2004)

At 17:35 02/04/2008, Matt Mathis wrote:
>I just attended the "The Future of TCP: Train-wreck or Evolution?" 
>at Stanford last week, and it solidified my thoughts on a subject 
>that is sure to be controversial.
>
>I think it is time to abandon the concept of "TCP-Friendly" and 
>instead expect the network to protect itself and other users from 
>aggressive protocols and applications.  For the moment I am going to 
>assume two mechanisms, although I suspect that there will prove to be more.
>
>1) Deploy some form of Fair Queuing at the edges of the network.
>
>2) Protect the core by bottlenecks at the edges of the Internet.
>
>I observe that both of these mechanisms are already being 
>implemented due to existing market forces, and the natural 
>consequence of their implementation is to make TCP-friendliness a 
>whole lot less important.  I admit that it is not clear at this 
>point if these two mechanisms will ultimately prove to be sufficient 
>to address fairness all situations, such as over loaded core 
>routers, but I suspect that sufficient mechanisms do exist.
>
>Supporting arguments:
>
>FQ is already being deployed at the edges to solve several existing 
>and growing fairness problems:
>
>* Non-IETF, UDP protocols that are non-responsive.
>
>* P2P and other applications that open huge numbers of connections.
>
>* Stock TCP is egregiously unfair when very short RTT flows compete with wide
>  area flows.  This can be a real killer in a number of settings such as data
>  centers and university campuses.  The symptoms of this problem will become
>  more pronounced as TCP autotuning continues to be rolled out in Vista,
>  Linux, and various BSDs.
>
>* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
>  single TCP flow with sufficient data will cause congestion somewhere in the
>  network.  At the very least this will gradually force the retirement of
>  drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
>  by itself is insufficient to solve the other fairness problems, it will not
>  be the first choice replacement.
>
>I should note that "Fair Queuing"is overly specific.  The network 
>needs to do something to large flows to prevent them from 
>overwhelming smaller flows and to limit queue occupancy.  FQ is one 
>way, but there are others.
>
>If you have ever shared a drop-tail home router with a teenager, you 
>might have observed some of these issues first hand, as has 
>Comcast.[2] As I understand it, some form of enforced fairness is 
>now part of all commercial broadband services.
>
>The core of the Internet is already mostly protected by bottlenecks 
>at the edges.  This is because ISPs can balance the allocation of 
>revenue from customers between the relatively expensive access link, 
>its own backbone links and interconnections to other ISPs.  Since 
>congestion in the core has proven to cause complaints from 
>commercial customers (and perhaps SLA problems), most providers are 
>careful to keep adequate capacity in the core, and can do so pretty 
>easily, as long as their duty cycle models hold true.
>
>Are these two mechanisms sufficient to make TCP-friendliness 
>completely moot? Probably not.
>
>We still have some work to do:
>
>First, stop whining about non-TCP-friendly protocols.  The are here 
>to stay and they can't hear us.  We are wasting our breath and 
>impeding real progress in well designed alternatives to 
>"TCP-friendly".  This concept came from an era when the Internet was 
>a gentleman's club, but now it needs to be retired.
>
>Second, blame the network when the network deserves it.  In 
>particular if there are drop tail queues without AQM, be very 
>suspicious of RFC2309 problems.  In fact every drop tail queue 
>without AQM should be viewed as a bug waiting to bite 
>someone.  Likewise remember that "TCP-friendly" is extremely unfair 
>when the RTTs are extremely different.
>
>Third, think about the hard cases: over loaded interconnects, 
>failure conditions, etc.  Can FQ be approximated at core 
>scales?  Where else are my proposed mechanisms insufficient?  I sure 
>there are some.
>
>Fourth, start dreaming about what it would take to make Moore's law 
>apply to end-to-end protocol performance, as it does to just about 
>everything else in the computing universe.  I suspect that in some 
>future hindsight, we will come to realize that TCP-friendly was 
>actually a untenable position, and has held us back from important innovations.
>
>[1] RFC2309 "Recommendations on Queue Management and Congestion 
>Avoidance in the Internet", Bob Braden, et al.
>
>[2] Richard Bennett "New and Improved Traffic Shaping" 
> http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/ 
>------------------
>
>It was a very stimulating conference!
>Thanks Nandita and everyone else who made it happen!
>--MM--
>-------------------------------------------
>Matt Mathis    http://staff.psc.edu/mathis
>Work:412.268.3319    Home/Cell:412.654.7529
>-------------------------------------------
>
>
>_______________________________________________
>Iccrg mailing list
>[email-masked]
> http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

____________________________________________________________________________
Bob Briscoe, <[email-masked] >      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. 
____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080404/30efac4e/attachment-0001.html
>From [email-masked]  Fri Apr  4 21:24:31 2008
From: [email-masked] (Bob Briscoe)
Date: Fri Apr  4 21:15:16 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080404/beae45ca/attachment.html
>From [email-masked]  Fri Apr  4 21:29:29 2008
From: [email-masked] (Dirceu Cavendish)
Date: Fri Apr  4 21:21:26 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

Yes, I have read your manifesto... sorry, your paper ;-)

Regarding place in the network, I mean everywhere - I guess all, as per your question.
Sorry about the confusion.
Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Friday, April 4, 2008 1:24:31 PM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceu,

At 20:18 04/04/2008, Dirceu Cavendish wrote:

Bob,
I agree that even if we tried hard some form of per-session fairness, a smart application could work around it. So, effectively, we would be pushing the problem up the protocol stack. And agree again that, in that case, "instantaneous" metrics tend to make less and less sense, once you are looking at it from an application perspective.

Have you read this, which I assume prompted Matt's use of the word "Heresy"?
Flow Rate Fairness: Dismantling a Religion
< http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/refb/#rateFairDis >

About routers, they have the power to serve packets however way they decide to (and can afford to) do. With deep packet inspection, one can imagine a router "sorting packets" on any protocol level, up to soap/http, etc. So, about your differentiation between "any" router from a special router - yes, a legacy router may not do unlimited deep packet processing - it will stop somewhere in the protocol stack. In this case, there is always the "trick" to work on one level up to forfeit whatever resource management strategy is being exercised down below...

You've misunderstood my question. It's about which /location/ in the topology you are assuming routers are at. Have a re-read of the question below.

Bob

 

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Dirceu Cavendish <[email-masked]>
Cc: Matt Mathis <[email-masked]>; [email-masked]
Sent: Friday, April 4, 2008 12:05:07 PM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Dirceau,

Why do you want the comparison to be between /sessions/? Given sessions are defined by identifiers taken arbitrarily many times from a large number space.

And are all your metrics deliberately instantaneous metrics? Or do you think that fairness should actually be judged over time? 

And when you say that routers should have the power to share their link capacity, do you mean any router, or only a router able to see all the sessions of one user?

Bob

At 18:50 04/04/2008, Dirceu Cavendish wrote:

One more heteric coming out of the woods. However, my disclaimer is that I sympathize with the concept :-)

On top of all technical difficulties, the concept of "fairness" in itself is elusive. All things being equal, then one session should have the  same (pick one - bandwidth, transaction response time, packet loss) than the others. The utopy of it comes from the "all things being equal" part - in the Internet?

Zeroing into the bandwidth metric, a per flow queueing would give routers the power to share their link capacities however they find fit - WFQ, max-min fairness, etc - independent of other factors, such as sessions RTTs. In this scenario, and aggressive congestion control would cause its own traffic to be spilled, so "isolation" of sessions would be achieved.

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Matt Mathis <[email-masked]>
Cc: [email-masked]
Sent: Friday, April 4, 2008 6:53:43 AM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Matt,

>From one heretic to another...

0) Priority to shortest jobs
In your bullet list you missed a point that I believe is the most 
important one: about arrivals and departures of whole flows. If you 
have a mix of flow activity at a shared bottleneck, some continuously 
streaming and some intermittent (as we do on the Internet), you can 
make the intermittent flows go much much faster without hardly 
prolonging the completion time of the continuous flows. It's totally 
unfair (and very inefficient) for the intermittent flows to get the 
same bit rate as the continuous flows, because their behaviour is 
much more multiplexable. A picture may help: 
< http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0801cfp/shortestJobsPriority.png >

If you're intrigued by how we'd move to such a world, I recently 
posted a bit more on this here: 
< http://episteme.arstechnica.com/eve/forums?a=tpc&s=50009562&f=174096756&m=703000231931&r=160007441931#160007441931 >

Now thoughts on your main points:

1) Fair queuing
Many broadband deployments already do some form of per-flow FQ 
(usually WFQ) at the most likely bottleneck (the broadband remote 
access server, BRAS). I'm less sure what common practice is in 
broadband cellular networks. I believe WFQ is not such an obvious 
choice for radio access because of the unpredictable radio link rate. 
I think there are a range of schemes, some distributed at the base 
station and others centralised at the radio network controller (RNC).

Incidentally, back to the first point, I recently realised that most 
deployed per-flow FQ tends to help short duration flows, by giving 
higher priority at the start of a flow and reducing the longer each 
flow continues. Altho this helps a little bit today, it's also 
actually a huge potential problem for the future - same problem as 
TCP: it still converges to the incorrect goal of sharing out bit-rate 
equally for each flow at the bottleneck - tho at least at a BRAS it's 
done separately per each user.

My point about priority to shortest jobs (and your point about huge 
differences between no.s of flows per app) shows that flow rates need 
to be /very/ unequal to be fair. So per-flow FQ embedded in the 
network will be fighting what we really need to do in the transport 
in future (tho operators would obviously turn off per-flow FQ if they 
were trying to encourage the new world I'm talking about).

2) Edge bottlenecks protecting core
Although it would be nice, we can't mandate that the core be 
protected by bottlenecks at the edge. Technology limits & economics 
determine these things, not the IETF/IRTF:
* At present technology trends are moving bottlenecks gradually 
closer to the core as access rates increase, because the newest 
access is now using the same technology as the core (optics) and 
there's nothing faster on the horizon.
* However, economics always pushes the bottleneck location the other 
way - outwards. The cost of a bps of logical channel capacity follows 
about a square root (?) law wrt to the bandwidth of the physical pipe 
in which the logical channel sits. Ie. where you need many physical 
cables/fibres to cover a dispersed area, the cost per bps is much 
greater than where each bps can be provided within a few large pipes.

The net effect of these two opposing forces is pushing bottlenecks 
inwards at the moment - particularly onto border routers. The message 
of a talk I gave to a recent workshop (on photonics research and 
future Internet design) was that the challenge is to find ways to 
push complex trust-related stuff currently done at border routers 
outward to the edge, so we can have dumb all-optical interconnection 
without electronics: 
< http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0709ecoc-fid >

3) RTT fairness
I'd say this is only a small part of the problem, because it's 
relatively easy to solve in the transport alone - e.g. FAST TCP 
[Jin04:FAST_TCP] ensures its dynamics are slower for longer RTTs but, 
even tho it takes longer to get there, it ends up at the same rate as 
competing FAST TCPs with shorter RTTs.

Bob

[Jin04:FAST_TCP] Cheng Jin, David Wei and Steven Low "FAST TCP: 
Motivation, Architecture, Algorithms, Performance", In "Proc. IEEE 
Conference on Computer Communications (Infocomm'04)" (March, 2004)

At 17:35 02/04/2008, Matt Mathis wrote:
>I just attended the "The Future of TCP: Train-wreck or Evolution?" 
>at Stanford last week, and it solidified my thoughts on a subject 
>that is sure to be controversial.
>
>I think it is time to abandon the concept of "TCP-Friendly" and 
>instead expect the network to protect itself and other users from 
>aggressive protocols and applications.  For the moment I am going to 
>assume two mechanisms, although I suspect that there will prove to be more.
>
>1) Deploy some form of Fair Queuing at the edges of the network.
>
>2) Protect the core by bottlenecks at the edges of the Internet.
>
>I observe that both of these mechanisms are already being 
>implemented due to existing market forces, and the natural 
>consequence of their implementation is to make TCP-friendliness a 
>whole lot less important.  I admit that it is not clear at this 
>point if these two mechanisms will ultimately prove to be sufficient 
>to address fairness all situations, such as over loaded core 
>routers, but I suspect that sufficient mechanisms do exist.
>
>Supporting arguments:
>
>FQ is already being deployed at the edges to solve several existing 
>and growing fairness problems:
>
>* Non-IETF, UDP protocols that are non-responsive.
>
>* P2P and other applications that open huge numbers of connections.
>
>* Stock TCP is egregiously unfair when very short RTT flows compete with wide
>  area flows.  This can be a real killer in a number of settings such as data
>  centers and university campuses.  The symptoms of this problem will become
>  more pronounced as TCP autotuning continues to be rolled out in Vista,
>  Linux, and various BSDs.
>
>* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
>  single TCP flow with sufficient data will cause congestion somewhere in the
>  network.  At the very least this will gradually force the retirement of
>  drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
>  by itself is insufficient to solve the other fairness problems, it will not
>  be the first choice replacement.
>
>I should note that "Fair Queuing"is overly specific.  The network 
>needs to do something to large flows to prevent them from 
>overwhelming smaller flows and to limit queue occupancy.  FQ is one 
>way, but there are others.
>
>If you have ever shared a drop-tail home router with a teenager, you 
>might have observed some of these issues first hand, as has 
>Comcast.[2] As I understand it, some form of enforced fairness is 
>now part of all commercial broadband services.
>
>The core of the Internet is already mostly protected by bottlenecks 
>at the edges.  This is because ISPs can balance the allocation of 
>revenue from customers between the relatively expensive access link, 
>its own backbone links and interconnections to other ISPs.  Since 
>congestion in the core has proven to cause complaints from 
>commercial customers (and perhaps SLA problems), most providers are 
>careful to keep adequate capacity in the core, and can do so pretty 
>easily, as long as their duty cycle models hold true.
>
>Are these two mechanisms sufficient to make TCP-friendliness 
>completely moot? Probably not.
>
>We still have some work to do:
>
>First, stop whining about non-TCP-friendly protocols.  The are here 
>to stay and they can't hear us.  We are wasting our breath and 
>impeding real progress in well designed alternatives to 
>"TCP-friendly".  This concept came from an era when the Internet was 
>a gentleman's club, but now it needs to be retired.
>
>Second, blame the network when the network deserves it.  In 
>particular if there are drop tail queues without AQM, be very 
>suspicious of RFC2309 problems.  In fact every drop tail queue 
>without AQM should be viewed as a bug waiting to bite 
>someone.  Likewise remember that "TCP-friendly" is extremely unfair 
>when the RTTs are extremely different.
>
>Third, think about the hard cases: over loaded interconnects, 
>failure conditions, etc.  Can FQ be approximated at core 
>scales?  Where else are my proposed mechanisms insufficient?  I sure 
>there are some.
>
>Fourth, start dreaming about what it would take to make Moore's law 
>apply to end-to-end protocol performance, as it does to just about 
>everything else in the computing universe.  I suspect that in some 
>future hindsight, we will come to realize that TCP-friendly was 
>actually a untenable position, and has held us back from important innovations.
>
>[1] RFC2309 "Recommendations on Queue Management and Congestion 
>Avoidance in the Internet", Bob Braden, et al.
>
>[2] Richard Bennett "New and Improved Traffic Shaping" 
> http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/ 
>------------------
>
>It was a very stimulating conference!
>Thanks Nandita and everyone else who made it happen!
>--MM--
>-------------------------------------------
>Matt Mathis    http://staff.psc.edu/mathis
>Work:412.268.3319    Home/Cell:412.654.7529
>-------------------------------------------
>
>
>_______________________________________________
>Iccrg mailing list
>[email-masked]
> http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

____________________________________________________________________________
Bob Briscoe, <[email-masked] >      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. 

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. 
____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080404/00f758b1/attachment-0001.html
>From [email-masked]  Sat Apr  5 11:42:24 2008
From: [email-masked] (Bob Briscoe)
Date: Sat Apr  5 11:33:31 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Dirceau,

At 21:29 04/04/2008, Dirceu Cavendish wrote:
>Regarding place in the network, I mean everywhere - I guess all, as 
>per your question.

So then, a follow-on question is: if there are flows passing through 
router R1 from a user, how does R1 take account of flows from the 
same user not going through R1 at all?

IOW, I'm questioning whether the fairness question should be focused 
on sharing out each link (whether between flows or users) as it has 
been for the last n decades, or whether that was a distraction 
because we should have been looking at how much traffic is crossing 
trust boundaries between the principals (individuals, organisations, 
network operators), and how much of that traffic is causing 
congestion on /any/ other link. For instance, you have to be able to 
bring swarmcasting (e.g. BitTorrent) into your fairness framework.

I think that's where Matt is going as well - moving the focus of 
fairness to the edges (the trust boundaries).

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

From: [email-masked] (Dirceu Cavendish)
Date: Fri Apr  4 18:41:18 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

One more heteric coming out of the woods. However, my disclaimer is that I sympathize with the concept :-)

On top of all technical difficulties, the concept of "fairness" in itself is elusive. All things being equal, then one session should have the  same (pick one - bandwidth, transaction response time, packet loss) than the others. The utopy of it comes from the "all things being equal" part - in the Internet?

Zeroing into the bandwidth metric, a per flow queueing would give routers the power to share their link capacities however they find fit - WFQ, max-min fairness, etc - independent of other factors, such as sessions RTTs. In this scenario, and aggressive congestion control would cause its own traffic to be spilled, so "isolation" of sessions would be achieved.

Dirceu

----- Original Message ----
From: Bob Briscoe <[email-masked]>
To: Matt Mathis <[email-masked]>
Cc: [email-masked]
Sent: Friday, April 4, 2008 6:53:43 AM
Subject: Re: [Iccrg] Heresy following "TCP: Train-wreck"

Matt,

>From one heretic to another...

0) Priority to shortest jobs
In your bullet list you missed a point that I believe is the most 
important one: about arrivals and departures of whole flows. If you 
have a mix of flow activity at a shared bottleneck, some continuously 
streaming and some intermittent (as we do on the Internet), you can 
make the intermittent flows go much much faster without hardly 
prolonging the completion time of the continuous flows. It's totally 
unfair (and very inefficient) for the intermittent flows to get the 
same bit rate as the continuous flows, because their behaviour is 
much more multiplexable. A picture may help: 
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0801cfp/shortestJobsPriority.png>

If you're intrigued by how we'd move to such a world, I recently 
posted a bit more on this here: 
<http://episteme.arstechnica.com/eve/forums?a=tpc&s=50009562&f=174096756&m=703000231931&r=160007441931#160007441931>

Now thoughts on your main points:

1) Fair queuing
Many broadband deployments already do some form of per-flow FQ 
(usually WFQ) at the most likely bottleneck (the broadband remote 
access server, BRAS). I'm less sure what common practice is in 
broadband cellular networks. I believe WFQ is not such an obvious 
choice for radio access because of the unpredictable radio link rate. 
I think there are a range of schemes, some distributed at the base 
station and others centralised at the radio network controller (RNC).

Incidentally, back to the first point, I recently realised that most 
deployed per-flow FQ tends to help short duration flows, by giving 
higher priority at the start of a flow and reducing the longer each 
flow continues. Altho this helps a little bit today, it's also 
actually a huge potential problem for the future - same problem as 
TCP: it still converges to the incorrect goal of sharing out bit-rate 
equally for each flow at the bottleneck - tho at least at a BRAS it's 
done separately per each user.

My point about priority to shortest jobs (and your point about huge 
differences between no.s of flows per app) shows that flow rates need 
to be /very/ unequal to be fair. So per-flow FQ embedded in the 
network will be fighting what we really need to do in the transport 
in future (tho operators would obviously turn off per-flow FQ if they 
were trying to encourage the new world I'm talking about).

2) Edge bottlenecks protecting core
Although it would be nice, we can't mandate that the core be 
protected by bottlenecks at the edge. Technology limits & economics 
determine these things, not the IETF/IRTF:
* At present technology trends are moving bottlenecks gradually 
closer to the core as access rates increase, because the newest 
access is now using the same technology as the core (optics) and 
there's nothing faster on the horizon.
* However, economics always pushes the bottleneck location the other 
way - outwards. The cost of a bps of logical channel capacity follows 
about a square root (?) law wrt to the bandwidth of the physical pipe 
in which the logical channel sits. Ie. where you need many physical 
cables/fibres to cover a dispersed area, the cost per bps is much 
greater than where each bps can be provided within a few large pipes.

The net effect of these two opposing forces is pushing bottlenecks 
inwards at the moment - particularly onto border routers. The message 
of a talk I gave to a recent workshop (on photonics research and 
future Internet design) was that the challenge is to find ways to 
push complex trust-related stuff currently done at border routers 
outward to the edge, so we can have dumb all-optical interconnection 
without electronics: 
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0709ecoc-fid>

3) RTT fairness
I'd say this is only a small part of the problem, because it's 
relatively easy to solve in the transport alone - e.g. FAST TCP 
[Jin04:FAST_TCP] ensures its dynamics are slower for longer RTTs but, 
even tho it takes longer to get there, it ends up at the same rate as 
competing FAST TCPs with shorter RTTs.

Bob

[Jin04:FAST_TCP] Cheng Jin, David Wei and Steven Low "FAST TCP: 
Motivation, Architecture, Algorithms, Performance", In "Proc. IEEE 
Conference on Computer Communications (Infocomm'04)" (March, 2004)

At 17:35 02/04/2008, Matt Mathis wrote:
>I just attended the "The Future of TCP: Train-wreck or Evolution?" 
>at Stanford last week, and it solidified my thoughts on a subject 
>that is sure to be controversial.
>
>I think it is time to abandon the concept of "TCP-Friendly" and 
>instead expect the network to protect itself and other users from 
>aggressive protocols and applications.  For the moment I am going to 
>assume two mechanisms, although I suspect that there will prove to be more.
>
>1) Deploy some form of Fair Queuing at the edges of the network.
>
>2) Protect the core by bottlenecks at the edges of the Internet.
>
>I observe that both of these mechanisms are already being 
>implemented due to existing market forces, and the natural 
>consequence of their implementation is to make TCP-friendliness a 
>whole lot less important.  I admit that it is not clear at this 
>point if these two mechanisms will ultimately prove to be sufficient 
>to address fairness all situations, such as over loaded core 
>routers, but I suspect that sufficient mechanisms do exist.
>
>Supporting arguments:
>
>FQ is already being deployed at the edges to solve several existing 
>and growing fairness problems:
>
>* Non-IETF, UDP protocols that are non-responsive.
>
>* P2P and other applications that open huge numbers of connections.
>
>* Stock TCP is egregiously unfair when very short RTT flows compete with wide
>  area flows.  This can be a real killer in a number of settings such as data
>  centers and university campuses.  The symptoms of this problem will become
>  more pronounced as TCP autotuning continues to be rolled out in Vista,
>  Linux, and various BSDs.
>
>* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
>  single TCP flow with sufficient data will cause congestion somewhere in the
>  network.  At the very least this will gradually force the retirement of
>  drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
>  by itself is insufficient to solve the other fairness problems, it will not
>  be the first choice replacement.
>
>I should note that "Fair Queuing"is overly specific.  The network 
>needs to do something to large flows to prevent them from 
>overwhelming smaller flows and to limit queue occupancy.  FQ is one 
>way, but there are others.
>
>If you have ever shared a drop-tail home router with a teenager, you 
>might have observed some of these issues first hand, as has 
>Comcast.[2] As I understand it, some form of enforced fairness is 
>now part of all commercial broadband services.
>
>The core of the Internet is already mostly protected by bottlenecks 
>at the edges.  This is because ISPs can balance the allocation of 
>revenue from customers between the relatively expensive access link, 
>its own backbone links and interconnections to other ISPs.  Since 
>congestion in the core has proven to cause complaints from 
>commercial customers (and perhaps SLA problems), most providers are 
>careful to keep adequate capacity in the core, and can do so pretty 
>easily, as long as their duty cycle models hold true.
>
>Are these two mechanisms sufficient to make TCP-friendliness 
>completely moot? Probably not.
>
>We still have some work to do:
>
>First, stop whining about non-TCP-friendly protocols.  The are here 
>to stay and they can't hear us.  We are wasting our breath and 
>impeding real progress in well designed alternatives to 
>"TCP-friendly".  This concept came from an era when the Internet was 
>a gentleman's club, but now it needs to be retired.
>
>Second, blame the network when the network deserves it.  In 
>particular if there are drop tail queues without AQM, be very 
>suspicious of RFC2309 problems.  In fact every drop tail queue 
>without AQM should be viewed as a bug waiting to bite 
>someone.  Likewise remember that "TCP-friendly" is extremely unfair 
>when the RTTs are extremely different.
>
>Third, think about the hard cases: over loaded interconnects, 
>failure conditions, etc.  Can FQ be approximated at core 
>scales?  Where else are my proposed mechanisms insufficient?  I sure 
>there are some.
>
>Fourth, start dreaming about what it would take to make Moore's law 
>apply to end-to-end protocol performance, as it does to just about 
>everything else in the computing universe.  I suspect that in some 
>future hindsight, we will come to realize that TCP-friendly was 
>actually a untenable position, and has held us back from important innovations.
>
>[1] RFC2309 "Recommendations on Queue Management and Congestion 
>Avoidance in the Internet", Bob Braden, et al.
>
>[2] Richard Bennett "New and Improved Traffic Shaping" 
>http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/
>------------------
>
>It was a very stimulating conference!
>Thanks Nandita and everyone else who made it happen!
>--MM--
>-------------------------------------------
>Matt Mathis    http://staff.psc.edu/mathis
>Work:412.268.3319    Home/Cell:412.654.7529
>-------------------------------------------
>
>
>_______________________________________________
>Iccrg mailing list
>[email-masked]
>http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080404/8316c4ab/attachment-0001.html
>From [email-masked]  Fri Apr  4 19:12:22 2008
From: [email-masked] (Lachlan Andrew)
Date: Fri Apr  4 19:03:11 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Thanks for starting this thread, Matt.

FWIW, I tend to agree with Bob that locking "bandwidth fairness" into
the network is
(a) not necessarily the sort of fairness that applications need
and
(b) hard to reverse when we finally do see what applications need.

I agree that "TCP-friendliness" is overrated, but would be pushing the
Kelly/Briscoe line of charging users for the congestion they cause,
rather than arbitrarily limiting them to equality of a metric most
users don't care about (instantaneous bandwidth).

It may be sensible WFQ is used as an interim solution until we can get
the network-wide agreement necessary for charging.  However, if we
agree that charging is a better eventual solution, we should be
careful not to lock in design decisions which will make it harder
eventually.

$0.02,
Lachlan

On 04/04/2008, Dirceu Cavendish <[email-masked]> wrote:
>
> Zeroing into the bandwidth metric, a per flow queueing would give routers
> the power to share their link capacities however they find fit - WFQ,
> max-min fairness, etc - independent of other factors, such as sessions RTTs.
> In this scenario, and aggressive congestion control would cause its own
> traffic to be spilled, so "isolation" of sessions would be achieved.

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From: [email-masked] (Matt Mathis)
Date: Fri Apr  4 15:35:48 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Yes, I meant lockout and long queues.

The prescribed solutions in 2309 are not sufficient to help fairness.

Thanks,
--MM--
-------------------------------------------
Matt Mathis     http://staff.psc.edu/mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

On Fri, 4 Apr 2008, Bob Briscoe wrote:

> Matt,
>
> At 17:35 02/04/2008, Matt Mathis wrote:
>> * Autotuning will also greatly magnify RFC 2309 [1] problems, since every
>
> Just checking offlist that I understand you right, as "2309 problems" is 
> potentially ambiguous shorthand, but I think I know what you mean. Do you 
> mean the problems that 2309 is intended to solve (lock-out and long queues)?
>
> You might have meant further problems 2309 causes!?
>
>
> Bob
>
>
> ____________________________________________________________________________
> Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
> B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 
>

From: [email-masked] (Bob Briscoe)
Date: Fri Apr  4 15:06:56 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Matt,

At 17:35 02/04/2008, Matt Mathis wrote:
>* Autotuning will also greatly magnify RFC 2309 [1] problems, since every

Just checking offlist that I understand you right, as "2309 problems" 
is potentially ambiguous shorthand, but I think I know what you mean. 
Do you mean the problems that 2309 is intended to solve (lock-out and 
long queues)?

You might have meant further problems 2309 causes!?

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

From: [email-masked] (Bob Briscoe)
Date: Fri Apr  4 14:44:23 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Matt,

 From one heretic to another...

0) Priority to shortest jobs
In your bullet list you missed a point that I believe is the most 
important one: about arrivals and departures of whole flows. If you 
have a mix of flow activity at a shared bottleneck, some continuously 
streaming and some intermittent (as we do on the Internet), you can 
make the intermittent flows go much much faster without hardly 
prolonging the completion time of the continuous flows. It's totally 
unfair (and very inefficient) for the intermittent flows to get the 
same bit rate as the continuous flows, because their behaviour is 
much more multiplexable. A picture may help: 
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0801cfp/shortestJobsPriority.png>

If you're intrigued by how we'd move to such a world, I recently 
posted a bit more on this here: 
<http://episteme.arstechnica.com/eve/forums?a=tpc&s=50009562&f=174096756&m=703000231931&r=160007441931#160007441931>

Now thoughts on your main points:

1) Fair queuing
Many broadband deployments already do some form of per-flow FQ 
(usually WFQ) at the most likely bottleneck (the broadband remote 
access server, BRAS). I'm less sure what common practice is in 
broadband cellular networks. I believe WFQ is not such an obvious 
choice for radio access because of the unpredictable radio link rate. 
I think there are a range of schemes, some distributed at the base 
station and others centralised at the radio network controller (RNC).

Incidentally, back to the first point, I recently realised that most 
deployed per-flow FQ tends to help short duration flows, by giving 
higher priority at the start of a flow and reducing the longer each 
flow continues. Altho this helps a little bit today, it's also 
actually a huge potential problem for the future - same problem as 
TCP: it still converges to the incorrect goal of sharing out bit-rate 
equally for each flow at the bottleneck - tho at least at a BRAS it's 
done separately per each user.

My point about priority to shortest jobs (and your point about huge 
differences between no.s of flows per app) shows that flow rates need 
to be /very/ unequal to be fair. So per-flow FQ embedded in the 
network will be fighting what we really need to do in the transport 
in future (tho operators would obviously turn off per-flow FQ if they 
were trying to encourage the new world I'm talking about).

2) Edge bottlenecks protecting core
Although it would be nice, we can't mandate that the core be 
protected by bottlenecks at the edge. Technology limits & economics 
determine these things, not the IETF/IRTF:
* At present technology trends are moving bottlenecks gradually 
closer to the core as access rates increase, because the newest 
access is now using the same technology as the core (optics) and 
there's nothing faster on the horizon.
* However, economics always pushes the bottleneck location the other 
way - outwards. The cost of a bps of logical channel capacity follows 
about a square root (?) law wrt to the bandwidth of the physical pipe 
in which the logical channel sits. Ie. where you need many physical 
cables/fibres to cover a dispersed area, the cost per bps is much 
greater than where each bps can be provided within a few large pipes.

The net effect of these two opposing forces is pushing bottlenecks 
inwards at the moment - particularly onto border routers. The message 
of a talk I gave to a recent workshop (on photonics research and 
future Internet design) was that the challenge is to find ways to 
push complex trust-related stuff currently done at border routers 
outward to the edge, so we can have dumb all-optical interconnection 
without electronics: 
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0709ecoc-fid>

3) RTT fairness
I'd say this is only a small part of the problem, because it's 
relatively easy to solve in the transport alone - e.g. FAST TCP 
[Jin04:FAST_TCP] ensures its dynamics are slower for longer RTTs but, 
even tho it takes longer to get there, it ends up at the same rate as 
competing FAST TCPs with shorter RTTs.

Bob

[Jin04:FAST_TCP] Cheng Jin, David Wei and Steven Low "FAST TCP: 
Motivation, Architecture, Algorithms, Performance", In "Proc. IEEE 
Conference on Computer Communications (Infocomm'04)" (March, 2004)

At 17:35 02/04/2008, Matt Mathis wrote:
>I just attended the "The Future of TCP: Train-wreck or Evolution?" 
>at Stanford last week, and it solidified my thoughts on a subject 
>that is sure to be controversial.
>
>I think it is time to abandon the concept of "TCP-Friendly" and 
>instead expect the network to protect itself and other users from 
>aggressive protocols and applications.  For the moment I am going to 
>assume two mechanisms, although I suspect that there will prove to be more.
>
>1) Deploy some form of Fair Queuing at the edges of the network.
>
>2) Protect the core by bottlenecks at the edges of the Internet.
>
>I observe that both of these mechanisms are already being 
>implemented due to existing market forces, and the natural 
>consequence of their implementation is to make TCP-friendliness a 
>whole lot less important.  I admit that it is not clear at this 
>point if these two mechanisms will ultimately prove to be sufficient 
>to address fairness all situations, such as over loaded core 
>routers, but I suspect that sufficient mechanisms do exist.
>
>Supporting arguments:
>
>FQ is already being deployed at the edges to solve several existing 
>and growing fairness problems:
>
>* Non-IETF, UDP protocols that are non-responsive.
>
>* P2P and other applications that open huge numbers of connections.
>
>* Stock TCP is egregiously unfair when very short RTT flows compete with wide
>   area flows.  This can be a real killer in a number of settings such as data
>   centers and university campuses.  The symptoms of this problem will become
>   more pronounced as TCP autotuning continues to be rolled out in Vista,
>   Linux, and various BSDs.
>
>* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
>   single TCP flow with sufficient data will cause congestion somewhere in the
>   network.  At the very least this will gradually force the retirement of
>   drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
>   by itself is insufficient to solve the other fairness problems, it will not
>   be the first choice replacement.
>
>I should note that "Fair Queuing"is overly specific.  The network 
>needs to do something to large flows to prevent them from 
>overwhelming smaller flows and to limit queue occupancy.  FQ is one 
>way, but there are others.
>
>If you have ever shared a drop-tail home router with a teenager, you 
>might have observed some of these issues first hand, as has 
>Comcast.[2] As I understand it, some form of enforced fairness is 
>now part of all commercial broadband services.
>
>The core of the Internet is already mostly protected by bottlenecks 
>at the edges.  This is because ISPs can balance the allocation of 
>revenue from customers between the relatively expensive access link, 
>its own backbone links and interconnections to other ISPs.  Since 
>congestion in the core has proven to cause complaints from 
>commercial customers (and perhaps SLA problems), most providers are 
>careful to keep adequate capacity in the core, and can do so pretty 
>easily, as long as their duty cycle models hold true.
>
>Are these two mechanisms sufficient to make TCP-friendliness 
>completely moot? Probably not.
>
>We still have some work to do:
>
>First, stop whining about non-TCP-friendly protocols.  The are here 
>to stay and they can't hear us.  We are wasting our breath and 
>impeding real progress in well designed alternatives to 
>"TCP-friendly".  This concept came from an era when the Internet was 
>a gentleman's club, but now it needs to be retired.
>
>Second, blame the network when the network deserves it.  In 
>particular if there are drop tail queues without AQM, be very 
>suspicious of RFC2309 problems.  In fact every drop tail queue 
>without AQM should be viewed as a bug waiting to bite 
>someone.  Likewise remember that "TCP-friendly" is extremely unfair 
>when the RTTs are extremely different.
>
>Third, think about the hard cases: over loaded interconnects, 
>failure conditions, etc.  Can FQ be approximated at core 
>scales?  Where else are my proposed mechanisms insufficient?  I sure 
>there are some.
>
>Fourth, start dreaming about what it would take to make Moore's law 
>apply to end-to-end protocol performance, as it does to just about 
>everything else in the computing universe.  I suspect that in some 
>future hindsight, we will come to realize that TCP-friendly was 
>actually a untenable position, and has held us back from important innovations.
>
>[1] RFC2309 "Recommendations on Queue Management and Congestion 
>Avoidance in the Internet", Bob Braden, et al.
>
>[2] Richard Bennett "New and Improved Traffic Shaping" 
>http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/
>------------------
>
>It was a very stimulating conference!
>Thanks Nandita and everyone else who made it happen!
>--MM--
>-------------------------------------------
>Matt Mathis     http://staff.psc.edu/mathis
>Work:412.268.3319    Home/Cell:412.654.7529
>-------------------------------------------
>
>
>_______________________________________________
>Iccrg mailing list
>[email-masked]
>http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

From: [email-masked] (Dirceu Cavendish)
Date: Wed Apr  2 19:57:59 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

Matt,

First of all, that you very much for the "summary" of the event. For those like myself, who were not able to attend, this is valuable.

Boy, you touched a lot of pending issues. From the description, participants zeroed to the right points.
RTT fairness...It is a idealistic concept, but NOT real. Not today (with TCP bias towards short RTT sessions), not ever (UNLESS  we assume PER-session queueing in the routers, and dimension their buffers according to their respective RTT - see http://cavendish.homedns.org/ControlTheoreticalCC.pdf  under scheduling schemes, for a discussion on queueing disciplines and closed loop congestion control).

About network protection, I couldnt agree more. I have seen TCP "nice" proposals, where router buffers are conservatively kept low, impacting session throughput, that people argue about why would anyone use a TCP that steps back in face of more aggressive protocols, even legacy ones...

Dirceu

----- Original Message ----
From: Matt Mathis <[email-masked]>
To: [email-masked]
Sent: Wednesday, April 2, 2008 9:35:11 AM
Subject: [Iccrg] Heresy following "TCP: Train-wreck"

I just attended the "The Future of TCP: Train-wreck or Evolution?" at Stanford 
last week, and it solidified my thoughts on a subject that is sure to be 
controversial.

I think it is time to abandon the concept of "TCP-Friendly" and instead expect 
the network to protect itself and other users from aggressive protocols and 
applications.  For the moment I am going to assume two mechanisms, although I 
suspect that there will prove to be more.

1) Deploy some form of Fair Queuing at the edges of the network.

2) Protect the core by bottlenecks at the edges of the Internet.

I observe that both of these mechanisms are already being implemented due to 
existing market forces, and the natural consequence of their implementation is 
to make TCP-friendliness a whole lot less important.  I admit that it is not 
clear at this point if these two mechanisms will ultimately prove to be 
sufficient to address fairness all situations, such as over loaded core 
routers, but I suspect that sufficient mechanisms do exist.

Supporting arguments:

FQ is already being deployed at the edges to solve several existing and 
growing fairness problems:

* Non-IETF, UDP protocols that are non-responsive.

* P2P and other applications that open huge numbers of connections.

* Stock TCP is egregiously unfair when very short RTT flows compete with wide
  area flows.  This can be a real killer in a number of settings such as data
  centers and university campuses.  The symptoms of this problem will become
  more pronounced as TCP autotuning continues to be rolled out in Vista,
  Linux, and various BSDs.

* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
  single TCP flow with sufficient data will cause congestion somewhere in the
  network.  At the very least this will gradually force the retirement of
  drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
  by itself is insufficient to solve the other fairness problems, it will not
  be the first choice replacement.

I should note that "Fair Queuing"is overly specific.  The network needs to do 
something to large flows to prevent them from overwhelming smaller flows and 
to limit queue occupancy.  FQ is one way, but there are others.

If you have ever shared a drop-tail home router with a teenager, you might 
have observed some of these issues first hand, as has Comcast.[2] As I 
understand it, some form of enforced fairness is now part of all commercial 
broadband services.

The core of the Internet is already mostly protected by bottlenecks at the 
edges.  This is because ISPs can balance the allocation of revenue from 
customers between the relatively expensive access link, its own backbone links 
and interconnections to other ISPs.  Since congestion in the core has proven 
to cause complaints from commercial customers (and perhaps SLA problems), most 
providers are careful to keep adequate capacity in the core, and can do so 
pretty easily, as long as their duty cycle models hold true.

Are these two mechanisms sufficient to make TCP-friendliness completely moot? 
Probably not.

We still have some work to do:

First, stop whining about non-TCP-friendly protocols.  The are here to stay 
and they can't hear us.  We are wasting our breath and impeding real progress 
in well designed alternatives to "TCP-friendly".  This concept came from an 
era when the Internet was a gentleman's club, but now it needs to be retired.

Second, blame the network when the network deserves it.  In particular if 
there are drop tail queues without AQM, be very suspicious of RFC2309 
problems.  In fact every drop tail queue without AQM should be viewed as a bug 
waiting to bite someone.  Likewise remember that "TCP-friendly" is extremely 
unfair when the RTTs are extremely different.

Third, think about the hard cases: over loaded interconnects, failure 
conditions, etc.  Can FQ be approximated at core scales?  Where else are my 
proposed mechanisms insufficient?  I sure there are some.

Fourth, start dreaming about what it would take to make Moore's law apply to 
end-to-end protocol performance, as it does to just about everything else in 
the computing universe.  I suspect that in some future hindsight, we will come 
to realize that TCP-friendly was actually a untenable position, and has held 
us back from important innovations.

[1] RFC2309 "Recommendations on Queue Management and Congestion Avoidance in 
the Internet", Bob Braden, et al.

[2] Richard Bennett "New and Improved Traffic Shaping" 
http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/
------------------

It was a very stimulating conference!
Thanks Nandita and everyone else who made it happen!
--MM--
-------------------------------------------
Matt Mathis    http://staff.psc.edu/mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------

_______________________________________________
Iccrg mailing list
[email-masked]
http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080402/c37cba8b/attachment.html
>From [email-masked]  Wed Apr  2 20:50:27 2008
From: [email-masked] (Matt Mathis)
Date: Wed Apr  2 20:41:28 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On Wed, 2 Apr 2008, Dirceu Cavendish wrote:
> First of all, that you very much for the "summary" of the event. For those 
> like myself, who were not able to attend, this is valuable.

Oops... I should have made it clear that my message was NOT a summary of the 
meeting.  Most of the presentations touched on some aspect of my message, but 
by and large everyone was trying to stay inside the current TCP-friendly 
paradigm.

Sorry for any confusion.

Thanks,
--MM--

From: [email-masked] (Dirceu Cavendish)
Date: Tue Apr  1 00:33:59 2008
Subject: [Iccrg] SSDT Scope - summary update
Message-ID: <[email-masked]>

OK, I guess we have a short list of topics to work on in the next six months.

- Transaction latencies taking too long to complete, in a large bandwidth delay product networks.
- Buffer overshoot, causing multiple segment losses/retransmits, potentially causing RTOs
- Session returning to SS and starting from scratch due to "application constrained" scenario

If there aren't others, lets go to work:-)

Whoever is interested in contributing to the topics above, please join the SSDT mailing list
mailto:[email-masked] with "subscribe YOUR NAME" as the body of the message.

Dirceu

Looking for last minute shopping deals? Find them fast with Yahoo! Search.

      ____________________________________________________________________________________
Special deal for Yahoo! users & friends - No Cost. Get a month of Blockbuster Total Access now 
http://tc.deals.yahoo.com/tc/blockbuster/text3.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080331/a2ece34c/attachment.html
>From [email-masked]  Tue Apr  1 18:40:29 2008
From: [email-masked] (Dirceu Cavendish)
Date: Tue Apr  1 18:31:54 2008
Subject: [Iccrg] Re: SSDT Scope - summary update
Message-ID: <[email-masked]>

If the group agrees, I will be taking discussions and work on the topics listed below to the SSDT mailing list.

Rgds to all,
Dirceu

----- Original Message ----
From: Dirceu Cavendish <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: Monday, March 31, 2008 4:42:51 PM
Subject: SSDT Scope - summary update

OK, I guess we have a short list of topics to work on in the next six months.

- Transaction latencies taking too long to complete, in a large bandwidth delay product networks.
- Buffer overshoot, causing multiple segment losses/retransmits, potentially causing RTOs
- Session returning to SS and starting from scratch due to "application constrained" scenario
 
If there aren't others, lets go to work:-)
 
Whoever is interested in contributing to the topics above, please join the SSDT mailing list
mailto:[email-masked] with "subscribe YOUR NAME" as the body of the message.
 
Dirceu

Looking for last minute shopping deals? Find them fast with Yahoo! Search.

You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20080401/c1958c96/attachment.html
>From [email-masked]  Wed Apr  2 17:35:11 2008
From: [email-masked] (Matt Mathis)
Date: Wed Apr  2 17:26:38 2008
Subject: [Iccrg] Heresy following "TCP: Train-wreck"
Message-ID: <[email-masked]>

I just attended the "The Future of TCP: Train-wreck or Evolution?" at Stanford 
last week, and it solidified my thoughts on a subject that is sure to be 
controversial.

I think it is time to abandon the concept of "TCP-Friendly" and instead expect 
the network to protect itself and other users from aggressive protocols and 
applications.  For the moment I am going to assume two mechanisms, although I 
suspect that there will prove to be more.

1) Deploy some form of Fair Queuing at the edges of the network.

2) Protect the core by bottlenecks at the edges of the Internet.

I observe that both of these mechanisms are already being implemented due to 
existing market forces, and the natural consequence of their implementation is 
to make TCP-friendliness a whole lot less important.  I admit that it is not 
clear at this point if these two mechanisms will ultimately prove to be 
sufficient to address fairness all situations, such as over loaded core 
routers, but I suspect that sufficient mechanisms do exist.

Supporting arguments:

FQ is already being deployed at the edges to solve several existing and 
growing fairness problems:

* Non-IETF, UDP protocols that are non-responsive.

* P2P and other applications that open huge numbers of connections.

* Stock TCP is egregiously unfair when very short RTT flows compete with wide
   area flows.  This can be a real killer in a number of settings such as data
   centers and university campuses.  The symptoms of this problem will become
   more pronounced as TCP autotuning continues to be rolled out in Vista,
   Linux, and various BSDs.

* Autotuning will also greatly magnify RFC 2309 [1] problems, since every
   single TCP flow with sufficient data will cause congestion somewhere in the
   network.  At the very least this will gradually force the retirement of
   drop-tail equipment, creating the opportunity for RED and/or FQ.  Since RED
   by itself is insufficient to solve the other fairness problems, it will not
   be the first choice replacement.

I should note that "Fair Queuing"is overly specific.  The network needs to do 
something to large flows to prevent them from overwhelming smaller flows and 
to limit queue occupancy.  FQ is one way, but there are others.

If you have ever shared a drop-tail home router with a teenager, you might 
have observed some of these issues first hand, as has Comcast.[2] As I 
understand it, some form of enforced fairness is now part of all commercial 
broadband services.

The core of the Internet is already mostly protected by bottlenecks at the 
edges.  This is because ISPs can balance the allocation of revenue from 
customers between the relatively expensive access link, its own backbone links 
and interconnections to other ISPs.  Since congestion in the core has proven 
to cause complaints from commercial customers (and perhaps SLA problems), most 
providers are careful to keep adequate capacity in the core, and can do so 
pretty easily, as long as their duty cycle models hold true.

Are these two mechanisms sufficient to make TCP-friendliness completely moot? 
Probably not.

We still have some work to do:

First, stop whining about non-TCP-friendly protocols.  The are here to stay 
and they can't hear us.  We are wasting our breath and impeding real progress 
in well designed alternatives to "TCP-friendly".  This concept came from an 
era when the Internet was a gentleman's club, but now it needs to be retired.

Second, blame the network when the network deserves it.  In particular if 
there are drop tail queues without AQM, be very suspicious of RFC2309 
problems.  In fact every drop tail queue without AQM should be viewed as a bug 
waiting to bite someone.  Likewise remember that "TCP-friendly" is extremely 
unfair when the RTTs are extremely different.

Third, think about the hard cases: over loaded interconnects, failure 
conditions, etc.  Can FQ be approximated at core scales?  Where else are my 
proposed mechanisms insufficient?  I sure there are some.

Fourth, start dreaming about what it would take to make Moore's law apply to 
end-to-end protocol performance, as it does to just about everything else in 
the computing universe.  I suspect that in some future hindsight, we will come 
to realize that TCP-friendly was actually a untenable position, and has held 
us back from important innovations.

[1] RFC2309 "Recommendations on Queue Management and Congestion Avoidance in 
the Internet", Bob Braden, et al.

[2] Richard Bennett "New and Improved Traffic Shaping" 
http://bennett.com/blog/index.php/archives/2008/03/27/new-and-improved-traffic-shaping/
------------------

It was a very stimulating conference!
Thanks Nandita and everyone else who made it happen!
--MM--
-------------------------------------------
Matt Mathis     http://staff.psc.edu/mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------