Subject: Re: [tcpm] tcpm Digest, Vol 79, Issue 24

From: Idris Rai <[email-masked]>
Date: Tue, 16 Nov 2010 20:18:36 +0300

The results presented by Jerry in Beijing for IW10 show an improved response time and  increase in loss rate at low speed links --  but the buffer used was too large (about double the DBP).

Can we get the same results at lower, more acceptable buffer sizes??. Not intuitive at all to me. Perhaps this is one experiment we need to do next 
 
Idris

Sent from my iPhone

On 16 Nov 2010, at 19:41, [email-masked] wrote:

> If you have received this digest without all the individual message
> attachments you will need to update your digest options in your list
> subscription.  To do so, go to 
> 
> https://www.ietf.org/mailman/listinfo/tcpm
> 
> Click the 'Unsubscribe or edit options' button, log in, and set "Get
> MIME or Plain Text Digests?" to MIME.  You can set this option
> globally for all the list digests you receive at this point.
> 
> 
> 
> Send tcpm mailing list submissions to
>    [email-masked]
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>    https://www.ietf.org/mailman/listinfo/tcpm
> or, via email, send a message with subject or body 'help' to
>    [email-masked]
> 
> You can reach the person managing the list at
>    [email-masked]
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of tcpm digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: WG status update (Lars Eggert)
>   2. I-D Action:draft-ietf-tcpm-tcp-timestamps-01.txt
>      ([email-masked])
>   3. Re: [Tmrg]  Increasing the Initial Window - Notes
>      ( Ilpo J?rvinen )
>   4. Re: [Tmrg]  Increasing the Initial Window - Notes (Fred Baker)
>   5. Re: [Tmrg]  Increasing the Initial Window - Notes
>      (Scheffenegger, Richard)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Tue, 16 Nov 2010 13:48:12 +0200
> From: Lars Eggert <[email-masked]>
> Subject: Re: [tcpm] WG status update
> To: [email-masked]
> Cc: [email-masked]
> Message-ID: <[email-masked]>
> Content-Type: text/plain; charset="us-ascii"
> 
> On 2010-11-15, at 17:39, [email-masked] wrote:
>>> But since we are doing MPTCP, I simply don't see the point of specifying a
>>> subset of it in a way that is incompatible.
>> 
>> Lars, are you speaking as an individual contributor or as Area director here?
> 
> Individual. (Thanks for reminding me to make this clear.)
> 
> Lars
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: smime.p7s
> Type: application/pkcs7-signature
> Size: 4367 bytes
> Desc: not available
> Url : <http://www.ietf.org/mail-archive/web/tcpm/attachments/20101116/43abe0cb/attachment.p7s>
> 
> ------------------------------
> 
> Message: 2
> Date: Tue, 16 Nov 2010 04:45:01 -0800
> From: [email-masked]
> Subject: [tcpm] I-D Action:draft-ietf-tcpm-tcp-timestamps-01.txt
> To: [email-masked]
> Cc: [email-masked]
> Message-ID: <20101116124501.2400.2831.idtracker@localhost>
> Content-Type: text/plain; charset="us-ascii"
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Tue, 16 Nov 2010 16:01:19 +0200 (EET)
> From: " Ilpo J?rvinen " <[email-masked]>
> Subject: Re: [tcpm] [Tmrg]  Increasing the Initial Window - Notes
> To: Fred Baker <[email-masked]>
> Cc: tcpm <[email-masked]>, tmrg <[email-masked]>
> Message-ID:
>    <[email-masked]>
> Content-Type: text/plain; charset=us-ascii
> 
> On Tue, 16 Nov 2010, Fred Baker wrote:
> 
>> On Nov 11, 2010, at 7:48 AM, Jerry Chu wrote:
>> 
>>> I wonder what factors will argue that the initial quantum can't be changed?
>>> Someone has argued in the past IW10 effectively "disable CC algorithm".
>>> But haven't we already done that years ago when moving up IW from 1 to 3 if
>>> one insisted then that the initial quantum must be 1?
>> 
>> Actually, no. I may be the person who said that this effectively 
>> disables congestion control in the average case; my reason for saying 
>> that is that per what little data I have, the average TCP transfer is on 
>> the order of 15K bytes, which is about ten 1460 byte segments. 
> 
> It only "disables" it if there was no congestion to control in the first 
> place.
> 
>> Now, the median is much smaller - on the order of two or three data 
>> segments each direction, which is where you get your comment. But the 
>> fact that browsers, Bittorrent, and other applications open multiple 
>> sessions simultaneously argues that a given endpoint having multiple 
>> packets in flight simultaneously doesn't hurt, and research done at the 
>> time by Mark Allman and others showed that IW=2..4 wasn't a problem and 
>> did in fact reduce the median session to a single RTT for data. But the 
>> median session doesn't create a congestion problem.
>> 
>> The question I have been asking is how IW=10 affects parallel sessions 
>> on relatively low speed links.
> 
> Could you on your behalf answer how does IW=3 affect parallel sessions on 
> the same kind of environment? ...I've done research on this (and Jerry has 
> too) and I know that the answer is not an intuitive one. Then we changed 
> to IW10 in the same environment and observe no (consistent) change. The 
> results are driven by problems already with IW3 so IW10 won't make it any 
> worse or the effect is just too small to be measured among the other 
> troubles that happen.
> 
>> My sense is that the sudden arrival of 
>> O(15K) bytes at a 56 KBPS line, which is to say the insertion of a tad 
>> over two seconds delay in all competing sessions plus an increase in 
>> their probability of loss, is likely to cause them all to be disrupted. 
> 
> Again, the existing research points to exactly _opposite_ thing to 
> happen, ie., IW10 hurts mostly itself, not the others! We'll never get 
> anywhere if these same "fuzzy feelings" (like Mark put it) just get 
> repeated and the existing _evidence_ overlooked. Jerry has tested a case 
> that is very close to what you're specifying and his results don't agree 
> what you think would happen. My results also confirm Jerry's observations 
> for the low end links though in my case it was a bit higher bw already 
> (160kbps).
> 
> I personally don't anymore believe we will find a convincing case against 
> IW10 from the very lowest end of links, afaict, it doesn't help but 
> doesn't really hurt significantly either. ...I understand this is quite 
> non-intuitive claim but it is what the performed studies seem to point 
> out (and I expect new guys every now and then to jump in with this same 
> argument if IW10 is to proceed). ...I'm more concerned about mid-ranged 
> links where it could be possible (at least in theory if AQM in use) to 
> have both TCP and low-latency requiring audio/video in parallel. 
> 
>> You told me verbally that in your tests it is net beneficial to the 
>> parallel sessions, but in thought experiments I'm not sure how that can 
>> be true; I'd be fascinated to see the data that shows that competing 
>> voice/video sees no frame loss or induced noise, that competing file 
>> transfers slow down but do not see redundant retransmissions, and so on.
> 
> In this I somewhat agree with you. The delay increase was confirmed by my 
> measurements, however, I think you also end up too cleverly mixing 
> IW3+IW10 and voice/video cases here. It seems that even if delay increases 
> and as a result voice/video could possible experience more losses (I'm yet 
> to confirm that losses actually happen as I didn't have actual CBR traffic 
> running), IW3 TCP in mixed case is not affected in a similar way. Instead, 
> IW3 TCP might actually get a boost in its performance against IW10 because 
> also IW10 TCP needs to respond to the congestion it caused.
> 
> Also, regardless of IW, TCP is well capable of leaving audio/video in 
> ruins as it will fill all buffers you give to it, so I find this exercise 
> quite pointless anyway unless there is some form of AQM in use.
> 
>> So, IW=2..4 demonstrably works pretty well everywhere, but IW=10 seems 
>> to me only sensible on MPBS and GBPS, not KBPS, scale links. 
> 
> IW3 doesn't work that well if you have parallel connections over 56kbps 
> but since the latencies are high anyway, you just won't notice that.
> 
> 
> -- 
> i.
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Tue, 16 Nov 2010 22:09:38 +0800
> From: Fred Baker <[email-masked]>
> Subject: Re: [tcpm] [Tmrg]  Increasing the Initial Window - Notes
> To: "Ilpo J?rvinen" <[email-masked]>
> Cc: tcpm <[email-masked]>, tmrg <[email-masked]>
> Message-ID: <[email-masked]>
> Content-Type: text/plain; charset=iso-8859-1
> 
> 
> On Nov 16, 2010, at 10:01 PM, Ilpo J?rvinen wrote:
> 
>> On Tue, 16 Nov 2010, Fred Baker wrote:
>> 
>>> On Nov 11, 2010, at 7:48 AM, Jerry Chu wrote:
>>> 
>>>> I wonder what factors will argue that the initial quantum can't be changed?
>>>> Someone has argued in the past IW10 effectively "disable CC algorithm".
>>>> But haven't we already done that years ago when moving up IW from 1 to 3 if
>>>> one insisted then that the initial quantum must be 1?
>>> 
>>> Actually, no. I may be the person who said that this effectively 
>>> disables congestion control in the average case; my reason for saying 
>>> that is that per what little data I have, the average TCP transfer is on 
>>> the order of 15K bytes, which is about ten 1460 byte segments. 
>> 
>> It only "disables" it if there was no congestion to control in the first 
>> place.
> 
> How do you turn off something that's not on?
> 
> I refer to this as in the average case turning off congestion control because the transmission requires ten data packets and they all get sent in a single burst. There is no congestion control, with IW=10, in the case of the average-sized (which is actually gar above the median-sized) file transfer.
> 
>> Could you on your behalf answer how does IW=3 affect parallel sessions on 
>> the same kind of environment? ...I've done research on this (and Jerry has 
>> too) and I know that the answer is not an intuitive one. Then we changed 
>> to IW10 in the same environment and observe no (consistent) change. The 
>> results are driven by problems already with IW3 so IW10 won't make it any 
>> worse or the effect is just too small to be measured among the other 
>> troubles that happen.
> 
> That may be true. I have asked to see data describing the impact. I haven't seen the data. Maybe I missed something?
> 
> ------------------------------
> 
> Message: 5
> Date: Tue, 16 Nov 2010 16:41:50 -0000
> From: "Scheffenegger, Richard" <[email-masked]>
> Subject: Re: [tcpm] [Tmrg]  Increasing the Initial Window - Notes
> To: "Fred Baker" <[email-masked]>, "Ilpo J?rvinen"
>    <[email-masked]>
> Cc: tcpm <[email-masked]>, tmrg <[email-masked]>
> Message-ID:
>    <[email-masked]>
> Content-Type: text/plain;    charset="iso-8859-1"
> 
> 
> Group,
> 
> Just one additional data point: Obviously, there are already applications in the wild, that use persistent connections and disable the slow-start-restart mechanism on these sessions. The expectations of the programmers making use of these socket options is to reduce the latency for their appliaction... In effect, these Apps seem use IW=44 (and hurting themselves more than anybody else).
> 
> See "Profiling Network Performance in Multi-Tier Data Center Applications", Section 6.1, "cwnd failing to prevent sudden bursts", http://www.cs.princeton.edu/~jrex/papers/snap10.pdf
> 
> 
> IMHO this is evidence that some action in this space is absolutely needed (even if it is "only" strong advice to apps programmers), or there will be even more proliferation of clumsy workarounds that are even less well understood and investigated than IW10 already is. Keeping IW3 "forever" may be more problematic than addressing emerging application needs with well-tested transport-layer approaches...
> 
> 
> Richard Scheffenegger
> 
> 
>> -----Original Message-----
>> From: Fred Baker [mailto:[email-masked]] 
>> Sent: Dienstag, 16. November 2010 15:10
>> To: "Ilpo J?rvinen"
>> Cc: tcpm; tmrg
>> Subject: Re: [tcpm] [Tmrg] Increasing the Initial Window - Notes
>> 
>> 
>> On Nov 16, 2010, at 10:01 PM, Ilpo J?rvinen wrote:
>> 
>>> On Tue, 16 Nov 2010, Fred Baker wrote:
>>> 
>>>> On Nov 11, 2010, at 7:48 AM, Jerry Chu wrote:
>>>> 
>>>>> I wonder what factors will argue that the initial quantum 
>> can't be changed?
>>>>> Someone has argued in the past IW10 effectively "disable 
>> CC algorithm".
>>>>> But haven't we already done that years ago when moving up 
>> IW from 1 
>>>>> to 3 if one insisted then that the initial quantum must be 1?
>>>> 
>>>> Actually, no. I may be the person who said that this effectively 
>>>> disables congestion control in the average case; my reason 
>> for saying 
>>>> that is that per what little data I have, the average TCP 
>> transfer is 
>>>> on the order of 15K bytes, which is about ten 1460 byte segments.
>>> 
>>> It only "disables" it if there was no congestion to control in the 
>>> first place.
>> 
>> How do you turn off something that's not on?
>> 
>> I refer to this as in the average case turning off congestion 
>> control because the transmission requires ten data packets 
>> and they all get sent in a single burst. There is no 
>> congestion control, with IW=10, in the case of the 
>> average-sized (which is actually gar above the median-sized) 
>> file transfer.
>> 
>>> Could you on your behalf answer how does IW=3 affect 
>> parallel sessions 
>>> on the same kind of environment? ...I've done research on this (and 
>>> Jerry has
>>> too) and I know that the answer is not an intuitive one. Then we 
>>> changed to IW10 in the same environment and observe no (consistent) 
>>> change. The results are driven by problems already with IW3 so IW10 
>>> won't make it any worse or the effect is just too small to 
>> be measured 
>>> among the other troubles that happen.
>> 
>> That may be true. I have asked to see data describing the 
>> impact. I haven't seen the data. Maybe I missed something?
>> _______________________________________________
>> tcpm mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/tcpm
>> 
> 
> 
> ------------------------------
> 
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
> 
> 
> End of tcpm Digest, Vol 79, Issue 24
> ************************************

---

From: Murari Sridharan <[email-masked]>
Date: Tue, 16 Nov 2010 19:08:47 +0000

The buffers in these low speed modems are in fact quite large as we have seen in Ledbat. In my home DSL I can build up latencies into the seconds by doing sustained uploads. There is a significant amount of buffering much larger than BDP. 

Thanks
Murari 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Idris Rai
Sent: Tuesday, November 16, 2010 9:19 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [tcpm] tcpm Digest, Vol 79, Issue 24

 
Idris

Sent from my iPhone

On 16 Nov 2010, at 19:41, [email-masked] wrote:

>>> But since we are doing MPTCP, I simply don't see the point of 
>>> specifying a subset of it in a way that is incompatible.
>> 
> -------------- next part -------------- A non-text attachment was 
> scrubbed...
> Name: smime.p7s
> Url : 
> <http://www.ietf.org/mail-archive/web/tcpm/attachments/20101116/43abe0
> cb/attachment.p7s>
> 
>>> But haven't we already done that years ago when moving up IW from 1 
>> that is that per what little data I have, the average TCP transfer is 
>> packets in flight simultaneously doesn't hurt, and research done at 
>> the time by Mark Allman and others showed that IW=2..4 wasn't a 
>> problem and did in fact reduce the median session to a single RTT for 
>> data. But the median session doesn't create a congestion problem.
>> 
>> The question I have been asking is how IW=10 affects parallel 
>> sessions on relatively low speed links.
> 
> Could you on your behalf answer how does IW=3 affect parallel sessions 
> won't make it any worse or the effect is just too small to be measured 
>> O(15K) bytes at a 56 KBPS line, which is to say the insertion of a 
>> tad over two seconds delay in all competing sessions plus an increase 
>> in their probability of loss, is likely to cause them all to be disrupted.
> 
> repeated and the existing _evidence_ overlooked. Jerry has tested a 
> case that is very close to what you're specifying and his results 
> don't agree what you think would happen. My results also confirm 
> Jerry's observations for the low end links though in my case it was a 
> bit higher bw already (160kbps).
> 
> doesn't really hurt significantly either. ...I understand this is 
> quite non-intuitive claim but it is what the performed studies seem to 
> point out (and I expect new guys every now and then to jump in with 
> this same argument if IW10 is to proceed). ...I'm more concerned about 
> mid-ranged links where it could be possible (at least in theory if AQM 
> in use) to have both TCP and low-latency requiring audio/video in parallel.
> 
>> parallel sessions, but in thought experiments I'm not sure how that 
>> can be true; I'd be fascinated to see the data that shows that 
>> competing voice/video sees no frame loss or induced noise, that 
>> competing file transfers slow down but do not see redundant retransmissions, and so on.
> 
> In this I somewhat agree with you. The delay increase was confirmed by 
> my measurements, however, I think you also end up too cleverly mixing
> IW3+IW10 and voice/video cases here. It seems that even if delay 
> IW3+increases
> and as a result voice/video could possible experience more losses (I'm 
> yet to confirm that losses actually happen as I didn't have actual CBR 
> traffic running), IW3 TCP in mixed case is not affected in a similar 
> way. Instead,
> IW3 TCP might actually get a boost in its performance against IW10 
> because also IW10 TCP needs to respond to the congestion it caused.
> 
> ruins as it will fill all buffers you give to it, so I find this 
> exercise quite pointless anyway unless there is some form of AQM in use.
> 
>> So, IW=2..4 demonstrably works pretty well everywhere, but IW=10 
>> seems to me only sensible on MPBS and GBPS, not KBPS, scale links.
> 
> IW3 doesn't work that well if you have parallel connections over 
> 56kbps but since the latencies are high anyway, you just won't notice that.
> 
> 
> --
> i.
> 
> 
>>>> But haven't we already done that years ago when moving up IW from 1 
>>> disables congestion control in the average case; my reason for 
>>> saying that is that per what little data I have, the average TCP 
>>> transfer is on the order of 15K bytes, which is about ten 1460 byte segments.
>> 
>> Could you on your behalf answer how does IW=3 affect parallel 
>> sessions on the same kind of environment? ...I've done research on 
>> this (and Jerry has
>> won't make it any worse or the effect is just too small to be 
>> measured among the other troubles that happen.
> 
> See "Profiling Network Performance in Multi-Tier Data Center 
> Applications", Section 6.1, "cwnd failing to prevent sudden bursts", 
> http://www.cs.princeton.edu/~jrex/papers/snap10.pdf
> 
> 
>> I refer to this as in the average case turning off congestion control 
>> because the transmission requires ten data packets and they all get 
>> sent in a single burst. There is no congestion control, with IW=10, 
>> in the case of the average-sized (which is actually gar above the 
>> median-sized) file transfer.
>> 
>> That may be true. I have asked to see data describing the impact. I 
>> haven't seen the data. Maybe I missed something?

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 16 Nov 2010 13:41:46 -0800

There is also a paper that studies this topic in details:

Characterizing residential broadband networks, IMC 2007
http://portal.acm.org/citation.cfm?id=1298306.1298313

Section 4.4
[...]
All ISPs deploy queues that are several times larger
than their bandwidth-delay products. Whereas downstream
queues can delay packets by more than 100 ms, the upstream
queueing delays can exceed several hundreds of milliseconds
and, at times, a few seconds.

Jerry picked the configuration based on other member's suggestions and also
these studies to better emulate today's residential network setup.

On Tue, Nov 16, 2010 at 11:08 AM, Murari Sridharan <[email-masked]>wrote:

> The buffers in these low speed modems are in fact quite large as we have
> seen in Ledbat. In my home DSL I can build up latencies into the seconds by
> doing sustained uploads. There is a significant amount of buffering much
> larger than BDP.
>
> Thanks
> Murari
>
> The results presented by Jerry in Beijing for IW10 show an improved
> response time and  increase in loss rate at low speed links --  but the
> buffer used was too large (about double the DBP).
>
> Can we get the same results at lower, more acceptable buffer sizes??. Not
> intuitive at all to me. Perhaps this is one experiment we need to do next
>
> Idris
>
> Sent from my iPhone
>
> because the transmission requires ten data packets and they all get sent in
> a single burst. There is no congestion control, with IW=10, in the case of
> the average-sized (which is actually gar above the median-sized) file
> transfer.
> >
> >    <[email-masked]
> >
> > Just one additional data point: Obviously, there are already applications
> in the wild, that use persistent connections and disable the
> slow-start-restart mechanism on these sessions. The expectations of the
> programmers making use of these socket options is to reduce the latency for
> their appliaction... In effect, these Apps seem use IW=44 (and hurting
> themselves more than anybody else).
> >
> > IMHO this is evidence that some action in this space is absolutely needed
> (even if it is "only" strong advice to apps programmers), or there will be
> even more proliferation of clumsy workarounds that are even less well
> understood and investigated than IW10 already is. Keeping IW3 "forever" may
> be more problematic than addressing emerging application needs with
> well-tested transport-layer approaches...
> >
> >
> > Richard Scheffenegger
> >
> >