Subject: Re: [Tsvwg] taking NewReno (RFC 2582) to Proposed Standard

From: Kacheong Poon <[email-masked]>
Date: Wed, 14 May 2003 02:25:18 -0500 (CDT)

Included message from "Armando L. Caro Jr." <[email-masked]>:

>----
>Anyway, is there a reason to leave the two burst limitation methods in the
>draft? One method is to cap the cwnd after Fast Recover only by setting
>cwnd = min (ssthresh, FlightSize + SMSS). The other method is to apply a
>limit (maxburst) on "the number of data packets that can be sent in
>response to a single acknowledgement".  Why not combine the two (as done
>in the SCTP implementer's guide) into a simple algorithm as follows?
>
>if((flightsize + Max.Burst*SMSS) < cwnd)
>         cwnd = flightsize + Max.Burst*SMSS
>----

I believe step 5 in section 3 is for setting the cwnd so that
after Fast Recovery, the sending rate is at most half of the
previous one.  If the above formula is used, cwnd will be wrong.
For example, if flightsize happens to be 0 (no new TCP segment
is sent during Fast Revoery), cwnd will be Max.Burst * SMSS
regardless of what the previous sending rate is.  But using
step 5 in section 3, it will be just 1 SMSS or ssthresh, which
is the intended value.  The note in the draft says that if
sshthresh is used, maxburst_ should probably be there to avoid
burstiness.  They are for different purposes.

I believe the reason why you don't see these written in RFC 2960
and the implementer's guide draft is just that the algorithm
is described differently.  Instead of "manipulating" cwnd to
force sending of new segments in TCP, RFC 2960 just says send
new data if allowed by cwnd, and calculation of cwnd already
takes the above into account.  Different wording, same idea...

							K. Poon.

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Sally Floyd <[email-masked]>
Date: Wed, 14 May 2003 16:57:06 -0700

>Could you send me the viewgraph of the trace you looked at?  I'm
>interested to see what the effects you mentioned are.  Thanks.

Brian Tierney reported terrible performance for TCP, with or without
HighSpeed TCP, at the First International Workshop on Protocols for
Fast Long-Distance Networks
("http://datatag.web.cern.ch/datatag/pfldnet2003/").  You can find
his viewgraphs there.

Matt Mathis and gang instrumented Web100 to report those congestion
window reductions due to "OtherReductionsCM", and Brian determined
that these "OtherReductionsCM" reductions were the cause of the
poor performance that he was seeing.

There were two places in the Linux code where congestion window
reductions were triggered that were reported by Web100 as
"OtherReductionsCM".  One is the tcp_moderate_cwnd() call in
tcp_input.c, as follows:

/* CWND moderation, preventing bursts due to too big ACKs 
 * in dubious situations.
 */
static __inline__ void tcp_moderate_cwnd(struct tcp_opt *tp)
{
        tp->snd_cwnd = min(tp->snd_cwnd,
                           tcp_packets_in_flight(tp)+tcp_max_burst(tp));
        tp->snd_cwnd_stamp = tcp_time_stamp;
}

In my opinion, this is a serious bad idea if one is trying to run
TCP in a high-delay, high-bandwidth environment, where large congestion
windows are needed to take advantage of the available bandwidth.

- Sally
http://www.icir.org/floyd/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Sally Floyd <[email-masked]>
Date: Mon, 19 May 2003 12:46:00 -0700

>> From: Armando L. Caro Jr. [mailto:[email-masked]]
>>
>> Yes, agreed. I think the right thing to do is to apply max 
>> burst always.
>> No matter what, each ack may only trigger a send of at most Max.Burst
>> packets.
>> 
>
>Does such a recommendation, if agreed, belong in a NewReno specific document?

I agree that maxburst might be useful, but that a discussion of the
general use of maxburst definitely doesn't belong in the NewReno document.

Some observations on maxburst: 

Maxburst has been in the NS simulator forever, in the form of
limiting how much is sent in response to a single ACK, not in the
form of limiting the congestion window.  Maxburst is turned off by
default in NS.

Maxburst is mostly useful for a path where the bottleneck queues
use Drop-Tail, and therefore are sensitive to very small bursts in
the sending rate.  In a network where the bottleneck queues used
some form of Active Queue Management, and therefore were properly
robust to small-scale burstiness, I would expect that some TCP
connections would perform better without maxburst.  In the long
run, one would like queues to perform robustly in the presence of 
small bursts.

Maxburst is in some sense a cheap, coarse-grained version of
rate-based pacing.  And probably could generate the same set of
papers detailing the cases where it is of benefit, and the cases
where it isn't.

For rate-based pacing, the papers include:

Amit Aggarwal, Stefan Savage, and Tom Anderson,
Understanding the Performance of TCP Pacing,
Proceedings of the 2000 IEEE INFOCOM Conference, Tel-Aviv, Israel, March 2000.

(I am not trying to denigrate maxburst, I think it might have been my
idea, many years ago.)

- Sally

---

From: Kacheong Poon <[email-masked]>
Date: Mon, 19 May 2003 20:03:21 -0500 (CDT)

Included message from "Henderson, Thomas R" <[email-masked]>:

>----
>> -----Original Message-----

It seems to me that this does not belong to the NewReno document
as it is not specific to NewReno.  But as a data point, Solaris'
TCP apply some form of max burst always for many years.  And I
have not heard any complaints.  So I guess it is OK.

>Another issue that may or may not be NewReno specific is whether it is a good
>idea to recommend suppressing the sending of pure window updates intermingled
>with dupacks, to avoid resetting the counter to enter fast retransmit (Jeffrey
>Hsu mentioned this to me earlier this week).  This probably is not an issue
>when SACK is used.  

Does this happen often?  When this happens, does the opening up
of the window also trigger new segments to be sent?  And is the
effect really bad?

							K. Poon.

---

From: "Henderson, Thomas R" <[email-masked]>
Date: Mon, 19 May 2003 21:11:24 -0700

> From: Kacheong Poon [mailto:[email-masked]]
> Sent: Monday, May 19, 2003 6:03 PM
> To: [email-masked]
> Subject: RE: [Tsvwg] taking NewReno (RFC 2582) to Proposed Standard
> 
> 

I don't know how often it happens.  However, it could be tricky
to intelligently suppress these, because if the sender is not yet 
in Fast Recovery, it would be better to suppress them, but if the
sender is in Fast Recovery, you would want to send pure window
updates so that true dupacks cause a new segment to be sent off
the top of the window (in some recent testing, I observed
cases in which pure window updates were not sent by the receiver,
leading to dupacks being interpreted by the sender (in Fast 
Recovery) as being window updates, causing FlightSize to be
much less than cwnd upon leaving Fast Recovery, causing a burst
of data, leading to thinking about maxburst...)

Another way of looking at this is that if window updates are 
sometimes suppressed, how does the sender know when
a dupack arrives with a different window than the previous, whether
to interpret it as a dupack triggered by an out-of-order segment
or whether it is a window update?  And then there is the issue of
dealing with heterogeneous (legacy) implementations if a change
is made.  

Tom

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 22 May 2003 01:18:39 -0500 (CDT)

---

From: Kacheong Poon <[email-masked]>
Date: Fri, 23 May 2003 00:41:12 -0500 (CDT)

Included message from Pasi Sarolahti <[email-masked]>:

>----
>But in the end, I think it is mostly an implementation issue to choose
>the most suitable variant of the actual burst avoidance algorithm. As
>long as the TCP congestion control RFC requires burst avoidance and
>describes what is meant by burst avoidance either by using (1) or (3),
>it should be ok, right?
>----

It should be OK as long as they really behave similarly.
But I guess scheme (3) so far has not gone through any
real implementation or simulation tests.  As Sally pointed
out, (1) has been in NS for many many years.  And it has
been in use in real Internet for many many years.  So I
really see no ground in saying that (1) is bad based on
pure opinion...  Well, just my $0.02...

							K. Poon.

---

From: Kacheong Poon <[email-masked]>
Date: Fri, 23 May 2003 01:24:49 -0500 (CDT)

Included message from Mark Allman <[email-masked]>:

>----
>The difference in (1) and (3) comes in the actual specification and
>implementation.  I think it is much cleaner to use scheme (3).  And,
>I do not think we're going to agree on this point.

Let's drop it then (-:

>Actually, I have had a nagging feeling about this... and, my brain
>has just clicked.  I don't think there is a problem with 3042.  If
>there is an app pause that causes cwnd > outstanding_data (and,
>therefore, for the test in 3042 to look bogus) then LT will not kick
>in anyway because TCP is app limited and there is no new data to
>send.  
>
>But, if outstanding_data < cwnd because of maxburst, this is not the
>case -- rendering the LT spec to be bogus.  But, that is sort of my
>point -- it would be bogus because of the maxburst retrofit.

I think Mark is correct.

>The we should also seem some study of maxburst, no?

Since Mark is so against maxburst, I assume that he will
write some papers describing its problems, as foreseen in

							K. Poon.