Subject: Re: [Tsvwg] (SCTP) Congestion window calculation and bundling

From: Stephane Chazelas <[email-masked]>
Date: Mon, 4 Dec 2006 11:23:39 +0000

On Wed, Nov 29, 2006 at 11:27:33AM -0500, Randall Stewart wrote:
> Stephane Chazelas wrote:
> >On Mon, Nov 20, 2006 at 07:42:29PM -0000, Brian F. G. Bidulock wrote:
> 
> >I was wrong about BSD and OpenSS7. BSD seems to use the first
> >method, and the code that uses max_burst in OpenSS7 seems to be
> >commented out. But you may know that better than me, I was
> >looking at strsctp-0.9.2.5
> >Do we know why max_burst didn't make its way to RFC 2960bis?
> 
> BSD allows a compile option to do it the opposite way. aka counting.
> The default is by adjusting the cwnd.
> 
> This whole topic was discussed in length on this list... Mark Allman
> was the proponent of cwnd based adjustment.. Kacheong Poon was the
> proponent of the counting method. Mark, if I remember right, was
> concerned about having two control variables and their interaction.
> Kacheong claimed that Solaris has done counting for years for TCP
> with no harm.. thats what I remember of the debate.. without a search
> of the archives :-)
> 
> As to why its not there.. I will go look.. it was obviously an
> oversite/cut/paste error..
> 
> I first though, want to get through this long thread :-D
[...]

Hi Randall,

I think there are several things wrong with that text.

 | D) When the time comes for the sender to transmit new DATA chunks,
 |    the protocol parameter Max.Burst SHOULD be used to limit the
 |    number of packets sent.  The limit MAY be applied by adjusting
 |    cwnd as follows:
 |
 |[METHOD1]    if((flightsize + Max.Burst*MTU) < cwnd)
 |       cwnd = flightsize + Max.Burst*MTU
 |
 |[METHOD2]    Or it MAY be applied by strictly limiting the number of packets
 |    emitted by the output routine.

1- It doesn't define what "when the time comes to transmit" new
DATA means.  From my understanding, if the sender does a for(;;)
send(fd, buf, 4*MTU); then that text will have no effect if
using [METHOD2].

Typical example:

hostA                       hostB
1-  HTTP GET /index.html  ->
2-  <- sending file (100MB), hostB's cwnd reaches 10000 MTUs
3-  hostA parses the HTML file, hostB's inflight is 0, cwnd
    unchanged. hostA requests the images in separate streams to
    HostB.
4-  HTTP GET GET GET GET  ->
5-  HostB user:
       while(n = read(file.jpg, buf, 4096))
	 send(sctp_fd, buf, n);
6-  HostB's SCTP stack will send 4 packets for each send as long
    as inflight < 10000*MTU. The maxburst limit will not apply
    as we only send 4 packets at a time.

about [METHOD1]
 if((flightsize + Max.Burst*MTU) < cwnd)
   cwnd = flightsize + Max.Burst*MTU

2- as the DATA header is not taken into account, it doesn't
prevent to send a lot of fullsize packets (up to about 60 for
a 1500 byte MTU). But, the problem is not only with max burst.

3- [METHOD1 again] it contradicts the statement that says that
the cwnd should be halved every RTO when nothing is sent. Here
it is reduced to its minimum after 1 RTT.

4- [METHOD1] it doesn't tell what to do with ssthresh. I think
it should be set to at least cwnd/2 if it was below that,
otherwise you may end up in congestion avoidance and it will not
be equivalent to the other solution. (maybe the same should be
said for halving of cwnd when not sending).

5- when sending fullsize packets, both algorithm should be more
or less equivalent to doing a slowstart (assuming ssthresh is
made greater than the updated cwnd in METHOD1), that is send 4
MTUs each time you receive a SACK (assuming that SACK
acknowledges 2 MTUs). And if we're buffering, it may be safe to
assume we're sending fullsize packets as we'll be likely to
bundle (I know, bundling is not required). The problem with
[METHOD2] is that it doesn't decay cwnd when under used. In my
typical example above, if the HTML page contains 20000 small
images (1K each) and the client requests one after the other,
then we will spend a long time transfering those with inflight
being 1K at max and cwnd still being 10000 MTUs. If after that
long time, the effective cwnd becomes 100 MTUs and we download a
big file again, it will take some time to bring the cwnd down
from 10000 to 100. I think we should have something to decay
cwnd when underutilised. I was suggesting something along the
lines of cwnd = (inflight + cwnd) / 2 every rto (if inflight <
cwnd).

Best regards,
Stephane

---

From: Randall Stewart <[email-masked]>
Date: Tue, 12 Dec 2006 09:32:54 -0500

Stephane:

I just re-submitted -03 of the BIS. I did
NOT address any of these items.. I just
made the text "re-appear".. that was lost.

My reasoning is this:

a) TCP does NOT have any such stricture on max.burst. So
    having such, is a more strict CC than 2581.

b) How one implements it still
    makes you less bursty than standard 2581.

c) The problem with not specifying adding the data header
    in is also in TCP.... none of its specs have you add
    any options in.. Yes I know that if a useless app
    sent 1 byte messages this far outweigh's TCP's packet
    transmission but Michael Tuexen has tested this.. reality
    gets in the way of this idea... the reality is if you send
    LOADS of 1 byte messages (or some small value) you end up
    getting a LOT of tracking structures doing a lot of mallocs
    in the kernel (or user land) and slowing down as you bundle
    the data... this is the cost of message boundaries and as
    Michael as shown in real tests ... it is not a factor linux
    does NOT add the data header in, BSD does.. but they still
    do NOT get much difference in throughput.. (see Michaels
    posts for more details). In BSD we actually put limits on
    how many messages one can have outstanding.. in fact most
    implementations that do not will find they quickly exhaust
    kernel resources and either crash.. or get in some interesting
    states to say the least .. .we have seen this at inter-ops :-)

R

-- 
Randall Stewart
803-345-0369 <or> 815-342-5222(cell)