Subject: Re: [Tsvwg] taking NewReno (RFC 2582) to Proposed Standard

From: Kacheong Poon <[email-masked]>
Date: Mon, 19 May 2003 19:27:56 -0500 (CDT)

Included message from Mark Allman <[email-masked]>:

>----
>  * To my way of looking at things, scheme (1) uses two controls for
>    how much data can be sent into the network.  That is, we can
>    either be limited by cwnd or we can be limited by maxburst.
>
>    And, since there are two controls there are interactions between
>    them that must be taken into account -- in specs and code.  And,
>    there are things that are currently speced that will break.
>    That is, TCP can be running in a state where there is less than
>    cwnd on the network because it is limited by maxburst instead of
>    cwnd and the specs/code do not envision the case and therefore
>    cannot be exepected to do something intelligent.  
>
>    An example of where this breaks things: The limited transmit
>    spec says that on dup acks TCP can inject at most cwnd+(2*MSS)
>    of new data into the network.  The hope was that this means that
>    we can send data on the first two dup acks.  If maxburst has
>    limited the amount of outstanding data to something less than
>    cwnd then limited transmit now allows the sending of more than
>    two new segments on dup ACKs.  That might be OK.  But, I think
>    that is clearly at odds with the spirit of RFC 3042.  I think
>    there are likely quite a number of examples of unintended
>    interactions/consequences.
>
>    But, that can be fixed... we can set a flag, add some
>    conditionals, some temporary variables, write some new text and
>    plug up these things right up...

Hmm, this seems like a hole in how the Limited Transmit algorithm
is described.  Basically, what Limited Transmit wants is to send
two new segments out, one per dup ACK, if allowed by rwnd.  Since
TCP will never send more than cwnd allows, RFC 3042 can just say
"send one new segment per dup ACK" instead of mentioning anything
about the number of outstanding segments in the network is at most
cwnd+2*SMSS.  This will avoid the pitfall described above.  There
is no need to introduce new temporary variables, write some new text
or plug up things...

>  * On the other hand, scheme (2) has only a single controller.  We
>    are always limited by cwnd -- and, maxburst is just an input to
>    this controller.  On the surface, the above snippets of code may
>    look the same, but they are not.  If you take the limited
>    transmit case again... LT has no problem under scheme (2)
>    because the cwnd is now tracking the amount of outstanding data.
>    All of our code and our specs this to be the way things are.
>    So, this works nicely.

I guess we should separate the algorithm from code for a moment.
Assuming Limited Transmit was described as I suggested above, 
would the maxburst variable affect it?  I guess not.  So it becomes
an implementation error if the maxburst variable actually affects
Limited Transmit.  It works nicely as well (-:

>  * Now let me take a minute and say that (a) I do not care for the
>    downsides of scheme (2) that Sally illustrated and (b) I think
>    that scheme (1) is equally bogus because it adds a second
>    control for the amount of outstanding data -- and no end of
>    headaches will come of it.  Also, I do not buy that because TCP
>    sends "micro" and "macro" bursts that we should have two
>    controllers for these sending patterns.  It makes no sense to me
>    since in the end they control the same thing: the amount of data
>    that can be dumped into the network.

Well, if Mark does not care about problems with scheme (2), there
is no need for discussion (-:  I guess we should forget about that
there is actually a variable called maxburst and scheme (2) for a
moment.  Consider this case, an app using TCP has a brief moment
(much shorter than one RTT) of pause such that the number of
outstanding segments in the network is less than the current cwnd.
Should TCP reduce the cwnd because of this?  I guess most, if not
all, people agree that cwnd should probably be reduced if the pause
is longer than one RTT, as described in RFC 2861.  This is the idea
behind "Use-it or Lose-it."  But for a period less than one RTT, I
suspect that most people will say no.

Using the variable maxburst, there is no effect on cwnd in the above
case.  It still tracks the current "macro" sending rate and I
guess most people are happy with that.  In fact, maxburst is kind
of like an "artifical app pause" in other cases as described in this
mail thread.  Using scheme (2), TCP will immediately forget about
the current sending rate and start the whole slow start process once
again.  Do most people agree that it is the appropriate action,
especially in the case with long RTT, such as satellite link?

>  * So, the question I am left puzzling over is: can we combine
>    schemes (1) and (2) in a sensible way to avoid the downsides
>    that Sally has shown while also avoiding a second control on
>    outstanding data?

But I guess we should first ask ourself if scheme (2) is actually a
good thing to do before tackling the above question.

>  * One more observation: If maxburst is 4, we use scheme (1), we
>    use delayed ACKs, cwnd = outstanding_data = 10 segments and we
>    get an ACK that covers 10 previously unACKed segments (because
>    of dropped ACKs) the events will look like this:
>
>	get ACK for 10 segments; send 4 segments
>	    (outstanding = 4; cwnd = 10)
>	get ACK for 2 segments; send 4 segments
>	    (outstanding = 6; cwnd = 10)
>	get ACK for 2 segments; send 4 segments
>	    (outstanding = 8; cwnd = 10)
>	get ACK for 2 segments; send 4 segments
>	    (outstanding = 10; cwnd = 10)
>	get ACK for 2 segments; send 2 segments
>	    (outstanding = 10; cwnd = 10)
>
>    I.e., it's slow start.
>
>  * So, what if we introduce a scheme (3), which is really just a
>    riff of scheme (2):
>
>    if ((outstanding_data + maxburst) < cwnd)
>        if (ssthresh < cwnd)
>	    ssthresh = cwnd
>        cwnd = outstanding_data + maxburst
>    while (outstanding_data < cwnd)
>        send_segment ()
>        outstanding_data++
>
>    In other words, we clamp cwnd to limit the burst, but we also
>    set ssthresh in such a way that we can slow start to climb out
>    of the cwnd reduction.
>
>    It is late, but I think this gives us the good points of (1) and
>    (2) -- that is, one controller with the same basic pattern of
>    sending given by scheme (1).
>
>Thoughts?

Hmm, I guess you want to simplify things...  But it seems to
me that the above actually make things more complicated.  What
do we gain?

>[Also, with appologies to Joe Touch & Amy Hughes --- who probably
> had all this figured out long ago (but, their exact ideas have
> fallen out of my head.]

Yeah, I remembered Joe Touch mentioned something at one IETF long
ago and said that he would write up a draft or something.  But I
don't know if that actually happened...

>----

							K. Poon.
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Mark Allman <[email-masked]>
Date: Mon, 19 May 2003 23:00:40 -0400

> is no need to introduce new temporary variables, write some new
> text or plug up things...

You made my point.  If we are to use a simple maxburst limit then
the limited transmit rfc has a "hole" and we do need new text.  It
might not be complicated, but it will become busted and in need of a
fix.  And, LT was only an example.  My bet is that we have to revist
a bunch of things for similar reasons.  And, as we evolve things
we're going to have to remain cognizant that now we have two things
controlling the amount of data being sent into the network (possibly
requiring two different decision paths, etc.).

> Limited Transmit.  It works nicely as well (-:

Hm, if one were to implement rfc3042 and we reach consensus that
said implementation did the wrong thing it would be more than an
implementation error -- it would be a spec error.

> Consider this case, an app using TCP has a brief moment (much
> shorter than one RTT) of pause such that the number of outstanding
> segments in the network is less than the current cwnd.  Should TCP
> reduce the cwnd because of this?  I guess most, if not all, people
> agree that cwnd should probably be reduced if the pause is longer
> than one RTT, as described in RFC 2861.  This is the idea behind
> "Use-it or Lose-it."  But for a period less than one RTT, I

The problem is that discussing things only in terms of cwnd misses
the whole point.  Of course, it might be natural to think "ah, I
don't want to sacrifice cwnd".  But, by limiting the bursts in
response to each ACK we are basically doing exactly the same thing
--- that is, we are limiting the amount of data that can be put on
the wire.

I agree that scheme (2) is somewhat naive -- as folk have been
saying. 

But, with scheme (3) (which is a slight riff on scheme (2)) I bet we
see basically the same sort of behavior as scheme (1) and with the
bonus that (a) we only use one control and so interactions are nil
and (b) cwnd tracks the amount of outstanding data, which is in
better alignment with the expectations of our code and specs.

> don't know if that actually happened...

(Joe and Amy did have a draft and did discuss it in tcpimpl a bit a
while back.  The tcpimpl archives are at:
http://tcp-impl.grc.nasa.gov/tcp-impl) 

allman

--
Mark Allman -- BBN/NASA GRC -- http://roland.grc.nasa.gov/~mallman/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg