Subject: [Tsvwg] SCTP Implementor's Guide

From: Janardhan Iyengar <[email-masked]>
Date: Wed, 27 Jun 2001 09:37:03 -0400 (EDT)

Hi all,

Section 2.8 (Fast Retransmit issues), new text for section 7.2.4 of
RFC2960:
"   5) Mark the DATA chunk(s) as being fast retransmitted and thus
       ineligible for a subsequent fast retransmit until after the
       T3-rxt timer expires.
"

Should a sender have *any* fast retranmits after a timeout? If so, there
needs to be a mechanism for determining which TSNs being SACK'd were sent
prior to the timeout rtx and which ones were sent afterwards. Only those
which were sent aftewards, should be counted towards the fast rtx trigger.

We feel that the text should read as follows:
"   5) Mark the DATA chunk(s) as being fast retransmitted and thus
       ineligible for a subsequent fast retransmit.
"

Regards,
Jana and Armando

-----------------------------------------------------------------------
"The future belongs to those who believe in the beauty of their dreams"
						    - Eleanor Roosevelt
-----------------------------------------------------------------------
Janardhan R. Iyengar				   [email-masked]
University of Delaware		       http://www.cis.udel.edu/~iyengar
-----------------------------------------------------------------------

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Randall Stewart at work <[email-masked]>
Date: Wed, 27 Jun 2001 10:06:06 -0500

Janardhan:

I don't beleive this is strictly how things work in TCP and I
don't see why things are not eligible for fast retransmit
after a timeout.

If you consider:

----FR(TSNx)---->
..
..
..
---Time-Out---
---All outstanding marked for retran---
so we end up

---Re-Tran(TSNx)---->
<plus possibly others>

At this point we have now reach the same state as before. All
of our scores are now 0. We have obviously lost our ack-clock
and we allow very little out. Probably our TSNx and no more
than a few others depending on how big TSNx is. Since we did
FR TSNx it is probably the smallest TSN... it may be capable
of being bundled... 

Now once I send these again and start to get an Ack-Clock, all
the scores are reset to 0. So I am now waiting for a 4 missing
reports to occur.

I see nothing wrong with this... Maybe a further clearification

such as..

"   5) Mark the DATA chunk(s) as being fast retransmitted and thus
       ineligible for a subsequent fast retransmit until after the
       T3-rxt timer expires. Note all missing reports for any 
       chunks that were outstanding at the time of retransmission
       MUST be reset to 0 so that four additional missing reports
       will be necessary before a fast retransmit will occur.
"

Also I believe you will find in Mark Allman and Ethan Blanton draft:
draft-allman-tcp-sack-04.txt

A section under transmission time-out that illustrates that the
scoreboard is purged but new SACK's arriving do continue to use
information gained during the slow-start period following a RTO 
event (At least if I have read it correctly :>)

What do you think Mark/Sally? Should new SACK's coming in after
a RTO be used to count a segment towards a subsequent FR?

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 27 Jun 2001 23:16:35 -0700 (PDT)

> don't see why things are not eligible for fast retransmit

So let's think in terms of how TCP works for the moment.

> FR TSNx it is probably the smallest TSN... it may be capable

Assume that TSNx is the first fast retranmitted chunk.  Note that
with SACK, it is possible to fast retransmit several chunks during
the fast retransmit phase.  Since timeout happens, it means that
the fast retransmitted TSNx is also dropped.  Here I make the
assumption that RTO is sufficient for the fast retransmitted TSNx
to get to the other side and the cumulative ACK to come back without
a timeout.  

So Janardhan's question is during the slow start after the timeout,
what should be sent out?  TSNx has to be sent out for sure.  How
about other fast retansmitted chunks during the fast retransmit
phase?  The issue here is that those other fast retransmitted chunks
can also be dropped.  If they are not dropped, the cumulative ACK
triggered by the retransmitted TSNx after the timeout should have
covered everything, as they are sent before the timeout happens.
Here I make another assumption that packets are not re-ordered.

It seems to me that the correct behavior is after getting the
cumulative ACK, SCTP immediately retransmits the next TSN as it
is probably also lost.  Note the interesting part that before the
cumulative ACK for TSNx arrives, new SACK chunks can keep coming
because they are triggered by either the fast retransmitted chunks
or newly sent chunks during the fast retransmit phase.  What should
SCTP do with them?

It seems to me that the correct conservative behavior is to record
the SACK gaps, but do not do any fast retransmit during the slow
start after timeout phase.  This is the phase which starts at the 
timeout and ends when all the data chunks sent (including those newly
sent data chunk during fast retransmit phase) before the timeout
has been cumulatively ack'ed.  If I remember correctly, Sally has
a short note on this phase when she investigated the false fast
retransmit behavior of Reno. 

The reason for noting those SACK gaps is that during the slow start 
after timeout phase, even though we can be pretty sure that the 
chunk right after the cumulatively ack'ed chunk (TSNy) is also lost, 
but if SCTP is allowed by cwnd to send more than one chunk, should it
also send the chunks (TSNy + 1, TSNy + 2, ...).  I believe the SACK
gaps received after the timeout can help in this case.  SCTP can
use the info to decide whether TSNy + 1 and subsequent chunks should
be retransmitted.

The reason not to do fast retransmit during the slow start after
timeout phase is just for simplicity.  Note that by using the above
behavior, fast retransmit during the slow start after timeout phase
is useful only if the retransmitted chunk during this phase are
also lost.  Since SCTP is already in slow start mode, more drops
mean that the network is really congested.  I guess the most network
friendly behavior is to timeout, instead of using yet another fast
retransmit to avoid that.

So I am still firmly believe that the SCTP retransmission behavior
should be exactly the same as TCP.  I really don't see the need to
have yet another behavior as the default.  Note that people can
still investigate what a better behavior is.  But this is probably
research issue and does not need to put down in a draft at this
moment.  When the research becomes mature enough, implementors will
adopt them.

Sally and Mark, any comments?

							K. Poon.
							[email-masked]

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Thu, 28 Jun 2001 02:52:55 -0400 (EDT)

> Here I make another assumption that packets are not re-ordered.

I think you misunderstood his/our question. Should a fast rtx be allowed
on TSNx after a timeout has already occurred on TSNx?

Now, you have brought a point which I haven't thought about... I need to
think some more before replying though.

-----------------------------------------------------------------------
Armando L. Caro Jr.                                  [email-masked]
University of Delaware                   http://www.cis.udel.edu/~acaro
-----------------------------------------------------------------------

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 28 Jun 2001 00:05:51 -0700 (PDT)

> on TSNx after a timeout has already occurred on TSNx?

Maybe I mis-stated your question.  Hmm, I think my previous mail included
the case you asked about.  During the slow start after timeout phase, SCTP
should not do fast retransmit.  Does this answer your question?

							K. Poon.
							[email-masked]

---

From: Randall Stewart at work <[email-masked]>
Date: Thu, 28 Jun 2001 09:26:53 -0500

> So let's think in terms of how TCP works for the moment.

Your insights are helpful here :)

> to get to the other side and the cumulative ACK to come back without

Yes, in Janardhan's NS2 simulation graph TSNx was dropped multiple
time, first time, second (fr'd) time, and then I think on the
third retranmit it made it through...

> Here I make another assumption that packets are not re-ordered.

Here, we know TSNx is sent, but as to if TSNy -- on up is sent
is unclear. They will ALL get marked for retransmission. But only
one packet is actually sent. So if TSNx can fit together with TSNy
then it would get retransmitted.. otherwise it would only be
sent when the SACK arrived for TSNx. Which would then cause
TSNy go out (and possibly other bundled chunks).

> or newly sent chunks during the fast retransmit phase.  What should

Yes, it would send TSNy when the SACK for TSNx arrives.. Assuming
that both were marked during the timeout...

> retransmit behavior of Reno.

So what we really need to do here, is to record the highest TSN
when the timeout occurs that is "missing" and NOT do any scoring
for FR until after that chunk shows up in the Cum-Ack so we know
we are past the time-out cycle .. so to speak.. is this correct?

This WOULD mean (if I interpret you correctly) that you DO NOT
FR a TSN more than once... since once you hit a Time-out you would
NOT FR anything until you get the Cum-Ack past the timeout point.

> use the info to decide whether TSNy + 1 and subsequent chunks should

Well, currently if the time-out happens all chunks pending TSNy, TSNy+1,
TSNy+2 would be marked for retransmission. Only one packet is sent
out say with TSNx so has the SACK's arrive no matter what they will
let more data out presumably those marked to retransmit, unless of
course the SACK ack's them :>

> retransmit to avoid that.

But all those chunks that were pending (when the time-out occured) WOULD
all be marked for retransmit. So if one prohibited FR until the cum-ack
passes the highest point when the time-out occured you would have the
above behavior.. with no need to look at the gap ack reports to count
missing reports for FR behavior.

> moment.  When the research becomes mature enough, implementors will

I think you are saying SCTP retransmission behavior
should be exactly the same as TCP+SACK? 

R

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 28 Jun 2001 14:15:45 -0700 (PDT)

> third retranmit it made it through...

So what is their simulation?  Maybe they can show the list what they've
done.

> TSNy go out (and possibly other bundled chunks).

The reason I did not include the bundling issue in my previous mail
is that I believe SCTP stack normally does not re-bundle chunks.
So if an IP packet contains 5 chunks, and is lost, all chunks are
lost.  This suggests strongly that when SCTP times out, all 5 chunks
should be retransmitted together.  This is my preference.  But other 
implementations may want to just retransmit the first not cumulatively
ack'ed chunk.  In reading my previous mail, one can view that a data 
chunk consuming a TSN actually occupies the whole IP packet.  Or a TSN
is actually a list of TSNs in one IP packet which is dropped.

> that both were marked during the timeout...

This is one thing that I believe we should clear up.  What is meant
by marked for retransmission after a timeout?  And I suggest we
tighten up the use of SACK and cumulative ACK.  I believe what you
meant above is cumulative ACK for TSNx arrives.  TSNx, in the context
of our discussion, is the smallest TSN not cumulatively ack'ed.  So
it won't be sack'ed as the cumulative ACK should go up.  If I mis-
understood this, correct me.

What I suggest, which is what TCP does, is when a timeout happens,
SCTP goes into the slow start after timeout phase.  The SACK info
is purged at the timeout.  And all chunks not cumulatively ack'ed
are marked for retransmission.  Subsequent SACK info received because
of those fast retransmitted chunks and newly sent chunks during fast
retransmit phase will be recorded.  And when SCTP needs to decide what
to retransmit during this phase, use the SACK info appropriately.
And during this slow start after timeout phase, do not do fast
retransmit again.  This is a conservative suggestion.

> NOT FR anything until you get the Cum-Ack past the timeout point.

The above is correct.  And the reasons not to do fast retransmit
during slow start after timeout phase were stated in my previous mail.
This is conservative and simple (-:  If we believe that timeouts do
not happen often if SCTP is friendly to the network, then when one
does happen, the network is in pretty bad shape, SCTP should be
more conservative.  Fast retransmit is a way to avoid timeout and
yet be network friendly.  So if a fast retransmitted packet or 
retransmitted packet during this phase is again dropped, I think SCTP
should be very conservative and not do fast retransmit again.

> course the SACK ack's them :>

Agree.

> missing reports for FR behavior.

Yes, The SACK info (gap ack report) is used in the slow start after 
timeout phase as the guideline on what to retransmit, not for fast
retransmit as it is not done during this phase.

> should be exactly the same as TCP+SACK? 

Well, I believe nowadays every TCP stack has SACK...  So I consider TCP
has SACK builtin already (-:

							K. Poon.
							[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 28 Jun 2001 16:47:34 -0500

Kacheong Poon wrote:
...
> should be retransmitted together.

This assumption is very questionable, due to the fact that at the time
of retransmission either or both the cwnd and PMTU can change
dramatically (especially the cwnd since the retran very likely will be
sent to a different dest address).

-Qiaobing

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 28 Jun 2001 15:27:52 -0700 (PDT)

> sent to a different dest address).

For cwnd, I consider a good implementation to increase it my PMTU
size, not by bytes.  This is a little more efficient because chunks
can be bundled in the largest possible way.  The only change is the
way to do accounting.  The rate of increasing cwnd per RTT is the same.
And SCTP won't break up chunks unnecessarily to fit cwnd.  This is the
same for TCP, cwnd is increased by MSS size.

Even if PMTU changes (say because it is sent to a different dst), the 
method outlined in my previous mail will still work.  It is just an outline,
I don't see any problem in working out the details...  For example, an
implementation may choose the most conservative way of only sending one data
chunk after the timeout.  This may be good because SCTP does not really
know where the congestion is.  It may well be very close to sender and
regardless of which dst address SCTP uses, it will get hit.

							K. Poon.
							[email-masked]