Subject: Retransmissions without explanation

From: Jaques Rechtand <[email-masked]>
Date: Tue, 18 May 1999 15:46:27 -0300

Dear friends;

  We were making some FTP transfers between a Sun and a NT machine through a
satellite link simulator, with a round trip delay of about 600 msec through
a 256 Kbps link. This application runs in our network (really through
satellite) and it is too slow. That's why we tried to simulate it to better
diagnose the problem.

  We were monitoring the transfer with a LAN Analyzer, and we saw a message
reporting excessive retransmissions. We looked at the traces, and found that
the Sun was periodically retransmitting some packets. There was no packet
discard in the routers, no CRC errors, and no TCP segment asking for
retransmission (with an already acknowledged ACK). So, we concluded that the
retransmission timeout was expiring at the Sun. We were surprised, because
the RTO for TCP should be dinamically adjusted to reflect the round trip
delay of the link. 
 Searching in the Internet, we found an App Note :"Internet Performance on a
Satellite Link", from Intelsat, that said that at the Sun there was a
parameter for minimum retransmit time, and that the protocol periodically
resets the RTO to this minimum, causing some retransmits. We found that
there is a parameter called tcp_rxmit_interval_min that was defaulted to 200
msec. We changed this parameter to 600 msec and the retransmissions
disappeared.

 We have some questions :

- Is it normal for the RTO to periodically reset to its minimum value ?
- Is there any available specification of the algoritm that calculates the
RTO, that is used by the implementors ?

 Then we tried to do an ftp, transfering a file from the NT to the Sun, and
the problem still happened in that direction. We didn't find any parameter
equivalent to the parameter in the Sun to configure in the NT or 95 .

- Do you know if the problem in the NT is the same as in the Sun?
- Do you know how can we change it in the NT ? We looked at the parameters
in the registry and didn't find anyone.

 Any help is welcome.

 Best Regards,

            Jaques Rechtand
            VICOM S.A.
            System Engineer
            Tel : 55-21-5366136
            Fax : 55-21-5366149
            e-mail : [email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 18 May 1999 16:02:38 -0700 (PDT)

> the RTO for TCP should be dinamically adjusted to reflect the round trip

This question came up some time before in this mailing list.  The following
was my response.  Note that in Solaris 7, the workaround mentioned in the
mail should not be needed.

							K. Poon.
							[email-masked]

>----- Begin Included Message -----<

Date: Mon, 10 Nov 1997 16:46:33 -0800 (PST)
From: "Kacheong Poon" <[email-masked]>
Subject: Re: TCP over GEO < 512kbps
To: "Chuck Nunez" <[email-masked]>
Cc: [email-masked]

> The reference in another message today about a patch (SUN's CONSULT-TCPLFN)
> for Solaris 2.5.1 and below, adds support for Long Fat Networks
> (RFC1323-TCP Extensions for High Performance).  It is included in 2.6 (so
> I've been told).

Yes, Solaris 2.6 has all the RFC 1323 extensions.
 
> Finally, there is one other nit that might present an obstacle to high
> performance.  If the default values for Solaris' retransmit timers
> (tcp_rexmit_interval_initial, tcp_rexmit_interval_min,
> tcp_rexmit_interval_max) are set below the RTT, then Solaris will
> retransmit unacknowledged packets in spite of the fact that the RTT has not
> yet expired.  For example, it the RTT is 750 ms, make sure that the
> retransmit timers are set to 1000 ms or more to preclude unnecessary
> retransmissions.  

Well, RTT (round trip time) won't expire (-:  The problem you mentioned was
that the calculated RTO (timeout) was lower than RTT.  This causes packets to
be retransmitted unnecessarily.  I think this RTO problem was discussed in the
last IETF tcp-impl WG meeting.  RFC 1323 timestamp option allows TCP to get
very accurate samples of RTT from every packets.  TCP's RTO is calculated using
the following formula:

	RTO = sa + 4 * sd

sa is the smoothed average of RTT samples and sd is the smoothed mean
deviation of RTT samples.  With timestamp, sa converges quickly to the
real RTT while sd becomes very small.  As you mentioned in a previous mail,
RTT of satellite links may sometimes jump to 800ms.  This kind of sudden jump
may not be captured by sd.  This can cause some unnecessary retransmissions.

In BSD's TCP implementation, RTO usually has a ~500ms "buffer zone."  So
the above problem is not seen often.  Solaris has a fine grain RTO, so you
may see this problem sometimes.  In 2.6, the workaround is to set
tcp_rexmit_interval_extra to 500 to get an extra 500ms buffer zone like BSD.

The tuning you suggested above is for a different bug in earlier releases of
Solaris.  2.6 should not have that problem.

							K. Poon.
							[email-masked]

>----- End Included Message -----<

---

From: Jamshid Mahdavi <[email-masked]>
Date: 19 May 1999 09:14:39 -0700

[This is in response to a note on tcpsat about retransmit timers
firing too early on long connections.]

I have seen unwanted retransmit timouts (i.e. bugs) of this nature in
two other implementations as well.  All of these implementations have
in common more accurate timers.  I suspect it would be useful to have
more detailed documentation on the use of timers and computation of
RTO.  However, it isn't clear to me that there is a lot of consensus
on a number of the issues.

Here are some issues which I think are unresolved, or could use better
documentation:

- How to modify the RTO calculation when doing 1323 timestamps.  The
  idea here is that you are getting measurements more quickly, so you
  might want to adjust the moving average calculation.

- Running with fine-grained clocks.
  1.  Should you?
  2.  Are adjustments to the computations needed?  (Remember delayed
      ack might tack on 200msec).

- Always time the RTO from the last packet in flight.

- Should there be a minimum RTO?

- Should there be a minimum deviation?

Interestingly, I see no discussion of these issues in the TCPSAT
drafts, and I don't believe I have seen them covered in any of the
other relevant places either (i.e. pilc or tcl-impl).

I'd be happy to pull together ideas on this subject if people are
interested in seeing such a document.

--Jamshid

PS.  On a process note, I'm thinking this belongs under tcp-impl, so
I've sent this note to tcp-impl and BCC'd tcpsat (where the
conversation started).

---

From: "ed" <[email-masked]>
Date: Wed, 19 May 1999 09:35:36 -0000

---

From: Marian Stagarescu <[email-masked]>
Date: Wed, 26 May 1999 18:01:18 -0400 (EDT)

Can somebody address the equivalence of the tcp_rxmit_interval on Linux ?

Thanks,
Marian Stagarescu

On Tue, 18 May 1999, Jaques Rechtand wrote:
>  We found that
> msec.