Subject: Re: Upper limit on RTT?

From: Vernon Schryver <[email-masked]>
Date: Fri, 24 Sep 1999 20:13:31 -0600 (MDT)

> From: Jamshid Mahdavi <[email-masked]>

> ...
> Not any that I know of.  But paths do seem to be getting longer
> and longer as the Internet grows.  I can't imagine that anyone would
> *want* to go through a path this long, but it might happen.

No, they will not happen, because the speed of light is what it is,
except when the result from basically broken routers that should be
dropping packets to signal congestion.

> ...
> > To get reasonable performance with a 6 sec. RTT, say 0.1 to 1.0 MByte/sec,
> > you would need to have at least 0.6 to 6.0 MByte in flight.  Since it
> > seems unlikely that the data would be flying through speed of light delays,
> > the vast majority of it must be flying in holding patterns in router buffer
> > queues.  Do you think adding several MBytes per TCP connection to router
> > buffering is a good idea?
>
> I'm not going to touch this one :-). 

For even a 1 second RTT to be worth worrying about,
you must handle that one.  

In real life, at least in my experience, multi-second  RTT's happen only
when too many bits are setting in queues.  There are some things that can
help such cases, such as TOS queuing and reducing the sizes of queues
so they drop sooner and don't bloat so badly.  The worst thing you can do
is to try to put more data into those bloated buffers; that is, of course,
the whole point of slow start.

Large windows are good things, but only for genuinely fat and long pipes.
Otherwise, larger windows can be the direct cause of multi-second RTT's.
About 10 years ago, I changed the default window of my then-employer's
product from my previous choice of 30K to 60K, to make my FDDI stuff
benchmark faster.  Then I moved back to Colorado, and started using a
shared, 1200 mile long, narrow pipe to talk source trees, test systems,
and so forth.  I found that my interactive access had consistent 0 to 10
second sawtooth delays, due to big router buffers being filled by other
people doing bulk transfers.

> > By the way, regardless of initial behavior, how long would it take slow
> > start on a link with 6 second RTT to reach a large enough congestion window
> > to allow reasonable performance?
>
> Well, for my 3 seconds at 128kB, with an initial window of 3, about 30
> seconds.  Adequate if you were downloading a new Netscape on the North
> pole; not so good if you want to websurf though...

No, at the North Pole, they'll be able to see some kind of satellite,
and will have RTT's of less than 1 second.  When that doesn't work,
they'll use good ol' station wagon net, a twice weekly air-drop of a
large car filled with CDROM's.  That will be more reliable, have far
higher bandwidth, and latency not much worse.

People simply do not tolerate multi-second RTT's.  They do something else,
such as using station wagon net or not using the net at all.

 ....
 
In TCP, IP, or anything, the trick is not to find something to change,
since anyone can always find something to change (and sometimes even
improve) in anything.  The trick is to find necessary changes.

Vernon Schryver    [email-masked]

---

From: "Johnson, Gregory LCDR" <[email-masked]>
Date: Mon, 27 Sep 1999 14:55:24 -0400

>How many paths have 3-6 second RTT's? 
	>10 geo-sync satellite hops is more than I've heard about lately.

	It's possible to get a 3 second RTT very easily on actual satellite
systems; the round trip delay in an actual GEO satellite system is typically
more than just the speed of light delay. We are doing some work with using
TCP/IP over slow satellite links (since in many cases that is all that is
available for mobile users). Using the AMSC system, you have a round-trip
delay of 1400 ms (speed of light plus interleaving/deinterleaving and other
delay introduced at the Earth Station). Add on to that the transmit time for
a packet at 4800 bps (which is non-negligible). If you are using multiple
connections, so you try to start a new connection while data is already
being received for the first connection your packets can easily experience a
delay of more than 3 seconds due to the round-trip time, the transmit time,
and the queuing delay.

	LCDR Gregory Johnson
	Ass't Professor, Electrical Engineering
	USCG Academy
	New London CT 06320

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 27 Sep 1999 16:56:16 -0600 (MDT)

> From: "Johnson, Gregory LCDR" <[email-masked]>

> delay of more than 3 seconds due to the round-trip time, the transmit time,

It sounds as if you make the obvious fixes, including:
  - TOS or other smart queuing in the routers wiht 3-1.4=1.6 seconds of queue
  - teach the necessarily special applications involved to not put/ask
      for too much data in flight.
  - use reasonable sized packets by setting the MSS low (i.e. use the
      packet sizes that were known to be reasonable 10 or 15 years ago
      when 4800 bps was fast).
there would be no 3 second RTT's for SYN's or SYN-ACK's.

Besides, as I said before, 3-6 seconds is over the boundary for interactive
traffic.  Human users will simply not tolerate such delays, as has been
known from studies of interactive (actually "timesharing") applications
from 25+ years ago.  If the real RTT's are 6 seconds, they'll do something
else.

That leaves computers talking over such slow links with 1.6 second RTT's,
and computers can be taught to not open zillions of simultaneous and/or
short-lived TCP connections at once (HTTP 1.0 not withstanding).

To put it still another way, the BSD code evolved back in the days when
2400 bps was fast, and later did ok with 9600 bps Telebit modems, with
their ~3 baud signaling.

---

From: Markus Buchhorn <[email-masked]>
Date: Tue, 28 Sep 1999 08:58:21 +1000

>> 10 geo-sync satellite hops is more than I've heard about lately.

How about a path involving a dial-on-demand link ? Modem negotiation can be
much longer than 3-6 seconds...

Markus Buchhorn,  Advanced Computational Systems CRC     | Ph: +61 2 62798810
email: [email-masked], snail: ACSys, RSISE Bldg,|Fax: +61 2 62798602
Australian National University, Canberra 0200, Australia |Mobile: 0417 281429

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 29 Sep 1999 16:14:12 -0500

I would vote for "SLOW", for the recommendation on maximum MTU transmission
delay.

We're beating up the "LINK" recommendation right now (it's not a secret -
the version discussed in Oslo has been at
http://people.qualcomm.com/karn/pilc.txt since about five minutes after the
Oslo cut-off, but this got lost in the cut-off forest, and I wasn't paying
attention in July and August). I'm thinking the guidance to link designers
(called "subnetwork designers" in PILC) could go in "Connection-Oriented
Subnetworks" or "Bandwidth on Demand (BoD) Subnets".

My view is that TCPSAT has handled most of the "LONG" recommendations
already, so PILC wasn't planning on doing a "LONG" document as a WG product
- but Mark Allman has just reissued the TCPSAT Research Issues I-D. Mark,
would the recommendation about SYN timeouts fit there? I didn't see this
topic in version 11, and it might fit before the T/TCP section (currently
3.1). (I'll understand if you say "no"!)

Spencer

> -----Original Message-----
> From:	Jamshid Mahdavi [SMTP:[email-masked]]
> Sent:	Wednesday, September 29, 1999 11:32 AM
> To:	[email-masked]
> Subject:	Re: Upper limit on RTT?
> 
> Vernon Schryver <[email-masked]> writes:
> 
> > > From: Markus Buchhorn <[email-masked]>
> > 
> > Yes, but that's not an RTT, or at least not an RTT that keeps the
> > congestion window from opening at a reasonable rate when the connection
> > is finally established, is it?
> 
> Actually, I believe that the way most implementations today work, if
> the SYN is delayed, the sender takes the timeout and doesn't
> subsequently do slowstart.
> 
> From the recent messages, it certainly sounds like this qualifies as a
> "performance implication of a link characteristic" :-)
> 
> My take is that there are two interesting messages we want to get out:
> 
> 1. Link designers: Avoid very large delays, especially on connection
> setup, when designing a technology.
> 
> 2. TCP implementors: Don't reduce the initial SYN timeout too much
> (where too much probably means below 3 seconds) if you want any hope
> of achieving large windows/bandwidth on a high delay pipe.
> 
> Also, do we have something like Vernon's comment about packet sizes
> for slow links in the "thin" draft?  It might be good to include some
> rule of thumb, like no more than 200msec for a packet transmission
> time, to help people choose MTU.  (200msec is a nice number, because
> it will enable most delayed ACK implementations to still function
> correctly).
> 
> Any suggestions about which documents all this should go in?
> 
> --J