Subject: Re: Upper limit on RTT?

From: Vernon Schryver <[email-masked]>
Date: Fri, 24 Sep 1999 15:52:33 -0600 (MDT)

> From: Jamshid Mahdavi <[email-masked]>

> I stumbled across something interesting yesterday.  Most TCP
> implementations use an initial timeout of something like 3 or 6
> seconds.  If the SYN-ACK is not received in this time, congestion
> avoidance is entered, with ssthresh set all the way down to 1 or 2
> packets.
>
> This means that for any path which is longer than this 3 or 6 second
> timeout, a connection will be guaranteed to take a retransmission
> timeout and do no slowstart.  Since these are exactly the connections
> that need slowstart in order to quickly develop a large cwnd, it means
> that network paths of this length will never get any reasonable
> performance.
>
> Interesting, huh?
>
> The only solution which quickly comes to mind is to detect this using
> our new DSACK (draft-floyd-sack-00.txt), and reset ssthresh to a large
> value once you've detected the problem.
>
> It might be good to add this to one of the PILC drafts...

How many paths have 3-6 second RTT's? 
10 geo-sync satellite hops is more than I've heard about lately.

More important, how do you get any reasonable performance from such a path
regardless of what you might do with slow start and initial congestion
windows?  Consider the maximum throughput of a path with a 6 second RTT
and with typical 8-16 KByte windows.  I think less than 3 KByte/sec has
been beyond unreasonable for at least 10 years.  With maximum, non-TCP-LW
windows of 64K you are still limited to ~10 KByte/sec.  Is 10 KB/s
considered reasonable?

To get reasonable performance with a 6 sec. RTT, say 0.1 to 1.0 MByte/sec,
you would need to have at least 0.6 to 6.0 MByte in flight.  Since it
seems unlikely that the data would be flying through speed of light delays,
the vast majority of it must be flying in holding patterns in router buffer
queues.  Do you think adding several MBytes per TCP connection to router
buffering is a good idea.

By the way, regardless of initial behavior, how long would it take slow
start on a link with 6 second RTT to reach a large enough congestion window
to allow reasonable performance?

Vernon Schryver    [email-masked]

P.S.  Please forgive me if this was a joke that went over my head.

---

From: Jamshid Mahdavi <[email-masked]>
Date: 24 Sep 1999 15:33:13 -0700

> 10 geo-sync satellite hops is more than I've heard about lately.

Someone always has to bring up pesky little things like reality, huh?
:-)

Not any that I know of.  But paths do seem to be getting longer
and longer as the Internet grows.  I can't imagine that anyone would
*want* to go through a path this long, but it might happen.

Another thing to consider is if people start hacking the RTO for the
initial SYN.  After all, who wants to wait 3 seconds if their SYN gets
lost?  If this number is reduced (and it could happen inadvertently if
people lower the clock tick granularity without changing the
initialization constants), 3 seconds might turn to 1 second or less,
which would then perhaps make this a more real problem.

I'm not saying its the end of the world, just that it is an
interesting "feature" of the way things currently work.

> windows of 64K you are still limited to ~10 KByte/sec.  Is 10 KB/s

I'm assuming that we have magically trained the world to use larger
windows.  Someday, everyone might have autotuning in their stacks, so
default window size won't be an issue.  Almost every vendor out there
already has TCP-LW shipping.  And even a 128kB window on a 3 second
path would give about 40kB/sec -- useful for many things.

> buffering is a good idea.

I'm not going to touch this one :-). 

> to allow reasonable performance?

Well, for my 3 seconds at 128kB, with an initial window of 3, about 30
seconds.  Adequate if you were downloading a new Netscape on the North
pole; not so good if you want to websurf though...

--J

---

From: Grenville Armitage <[email-masked]>
Date: Fri, 24 Sep 1999 22:51:54 -0400

> *want* to go through a path this long, but it might happen.

I'm not sure it will happen to anyone who cares enough about their
traffic to want TCP optimized for this case. Unless you're thinking
of inter-planetary TCP (which, I suppose, you might be) those sort of
delays either suggests a huge number of router hops adding small
queuing delays components, or a moderate number of hops adding
large queuing delay components.  Unless the physical path is
extremely wierd, I can't see speed-of-light delays contributing much
to 3-6 second RTTs.

Either way, queuing delays will be impacted by the advent of things
like DiffServ and/or IntServ - differentiated queuing for people with
cash, isolating them from cyclical overloading of the best-effort
queue. For these people they'll never see multi-second RTT unless
the network is broken. Theory aside, SLA requirements will begin to
force rationalization by major {I}SPs of both physical topology _and_
per-hop queuing behavior to keep RTT 'reasonable'.

I suppose the rest, dumped into the BE queue, will potentially
see the same large queue build up (and consequential multi-second
RTT), but such queues ought to be using RED (or some variant) to
keep avg per-hop latency down anyway.

Just some ramblings....

cheers,
gja
________________________________________________________________________
Grenville Armitage                            http://mh005.infi.net/~gja
Bell Labs Research, Lucent Technologies

---

From: Cannara <[email-masked]>
Date: Sun, 26 Sep 1999 21:14:15 -0700

To add another side of reality, it's surprisingly frequent that 3-second+ RTTs
are even seen on LANs, simply because the hosts are occasionally quite busy
and/or misconfigured.  If a transport uses this, rather than the receiver's
window, to decide to slow down, it's making a mistake, since there may not
even be any router queues to protect.  The catch-all nature of TCP's reaction
to a delayed/missing/repeated ack means that its adjustable parameters need
more care in setting -- defaults as shipped are generally well off optimal.

Alex