Subject: BER and TCP/IP performance

From: Jon Mansey <[email-masked]>
Date: Tue, 30 Mar 1999 15:09:26 -0800

Hi,

Can anyone point me to some research or tables showing how BER affects
TCP/IP thoughput?

At the same time, does anyone have any real world experience of what is an
acceptable BER to shoot for in designing end-end sat links for TCP/IP.

Thanks in advance.

Jon.
[email-masked]                          Chief Science Officer
------------------------------------------------------------------
 "Low-cost & high-speed access to the US Internet via Satellite"
InterPacket Group, Inc.                 http://www.interpacket.net
1901 Main St. 2nd Floor                         tel (310) 382 3300
Santa Monica, California 90405                  fax (310) 382 3310
------------------------------------------------------------------

"Unix IS user friendly... It's just selective about who its friends are."

---

From: Fred Baker <[email-masked]>
Date: Tue, 30 Mar 1999 16:11:17 -0800

>"Unix IS user friendly... It's just selective about who its friends are."

One could say the same thing about TCP being BER friendly. Most RFC
2001-compliant implementations assume that congestion has occurred if a
packet is lost, and halve their rate. Most TCPs also send segments of
either 512 or 1460 bytes in size - which with the TCP, IP, and link layer
overheads becomes on the order of 5000 to 12000 bits. A BER on the order of
10E-4 therefore stops them cold, and a BER on the order of 10E-N will limit
a single TCP stream to averaging 10E+(N/2) bits or less per round trip time.

That should break out something like this:

			bits	rate in kbps
			per 	 for 600 ms
exponent	BER	rtt	delay
	4	1E-04	1000	1.666666667 (actually, close to zero, 
	5	1E-05	1E+4	16.66666667  due to frame size)
	6	1E-06	1E+5	166.6666667
	7	1E-07	1E+6	1666.666667
	8	1E-08	1E+7	16666.66667
	9	1E-09	1E+8	166666.6667
	10	1E-10	1E+9	1666666.667
	11	1E-11	1E+10	16666666.67
	12	1E-12	1E+11	166666666.7

---

From: William D Ivancic <[email-masked]>
Date: Wed, 31 Mar 1999 08:22:16 -0500

>acceptable BER to shoot for in designing end-end sat links for TCP/IP.

1E-8 or better.  It really depends on the length of transfer.  As soon as
you get an error, your window is going to half it size due to congestion
control.  So for EXTREMELY long file transfers, you need near error free
links.  For short file transfers, you'll never get past slow start.  Also,
for hundreds of connections consisting of UDP and TCP, when one stream
reduces its rate, another will fill in the gap.

The following report has a graph showing TCP vs SPCS in an errored
environment.  Remember.  This is for a single stream.  SCPS performs very
well here, but was optimized for a specific environment that does not
correspond to that of a commercial satellite.  

http://www.scps.org/scps/assets/M22.pdf

Regards,

Will
*********************************************
William D. Ivancic
NASA Glenn Research Center
21000 Brookpark Rd.   MS 54-8
Cleveland, Ohio  44135
USA
Phone:   1 216 433 3494
FAX:     1 216 433 8705
Email:   [email-masked]
             [email-masked]	
http://ctd.grc.nasa.gov/5610/5610.html	
*********************************************

---

From: Gertjan van Oosten <[email-masked]>
Date: Wed, 31 Mar 1999 16:09:57 +0200 (MET DST)

As quoted from William D Ivancic:
> links.

Or you enable SACK (or any of its newer brethren), which is highly
recommended.  For some data points, have a look at my ARTES3
presentation on:

  http://www.estec.esa.nl/artes3/workshop/gertjan.html

Regards,
-- 
-- Gertjan van Oosten, [email-masked], West Consulting B.V., +31 15 2191 600

---

From: Eric Travis <[email-masked]>
Date: Wed, 31 Mar 1999 09:53:23 -0500 (EST)

On Wed, 31 Mar 1999, William D Ivancic wrote:

<snip>

> correspond to that of a commercial satellite.  

Huh?!?

OK - I really have to ask... what *specifically* does not 
"correspond to that of a commercial satellite"? 

Do you mean "commercial communications satellite"? Even there, 
I'm not sure what specific environment you mean by such a 
generalization. 

Can you please clarify?

If you mean that communications satellites have no errors
(or there are no economic impacts to scrubbing a satellite
link to 10E-10+), why do these questions keep coming up? :o)

Can you also be more specific about what optimizations SCPS-TP 
(I assume you *are* referring to the transport protocol, not the 
network, security protocols) makes that make it inappropriate
for use on a "commercial satellite" sunbnetwork? Where (and why) 
is it not applicable? 

Finally, not every satellite is a communications satellite - this 
is a point I tried (unsuccessfully) to make during the life of the
TCPSAT working group.

For the original poster, there is no simple answer[1]; 

  Are the errors uniform or bursty?

  What is your segment (and frame) size?

  What does your traffic look like?

Eric

[1] I lied - The simple (and unsatisfying) answer is to have as 
clean a link as your link-budget can provide without busting your
engineering/operational budget. TCP's interpretation of loss
as a signal for congestion demands operation over as clean a link 
as possible. Fred's answer of 10E-8 is usually a good rule of thumb, 
but you still only use a fraction of the satellite bandwidth for which 
you are paying.

TCP works over really crappy links - data will be reliably delivered, 
it just that the performance will suffer.

---

From: Jamshid Mahdavi <[email-masked]>
Date: 31 Mar 1999 10:23:25 -0800

The currently accepted best formula for TCP performance as a function
of packet loss is given in the following paper:

Padhye, J., Firoiu, V., Towsley, D., and Kurose, J., Modeling TCP
     Throughput: a Simple Model and its Empirical Validation, UMASS
     CMPSCI Tech Report TR98-008, Feb. 1998.

The URL is:
     ftp://gaia.cs.umass.edu/pub/Padhye-Firoiu98:TCP-throughput.ps.Z

You can find a compilation of other papers on the subject on the TCP
Friendly website:
     http://www.psc.edu/networking/tcp_friendly.html

You will need to do the same calculation Fred showed to convert BER to
packet loss rates.  I think you will find that BER at 1E-8 is the
highest acceptable, and for fast links you might find 1E-12 to be
desirable.

--Jamshid

---

From: [email-masked] (Robert C. Durst)
Date: Wed, 31 Mar 1999 12:53:12 -0500

Jon,

There's been a fair amount of traffic here already, but let me chime
in to provide some theory, some opinion, and some data.

Theory:  Matt Mathis, and folks developed and validated an equation for
representing the steady-state behavior of TCP in it's congestion avoidance mode
[1].  Roughly, achievable throughput for a TCP connection (BW) is related to
Maximum Segment Size (MSS), and inversely related to the Round Trip Time (RTT)
and to the square root of the probability of packet loss (p):

   BW < (MSS/RTT)*1/sqrt(p)

The probability of packet loss in their paper was based on congestion loss, but
works equally well with corruption loss since TCP doesn't distinguish.  If you
add to the probability of congestion-related loss the losses due to bit-errors,
you can get an idea of the bound on throughput that a TCP connection may
experience.  One can use a Bernoulli process as an estimator of packet loss
probability if you assume uniformly-distributed errors (different coding schemes
yield different distributions -- your mileage may vary!).  With a Bernoulli
process, the packet loss probability (due to corruption) is 

   p = (1-BER)**(8*L)

Where L is the length of the (entire) packet in bytes.

One thing to keep in mind is that this equation depends on the *full*
round trip time, not just that of the satellite link.  It's also a function
of the complete probabilty of packet loss (congestion *and* corruption).  
Note that if you mix long RTT connections with short RTT connections (say,
in the wired portions of the Internet), the long RTT connections will lose
to the short RTT connections.  That's one of the things that motivates the
LEO brigade as well as folks who advocate "impedance-matching" gateways.

Note also that this equation applies to the *congestion avoidance* phase of
a TCP connection.  The TCP congestion window, if you're running with delayed 
acknowledgments, opens at a rate of one segment every two RTTs in congestion
avoidance phase and doubles essentially every two round trips (not quite with
long delays) in slow start.  The exponential growth of slow start is important
to get a congestion window open quickly, and you're really disadvantaged if a
TCP connection takes a corruption hit early in slow start on a long RTT, high
(potential) bandwidth path, because it will fall out of slow start and into
congestion avoidance.  This causes a cessation of exponential rate increase and
a commencement of the linear rate increase of congestion avoidance.  Bummer.

Opinion:  Unless you're the only game in town, a commercial satellite provider
would be well advised to pay close attention to the performance achievable by
any single customer's TCP connections.  While it *may* be true that you can
maintain reasonable aggregate utilization of your satellite capacity if you're
supporting hundreds of simultaneous users, the performance seen by each one of
them has a large effect on whether they think that the service is worth the
cost.  Having suffered with very poorly performing trans-Atlantic TCP
connections, I can assure you that given the opportunity, I would have voted
with my feet!  Also, the ability fully utilize the satellite channel via lots of
users isn't necessarily a given, since each user attempting to consume
additional satellite channel capacity must also successfully consume additional
capacity in the wired portions of the Internet.  The availability of SACK is a
Good Thing (tm Martha Stewart, no doubt), but its reduction in retransmission
timeouts means that router buffers will tend to remain full, meaning that
additional capacity to other users will be harder to come by.[2]

Data:  The URL cited by Will, below, is a good reference.  Also at that same
web site is a report that was done using the ACTS satellite.  The URL for that
report is http://www.scps.org/scps/assets/images/SCPS_ACTS97.pdf
Unfortunately, we did not include tables of raw TCP throughput performance in
the document.  Here is the data on TCP corruption performance; this data was
taken between 11 and 14 August 1997 at Rome Labs in New York.

Conditions:  TCP (window scaling, 200k byte send and receive buffers, no SACK),
2Mbps channel (rate 1/2, constraint length 7 coding, transmit power adjusted to
achieve measured BER).

				   Throughput (kbps)
BER		Eb/No Range	1400-byte MSS	512-byte MSS
1.00E-08	6.4-6.6		1480.14 avg	820.52 avg
                Run 1:		1475.6		821.45
		Run 2:		1487.36		819.24
		Run 3:		1481.17		821.69
		Run 4:		1480.74		819.34
		Run 5:		1475.84		820.9
1.00E-07	5.8-6.2		1440.2		757.13
				1478.61		820.36
				1482.89		512.31
				1339.78		819.11
				1477.87		820.27
				1422		813.58
6.00E-07	5.4-5.7		1410.99		671.09
				1479.45		815.04
				1131.43		558.14
				1482.92		433.1
				1481.44		742.71
				1479.69		806.44
1.50E-06	5.2-5.4		527.25		469.62
				449.73		725.14
				520.04		402.25
				697.94		234.97
				383.6		164.94
				584.95		820.82
4.50E-06	4.9-5.1		298.57		207.44
				246.06		230.28
				351.45		191.26
				260.74		159.09
				388.68		240.61
				245.93		215.98
1.20E-05	4.6-4.8		158.74		102.99
				182.78		103.07
				148.47		114.44
				169.59		99.91
				143.64		79.52
				149.22		117.99
5.00E-05	4.1-4.3		71.35		40.77
				79.03		34.9
				73.1		44.69
				61.91		42.71

This same report has performance data for the SCPS transport protocol, which is
described on the SCPS web page (www.scps.org/scps).  Appendix A of that report
discusses the mechanics of a TCP connection as it responds to corruption and
congestion, and does the same for SCPS-TP (the SCPS transport protocol).

Best regards,

Bob Durst
[email-masked]

[1] M. Mathis, J. Semke, J. Mahdavi, T. Ott, "The Macroscopic Behavior of the
TCP Congestion Avoidance Algorithm",Computer Communication Review, volume 27,
number 3, pp. 67-82, July 1997. 

[2] M. Mathis, private communication.

---

From: William D Ivancic <[email-masked]>
Date: Wed, 31 Mar 1999 16:49:52 -0500

>"correspond to that of a commercial satellite"? 

Sorry, I left out the word necessarily between "not" and "correspond".  The
point I was trying to make was that you have to understand the environment
these protocols are operating when considering the data.  
*********************************************
William D. Ivancic
NASA Glenn Research Center
21000 Brookpark Rd.   MS 54-8
Cleveland, Ohio  44135
USA
Phone:   1 216 433 3494
FAX:     1 216 433 8705
Email:   [email-masked]
             [email-masked]	
http://ctd.grc.nasa.gov/5610/5610.html	
*********************************************