Subject: Re: [Tsvwg] sctp performance

From: Randall Stewart <[email-masked]>
Date: Thu, 07 Oct 2004 08:50:02 -0400

Brian/Salvatore:

Just a couple of comments...

Brian F. G. Bidulock wrote:
> Salvatore,
> 
> Performance tests on a clean network do not exercise SCTP's strong
> points.  SCTP was not really intended to outperform TCP on a clean
> network with a single interface: it was intended to outperform TCP on a
> real network with noise, failures and reconfiguration.  If you look at
> even single stream tests under noise SCTP rapidly outperforms TCP in
> dirty networks.
> 
> The SOCK_SEQPACKET test deviation from theoretical is probably more of a
> software glitch in bundling, Nagle'ing and Minshall'ing in the OpenSS7
> stack more than a problem with the protocol (that was noted in the
> results).

I would tend to agree with Brian here.. SOCK_SEQPACKET is really
an implementation shade.. In KAME other than controlling a
few minor things all the same code runs... so it sould make
very little difference....

> 
> The SOCK_STREAM tests on a single interface did outperform TCP, however,
> note that the tests saturated the network and both are running very
> close to theoretical maximums.
> 
> The interesting thing about the OpenSS7 tests were the multiple
> interface tests on SOCK_STREAM, that showed that max throughput was a
> multiple of the number of available interfaces.  Once TCP saturates
> a single interface it has nowhere else to go.  SCTP can load multiple
> interfaces to saturation.
> 
> These were throughput tests.  Latency tests show SCTP has comparable
> results to TCP (with slight improvements) on clean networks, and
> SCTP exhibits significant reduced latencies to TCP under even as little
> as 5% noise.  On distrupted networks (50% interface failure on a 1 or
> 2 second duty cycle) TCP stops and SCTP continues.
> 
> There are a lot of good papers out there testing more real environments
> as well as NS-2 simulations that demonstrate the benefits of SCTP over
> TCP for its original intended application, telephony signalling.  Search
> on "SCTP Performance" and you can read for the better part of a day.
> 

A couple of other things:

a) The CERN tests were done on an OLD OLD OLD version of lk-sctp which
    was NOT performance tuned at all.. I am not even sure if they have
    completed a pass yet of tuning.. I know its on their to-do list.

b) IPERF ... which is used on both sites.. to me.. is NOT a very good
    benchmark. SCTP sends "messages" from one end to the other. TCP sends
    a byte stream from one end to the other. You then take and use IPERF
    to test the number of bytes streaming from one end to the other.
    This puts SCTP at a dis-advantage. Especially considering that MOST
    ALL applications send messages NOT streams of bytes. So if you added
    some additional overhead to TCP.. say framing messages at the app
    layer.. this would slow TCP down I would wager.

    I have always wanted to develop a variable message sender that could
    send a sequence of variable messages and get a reading of messages
    per second for various sizes....

c) CRC32c is extremely expensive (in overall terms of CPU use) on the
    TCP side they generally have an off-load engine they can use to
    not even do the c-sum... so in such a case when you begin to
    get CPU limited SCTP will lag behind TCP... just on the checksum
    basis alone. Bottom line is once you get CRC32c offload SCTP will
    be able to do a bit better :-D

Regards

R

> --brian
> 
> On Thu, 07 Oct 2004, Salvatore Loreto wrote:
> 
> 
>>   Hi all,
>>
>>
>>
>>   reading the experiments on SCTP performance
>>
>>
>>
>>   LKSCTP
>>
>>   [1]http://datatag.web.cern.ch/datatag/WP3/sctp/tests.htm
>>
>>   OPENSS7
>>   [2]http://www.openss7.org/sctp_perf.html
>>
>>
>>
>>   If  we  ignore  the  gap  underline between  TCP  and  SCTP in LKSCTP,
>>   becose due   to   implementation   problem   ( and  I  think  now  the
>>   situation is  better,  isn't  it? does someone have up to date results
>>   about?)
>>
>>   and use just the result underline on OPENSSS7
>>
>>
>>
>>   I've some questions/doubts and I'd like know you opinion about:
>>
>>
>>
>>   1)  the  test  SCTP with 1 interface shows a substanzial equivalent to
>>   TCP.
>>
>>   Is  it  possible SCTP with 1 interface is TCP, or if you want if there
>>   is    not HoL block there are not difference between two protocols due
>>   to the same congestion algorithm.
>>
>>
>>
>>   2)  SCTP  with  2  interface has a better performance just for message
>>   size greater than 64-byte.
>>
>>   But  SCTP was born for Signaling transport, and signaling messages are
>>   yes lager than 64-bye but not very large.
>>
>>
>>
>>   when in the experiment is used SOCK_SEQPACKET
>>
>>   3)  the  real  start up with 1 or 2 interfaces are both worse than the
>>   Theoretical  one.  The  strange  is the reale one with 2 interfaces is
>>   worse than the reale one with just one interface.
>>
>>   4)  similar  considerations  ( done above ) could be done also for the
>>   other phases . So SOCK_SEQPACKET seems not so good.
>>
>>
>>
>>
>>
>>   regards
>>
>>   sal
>>
>>
>>
>>
>>     _________________________________________________________________
>>
>>   [3]Yahoo!  Mail  ti  protegge  gratis  dalla  posta indesiderata e dai
>>   virus!
>>
>>References
>>
>>   1. http://datatag.web.cern.ch/datatag/WP3/sctp/tests.htm
>>   2. http://www.openss7.org/sctp_perf.html
>>   3. http://it.rd.yahoo.com/mail/taglines/*http://it.mail.yahoo.com
> 
> 
>>_______________________________________________
>>tsvwg mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/tsvwg
> 
> 
> 

-- 
Randall Stewart
ITD
803-345-0369 <or> 815-342-5222

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 7 Oct 2004 17:25:11 +0200 (CEST)

Hi Randall
 
do you have some report or paper where I can find information
about SCTP in  KAME  ... 
I read it includes HighSpeed SCTP, is it true?
Can I have more information about?
 
thanks in advance
/Sal

Randall Stewart <[email-masked]> wrote:
Brian/Salvatore:

Regards

R

-- 
Randall Stewart
ITD
803-345-0369 815-342-5222

---------------------------------
Nuovo Yahoo! Messenger E' molto più divertente: Audibles, Avatar, Webcam, Giochi, Rubrica… Scaricalo ora!

---

From: Pawel Hadam <[email-masked]>
Date: Thu, 07 Oct 2004 16:03:33 +0200

(I am sorry for my previous, signed e-mail)

Hi

 >    completed a pass yet of tuning.. I know its on their to-do list.

I was working with linux kernel 2.6.7 with lkSCTP 1.0.1 and I obtained 
much better results.

 >    per second for various sizes....

Sounds like my own testing apps. I wrote a sender and receiver that 
accept some customization parameters (message size, buffers).

I wrote a raport about my measurements and experiments with lkSCTP on 
1Gb network, but:

1) It is now being read by my supervisors.
2) It is in french :) - I really am sorry, it was written for a french 
project.

Once the raport finished, I will, I hope, try to publish something in 
this subject.

If anybody wants the french version, I may send him the PDF (there are a 
lot of pictures :))). Any comments are always welcome.

Regards
Pawel

---

From: Pawel Hadam <[email-masked]>
Date: Thu, 07 Oct 2004 15:25:56 +0200

Hi

Regards
Pawel

---

From: Pawel Hadam <[email-masked]>
Date: Fri, 08 Oct 2004 12:55:42 +0200

Hi all

Together with my report I did a presentation. It is in english, and it 
presents the same work and the same results as described in the report.

http://www-lsr.imag.fr/Les.Personnes/Pawel.Hadam/work/index.html
->
"Multihoming & SCTP"

Available as PPT and PDF

Any comments are welcome.

Regards
Pawel

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 8 Oct 2004 15:19:45 -0600

Pawel,

Take a look at

  http://www.omg.org/news/meetings/workshops/RT_2003_Manual/Presentations/5-4_Thaker_etal.pdf

which presents failover results nominal 10ms in test situations similar
to your full mesh experiment on slide 35 of your presentation and also
show much better 1% packet loss latencies.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Wed, 13 Oct 2004 14:26:09 -0400 (EDT)

Sorry for reviving an old thread, but I'm just getting caught up on this
list.

> interfaces to saturation.

I am curious about SCTP is able to increase throughput as the number of
interfaces increase. Jana has showed that simply using all paths with SCTP
is counter-productive. The transport has to be "smart" about getting
around different issues when doing Concurrent Multipath Transfer (CMT).
Jana has found several issues with "naive" CMT, but has done a nice
job of addressing them one by one. He's still in the process of tackling
more of these issues.

http://www.cis.udel.edu/~iyengar/publications/2004.spects.iyengar.pdf
http://www.cis.udel.edu/~iyengar/publications/2004.icon.iyengar.pdf

~armando

0--                                              --0
| Armando L. Caro Jr.  |  Protocol Engineering Lab |
| www.armandocaro.net  |    University of Delaware |
0--                                              --0