Subject: Re: [Tsvwg] SCTP Max.burst and CWND (fwd)

From: "Armando L. Caro Jr." <[email-masked]>
Date: Tue, 5 Mar 2002 15:53:27 -0500 (EST)

I have sent 2 mails with attachments that seem to not make it through to
the mailing list. So I have sent the graphs to the people actively
involved in the discussion, but if anyone else is interested in seeing the
graphs... let me know.

The original mail is below...

---------- Forwarded message ----------
Date: Tue, 5 Mar 2002 15:44:17 -0500 (EST)
From: Armando L. Caro Jr. <[email-masked]>
To: Mark Allman <[email-masked]>
Cc: Randall Stewart <[email-masked]>,
     Rainer <[email-masked]>, [email-masked]
Subject: Re: [Tsvwg] SCTP Max.burst and CWND

On Tue, 5 Mar 2002, Mark Allman wrote:

> proposals I have seen for maxburst all involve *implicit* limiting
> of cwnd.  My opinion is that we should get rid of this maxburst
> notion and make the limit to cwnd *explicit*.  Having two controls
> is adding complexity and possibly (subtle) interactions that do not
> need to be introduced to get the desired effect.

Yes, I agree. However, if the sender were to go back to slow start, I am
sure that many would be concerned that SCTP would act too conservatively
in comparison to TCP. So I suggest a compromise... The sender goes back to
slow start with an intial cwnd of 4*MTU. In other words, when a SACK
arrives which acknowledges a fast rtx'd tsn, the following should be done
(instead of applying Max.Burst):

	cwnd = 4 * MTU
	ssthresh stays the same

I have attached 3 graphs for comparison:

	1. noBurst.ps     - applies Max.Burst=4 _all_ the time
	2. noBurst-ss.ps  - goes back to slow start w/ cwnd = 2 * MTU
	3. noburst-ss4.ps - goes back to slow start w/ cwnd = 4 * MTU

~armando

0--                                                --0
| Armando L. Caro Jr.     |       [email-masked] |
| University of Delaware  |  www.cis.udel.edu/~acaro |
0--                                                --0

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 5 Mar 2002 15:16:26 -0600

Armando,

I still don't understand why the normal cwnd and rwnd rules are not
enough of a limit in this case.  Could someone please explain it to
me?

When the association is opearting normallin and cwnd > rwnd, the
*application* is permitted to burst a full rwnd worth of packets.
What makes the FR case special?

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Tue, 5 Mar 2002 16:34:13 -0500 (EST)

> What makes the FR case special?

Cwnd was intended to give a limit to how many bytes are allowed in flight.
Slow start is used as an initial probe which later gets fine-tuned by
congestion avoidance. But slow start is _also_ used to gradually build the
sending rate up to a steady flow of data/acks flowing in both directions.
Cwnd was not originally intended to limit the line rate burst.

So with FR, it is possible that a gap in rwnd gets filled. As a result, a
large amount of rwnd is free'd up, which _could_ trigger a burst from the
sender. The burst could degrade performance of all flows passing through
the bottleneck router(s).

I hope this clears things up...

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 5 Mar 2002 16:07:34 -0600

Armando,

> I hope this clears things up...

I don't see the difference between this an the situation where the amount
of data to send jumps from 0 to rwnd.  That is, where the application has
nothing to send briefly and then sends >rwnd worth of data.  It appears to
me that the application (when permitted by cwnd) can always burst a full
rwnd worth of data to the line.  How would this not degrade performance
under normal circumstances in the same way?  So what if FR is completing
due to, say, a packet lost due to CRC error?  Why should the application
be limited in the amount of data it can burst to the line more strictly
than under normal circumstances?

What is the rationale for throttling bursts to less than rwnd after FR
when bursts are not throttled without FR?

--brian

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Tue, 5 Mar 2002 17:40:03 -0500 (EST)

On Tue, 5 Mar 2002, Brian F. G. Bidulock wrote:

> nothing to send briefly and then sends >rwnd worth of data.

Depending on how long "briefly" is, there may be a problem in this
scenario too. The scenario you describe is what RFC2861 refers to as
application-limited.

> It appears to me that the application (when permitted by cwnd) can
> always burst a full rwnd worth of data to the line.  How would this
> not degrade performance under normal circumstances in the same way?

It can if the burst is too big. Until now, we have been using the magic
number 4 as what the max burst should be. There probably needs to be some
investigation as to what the max burst allowed should be. (Maybe there is
and I don't know about it?)  Anyway, 4 seems reasonable. Under "ideal"
circumstances... the sender is network-limited, no loss, delayed sacks,
etc... a maximum burst of 3 packets are sent for an incoming sack.

> So what if FR is completing due to, say, a packet lost due to CRC
> error?

This is a different issue altogether. Determining between congestion
versus noise induced loss has always been a problem with TCP-style
congestion control.

> Why should the application be limited in the amount of data it can
> burst to the line more strictly than under normal circumstances?

Well, part of the problem is that some of the "normal" circumstances have
issues that should be addressed too. That is why I am arguing that we
should really address the issues and solutions presented in RFC2861.

> when bursts are not throttled without FR?

The cwnd is suppose to reflect the bandwidth-delay product... NOT just
bandwidth. To give you a better understanding, I think you should probably
read RFC2861 and Van Jacobson's paper Congestion Avoidance and Control
(especially sections 1 & 2 which talk about obtaining and conserving
equilibrium).

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 5 Mar 2002 16:50:05 -0600

Armando,

Thanks for the explanation.  I confess that I have not read
RFC 2861.  I will go away and read it now...

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 5 Mar 2002 21:55:43 -0600

Armando,

I went an read RFC2861, but I didn't get past the "Status of
this Memo" which reads:

   This memo defines an Experimental Protocol for the Internet
   community.  It does not specify an Internet standard of any kind.
   ...

Certainly the TSVWG was not considering applying any of the
procedures of this "Experimental Protocol" to SCTP which is a
"Proposed Standard", was it?

I have some additional questions...

> scenario too. The scenario you describe is what RFC2861 refers to as

For the cwnd to grow to rwnd and permit this to occur, the
application must have already successfully sent a full rwnd
worth of data within about an RTT.  Why limit it now?

> etc... a maximum burst of 3 packets are sent for an incoming sack.

Why?  cwnd already limits things to the maximum burst that was
successful in the past?

> versus noise induced loss has always been a problem with TCP-style

The FR algorithm requires the application to back off (cwnd/2)
before the retransmission.  This means that the burst is
already limited to no more than rwnd/2.

> should really address the issues and solutions presented in RFC2861.

RFC2861 is an "Experimental Protocol".  You are not suggesting
that we apply its experimental concepts to a "Proposed
Standard," are you?

> (especially sections 1 & 2 which talk about obtaining and conserving

cwnd under non-congestive conditions per RFC2960 reflects the
maximum burst per RTT+Te (where Te is the emission time
normally Te<<RTT) that was experience without packet loss in
the past.  As such, cwnd is no greater than the latest and
greatest successful burst.  Upon FR this value is halved
already.  There is certainly no reason to assume that the
network cannot handle 1/2 of the last successful burst and
doing the back-off calculation before FR ensure that the
back-off rate is even greater than under normal conditions.
The back-off of cwnd is cutting it in half within about an RTT
(again RTT+Te) instead of the RTO of 1 second minimum which is
the normal back-off rate for cwnd.

So, I put it to you that FR is already burst limited to a far
greater degree than under normal transmission (cwnd is cut in
have within RTT+Te instead of RTO), the burst can absolutely
be no greater than rwnd/2, and the sender must have already
bursted a full rwnd to have the effect mentioned in this
thread.  So, the sender sends rwnd worth of data (permitted by
cwnd == rwnd) and loses a packet which results in fast
retransmission which opens a_rwnd, permitting the sender to
send only cwnd/2 <= rwnd/2 immediately following the
restransmission if a_rwnd == rwnd.  If multiple packets are
missing and another SACK triggers the FR of another data
chunk, the cwnd could close to slow start in less than an RTT.
If this is the last FR, the cwnd has to grow an MTU per RTT+Te
back up from again no more than rwnd/2 and then only if the
application has more data to send after this hypothetical FR
event.

The very occurence of an FR event is an indication that the
network has only suffered marginal and transient congestion.
Otherwise, one would expect that all packets would be lost
after a certain point.  It makes perfect sense to test the
network with cwnd/2 after the FR event and any maximum burst
limit would be excessively conservative.

Until RFC2861 achieves some standard status with TCP, there is
no reason to disadvantage SCTP.  In fact, there is good reason
to not disadvantage SCTP wrt TCP: SCTP applications expect
higher performance from SCTP than is experienced using TCP.

The Max.Bursts should be removed from the Imp Guide.  RFC2960
is already very conservative on the FR cycle (cutting cwnd to
cwnd/2 even though an RTO has not occured) in comparison to
the normal packet loss case (where cwnd backs off only every
RTO).

--brian

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Wed, 6 Mar 2002 09:24:14 -0500 (EST)

> Why?  cwnd already limits things to the maximum burst that was

Brian, I know where you are coming from... I didn't understand at first
either and asked the same question. Later, I realized why. Cwnd's purpose
it to maintain an equilibrium in the entire roundtrip flow. Conserving the
flow, the sender sends a new packet every time a packet is taken off the
flow.

Think of it this way... We know that congestion induced drops occur
because packets get dropped at routers with full buffers. Now if cwnd says
the sender can have 16 MTUs in the flow, the intention is that the entire
roundtrip flow can have 16 MTUs outstanding... spreading them over various
routers along the way. If the sender bursts 16 MTUs, the first router will
get hit with 16 MTUs, which it may not be able to handle.

The following picture illustrates a simplified view of what a cwnd = 16
MTUs is really intending to do. There are 4 intermediate routers in both
directions. Also keep mind that I am only showing the packets queued from
this connection. The routers will have packets queued from other cross
traffic.

+--------+ ---> 16, 15 ---> 14, 13 ---> 12, 11  ---> 10, 9 ---> +-------+
| sender |                                                      | recvr |
+--------+ <--- ack 2  <--- ack 4  <--- ack 6   <--- ack 8 <--- +-------+

Now, if we allow a burst of cwnd = 16 MTUs, we are allowing the following
picture. They are obviously NOT the same picture. A lot of strain is
put on the first router. If the first router is not the bottleneck and
these packets happen to all get through, the strain will propogate across
to the next routers. When the strain hits the bottleneck router which
cannot handle the entire burst PLUS cross traffic, packets WILL be
dropped. All flows lose out.

+--------+ ---> 16,15,14,13,12,11 ... 3,2,1 ---> ---> ---> ---> +-------+
| sender |                                                      | recvr |
+--------+ <---        <---        <---         <---       <--- +-------+

Hope this helps...

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 6 Mar 2002 11:23:51 -0600

Armando,

What kind of queuing model are you using for this "bottleneck"
router which is incapable of buffering at line rate?  What kind
of queueing model are you using for the upstream router which
is not smoothing its output, but avoids sending bursts to this
brittle "bottleneck" router?  I don't think that it is the host's
responsibility to deal with these things.  Bursts (busy periods)
will develop between hosts all the time (it is the nature of
traffic).  I don't know where you found this "bottleneck" router,
but I suggest you remove it from your network: even a cheap PC
with a budget NIC can handle line rate input.

--brian

---

From: Janardhan Iyengar <[email-masked]>
Date: Wed, 6 Mar 2002 14:18:00 -0500 (EST)

Brian,

> with a budget NIC can handle line rate input.

Are you suggesting that there are no bottlenecks at routers on the
Internet?

I think Armando's example was to illustrate the difference between having
n TSNs outstanding on the roundtrip path as against having all the n
TSNs on the first hop.. throwing a burst at a router which is already
struggling to cope with the incoming traffic would certainly matter.

regards,
jana

> > this connection. The routers will have packets queued from other cross

-----------------------------------------------------------------------
Janardhan R. Iyengar				   [email-masked]
University of Delaware		       http://www.cis.udel.edu/~iyengar
-----------------------------------------------------------------------

---

From: Jacob Heitz <[email-masked]>
Date: Wed, 06 Mar 2002 11:32:01 -0800

How much should the burst be slowed down ?
I think it should be slowed down to as much as the
router can handle and not much slower.
Let's say the router can handle line rate.
Then the burst needs to be slowed down to line rate.
No problem. The line already does that.

-- 
// Jacob Heitz. Tel:510-747-2917

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Wed, 6 Mar 2002 14:43:51 -0500 (EST)

> router can handle and not much slower.

I think if the sender could know this... all LOT of things would be much
easier. :-)

> No problem. The line already does that.

If routers could route all incoming traffic at line rate (of all the
incoming lines), then they wouldn't need buffers. We already know that
bursts are bad for routers, so let's try and avoid them.

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 6 Mar 2002 13:58:14 -0600

Janardhan,

Please define "struggling to cope with incoming traffic".

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 6 Mar 2002 13:57:34 -0600

Armando,

If the router can't keep up it can't keep up.  This will
be true even if bursts are smoothed.

--brian

On Wed, 06 Mar 2002, Armando L. Caro Jr. wrote:

> easier.:-)

---

From: Janardhan Iyengar <[email-masked]>
Date: Wed, 6 Mar 2002 15:15:46 -0500 (EST)

Hi Brian,

> Please define "struggling to cope with incoming traffic".

I was thinking about a router whose buffers are full and is dropping
packets because the cumulative incoming traffic rate is high.

regards,
jana

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 6 Mar 2002 15:06:43 -0600

Janardhan,

> packets because the cumulative incoming traffic rate is high.

If applictions or hosts are bursting against a full queue, neither
the fairness of the queueing nor the probability of packet loss
changes significantly from smoothed bursts.  Run the model.  You
are saving nothing in terms of bottleneck router efficiency (it
will still pass close to its rated throughput) and yet by smoothing
bursts you are delaying application packets beyond what is required
if no such bottleneck exists.  And by smoothing application bursts
does not stop bursts that develop as part of the arrive process
between applications and between hosts and between routers.

These are QoS issues way outside of what SCTP or TCP needs to
address.

--brian