Subject: [Tsvwg] SCTP Max.burst and CWND

From: [email-masked]
Date: Fri, 1 Mar 2002 16:45:30 +0100 (MET)

Hello!

For those short in time first the summary of my mail.

Summary: The task of preventing bursts which cause congestion is addressed
by CWND,
but the rules are incomplete at the time.
Rules for decrease of CWND when it is partly used are missing. This would
also prevent 
bursts causing congestion in situations when no fast retransmit takes place.

A workaround of the problem is Max.Burst, but it must be always applied, not
only after
fast retransmission.

Concerning the 'Max.Burst' handling the current SCTP Imp. Guide 
is not very specific.

   Upon acknowledgment of a DATA chunk that has been fast retransmitted,
   the protocol parameter 'Max.Burst' MUST be applied to limit how many
   SCTP packets may be sent upon the completion of SACK processing.

Does this limit the application of Max.Burst only to the first send 
triggered by the first SACK after a FR, or has it a longer effect?

IMHO if we introduce Max.Burst because we think bursts are dangerous 
then we should always apply this limit to any response on a SACK,
independent from 
the 'history'.
The solution description mentions that clamping RWND to 0 over a period of
time is the 
cause for bursts, and I think this may happen most likely but not only in
combination 
with FR.

Let me describe also a more elegant solution to prevent the negative effects
of bursts without the Max.Burst method.
Please correct me if I'm wrong.
The problem is not the burst itself, but the burst after some time of very
low traffic, e.g 
zero window probing.
A second flow sharing bandwith of a limited bandwith link can increase its
CWND while we 
are zero window probing, but our CWND stays the same. 
After release of the RWND=0 condition we use our full CWND which causes
congestion
and losses on the link. So the real problem is, that our CWND doesnt decay
when we use 
only a part of it.
If we cut our CWND in half once per RTT as we do when idle would solve the
problem, 
because we shrink our CWND faster than other flows would increase their
CWND, and so 
we can use our full CWND any time without exeeding the total link capacity.

Another improvement is to define a general rule to decrease CWND when it is
not fully 
used, i. e. flightsize is smaller than CWND.
I could think of something like NEW_CWND=CWND-0.5*(CWND-flightsize).
Imagine the situation of two flows A,B sharing a link equally (50%) . Now A
reduces 
throuput to 10%, B can increase to 90%.
When A swithes back to 50% we have a massive burst and overload, and without
FR 
involved.
With CWND reduction mechanism A cannot switch back immedeately to 50% but
increases 
with less congestion in the network. 

It would be interesting if someone could simulate these suggestions, or if
there are any 
papers dealing with CWND decrease? 

Best regards

Rainer Benedict

-- 
GMX - Die Kommunikationsplattform im Internet.
http://www.gmx.net

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Fri, 1 Mar 2002 15:12:46 -0500 (EST)

Hi Rainer,

I like your idea of decaying cwnd when cwnd isn't being fully utilized. I
am not sure about the formula for decaying though. Intuitively, it seems
good, but I would need to think about it some more. I am interested in
simulating your idea using our ns-2 sctp module. Let's discuss this
offline.

I do not think it solves the problem that Max.Burst addresses though.
Decaying the cwnd isn't the problem, since a fast rtx already cut the cwnd
in half. The problem is more with rwnd. A missing tsn may cause the
receiver's window to fill up. Once the missing tsn is fast rtx'd and
arrives at the receiver, the entire window of tsns sitting in the
receiver's buffer is delivered up to the application. As a result, the
rwnd advertised goes from 0 to some fairly large number, which causes the
sender to send a burst of tsns. Normally, tsns are gradually clocked out
by incoming sacks. However, in the example scenario above, a single sack
triggers a burst of tsns.

Now that I think about it, there seems to be two types of bursts:

1. The sender wasn't fully utilizing cwnd due to lack of data to send.
Then, all of a sudden, the sender has a lot of data to send and sends a
cwnd worth of data in one shot.

2. The sender has plenty of data to send. Cwnd has plenty of room to send
some of this data, but rwnd doesn't allow it. Suddenly, the
rwnd is freed up and the sender can send a cwnd worth of data.

Your solution addresses burst type #1, while Max.Burst addresses burst
type #2.

Any other comments??

~armando

0--                                                --0
| Armando L. Caro Jr.     |       [email-masked] |
| University of Delaware  |  www.cis.udel.edu/~acaro |
0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 1 Mar 2002 15:42:31 -0600

Armando,

> cwnd worth of data in one shot.

So what?  It should be always permitted to utilize the entire cwnd when
permitted by rwnd.

> rwnd is freed up and the sender can send a cwnd worth of data.

So what?  Again one should be always permitted to utilize the entire cwnd
when permitted by rwnd.

> Your solution addresses burst type #1, while Max.Burst addresses burst

If you want to limit the maximum bursts always, clamp rwnd.

This can easily be done by the sender or receiver, or both.  Then the
sender will burst up to rwnd and then probe.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Fri, 1 Mar 2002 17:06:33 -0500 (EST)

> So what?  It should be always permitted to utilize the entire cwnd when

If the cwnd is not being fully utilized for a period of time, then the
cwnd doesn't accurately represent what the sender may send.  The idea is
similar to when a destination's cwnd is decayed while it is idle. The
difference is that the destination isn't completely idle; it's just not
using all of cwnd.

There is a change in the implementor's guide that follows the same
thinking. The guide now includes text that restricts the cwnd from growing
unless the cwnd was being fully utilized.

> So what?  Again one should be always permitted to utilize the entire cwnd

I am not sure about this case. I was simply pointing out that Max.Burst is
trying to address this problem. I was never sure about why burst
type #2 was a problem or why Max.Burst is really needed.

I have been told that the reason it was put in was to avoid "kicking them
while they are down". In other words, the sender is recovering from a
loss, so the sender shouldn't aggravate the situation further by sending a
burst. However, I don't understand why cutting back cwnd isn't enough.

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 1 Mar 2002 16:18:54 -0600

Armando,

> unless the cwnd was being fully utilized.

Well, that doesn't solve much.  If the sender fully utilizes cwnd and
it grows big, and then it sits almost idle, you still get the same
effect.  What would be required is decaying cwnd down to the point that
it is being utilized.  However, that would be far more conservative than
TCP Friendly.  The idea there is not to avoid congestion altogether, but
to back off quickly when it is experienced (as escried by packet loss).
CMIIWB TCP doesn't decay its cwnd under these circumstances either and
a similar burst is possible for TCP.
> burst. However, I don't understand why cutting back cwnd isn't enough.

It should be.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: Randall Stewart <[email-masked]>
Date: Fri, 01 Mar 2002 17:49:50 -0600

> unless the cwnd was being fully utilized.
I agree with Brian on this point. cutting cwnd when it is being
used but not fully used makes no sense to me.. it makes SCTP
over conservative in my mind...

As to the not growing unless cwnd is full.. that was in the
base RFC.. and I think we just fully clearified it in the
implementors guide...

> burst. However, I don't understand why cutting back cwnd isn't enough.
Because cwnd is not the problem here..

I have sent N chunks to the peer and have X more chunks
in queue. But in the last SACK that ack'd all N, I get a
cwnd clamped at 0. So I continually probe with N+1 and
get over and over a SACK... rwnd 0.

All of a sudden my peer reads everything. Notice that we 
never did NOT have data outstanding so we were NEVER idle.
Cwnd does not shrink here... so its up high...

Now when I next send N+1 I get a full rwnd... since the
peer read everything...

Now I have X-1 chunks to send.. I could in theory burst
the whole cwnd onto the wire.. This could be way
to much and end up causing the routers to drop a large
number of my packets (and others to I suppose).. by 
spreading out the first send say into Max.Burst and then
after the SACK (X-1)-Max.Burst . I don't do a immediate flood
of the network but slowly resume my transmissions...

So my burst is spread across 2 RTT... instead of all at once.

cwnd can do nothing to help this situation.. 

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 2 Mar 2002 02:53:22 -0600

Randall,

> cwnd can do nothing to help this situation.. 

But clamping rwnd would.

--brian

---

From: "Rainer" <[email-masked]>
Date: Sat, 2 Mar 2002 12:24:24 +0100

Hi Brian,

See some comments inline...

----- Original Message -----
From: "Brian F. G. Bidulock" <[email-masked]>
To: "Armando L. Caro Jr." <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Friday, March 01, 2002 10:42 PM
Subject: Re: [Tsvwg] SCTP Max.burst and CWND

> sender will burst up to rwnd and then probe.

I agree with you that bursts can easily be prevented.
But I think the burst itself is not a problem, only after some time when
sending at a very low rate, while other flows had the chance to 'steal'
bandwith from us.
CWND is our  indicator, how much bandwith of a link we may use. If we are
idle or sending at low rate for a while this indicator may be wrong and
resuming to send at the full bandwith may be fatal. That is why i proposed
to reduce CWND when it is not fully utilized. It will then grow again in a
well defined way.

 Best regards

 Rainer Benedict

---

From: "Rainer" <[email-masked]>
Date: Sat, 2 Mar 2002 12:12:44 +0100

Hi Armando,

----- Original Message -----
From: "Armando L. Caro Jr." <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, March 01, 2002 9:12 PM
Subject: Re: [Tsvwg] SCTP Max.burst and CWND

> simulating your idea using our ns-2 sctp module. Let's discuss this
I would be interested in simulation results, this behaviour is conservative
and may eventually be a disadvantage compared with TCP. But in my opinion it
can prevent massive link overload.

> triggers a burst of tsns.

But this burst is limited by CWND which ensures that the link is not
overloaded. I dont see which other negative effect a burst of TSNs should
have.
> Your solution addresses burst type #1, while Max.Burst addresses burst
My solution addresses both types. In situation #2 it reduces CWND while RWND
doesnt allow us to send, and so the resulting burst after RWND allows
sending again is limited by the now smaller CWND. The longer we were
blocked, the smaller the burst will be.

 Best regards

 Rainer Benedict

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 2 Mar 2002 13:14:30 -0600

Rainer,

Please see comments below...

> to reduce CWND when it is not fully utilized. It will then grow again in a

This would be more conservative that TCP in the same situation
resulting in SCTP transport users being less able to compete
fairly for bandwidth with TCP transport users.

--brian

---

From: "Rainer" <[email-masked]>
Date: Sat, 2 Mar 2002 20:31:48 +0100

Hi Randall ,

see comments inline...

----- Original Message -----
From: "Randall Stewart" <[email-masked]>
To: "Armando L. Caro Jr." <[email-masked]>
Cc: "Brian F. G. Bidulock" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Saturday, March 02, 2002 12:49 AM
Subject: Re: [Tsvwg] SCTP Max.burst and CWND

> cwnd clamped at 0. So I continually probe with N+1 and

guess you meant rwnd here.

> of the network but slowly resume my transmissions...

What you say here is that Max.Burst is applied ONCE after RWND changes from
0 to some other value.
There is no fast retransmit in this scenario - This is different from the
current wording in the implementors guide.
If bursts are such a problem, why dont apply Max.Burst always ?

But I dont understand why we should NOT reduce CWND while doing zero window
probing on one hand but introduce an additional restriction with Max.Burst
on the other hand. Both is more conservative than TCP is.

> cwnd can do nothing to help this situation..

CWND can help because it limits the burst. If we have only a very short zero
window probing period it should be allowed to use the full CWND. If we probe
for a longer time, we should reduce CWND, because other flows may have
consumed our bandwith and sending a full CWND would produce massive
overload.

Best regards

 Rainer Benedict

---

From: "Rainer" <[email-masked]>
Date: Sat, 2 Mar 2002 20:43:55 +0100

Hi Brian,

see comments below...

----- Original Message -----
From: "Brian F. G. Bidulock" <[email-masked]>
To: "Rainer" <[email-masked]>
Cc: <[email-masked]>
Sent: Saturday, March 02, 2002 8:14 PM
Subject: Re: [Tsvwg] SCTP Max.burst and CWND

> > > > I like your idea of decaying cwnd when cwnd isn't being fully
utilized.
> > I
> > > > Decaying the cwnd isn't the problem, since a fast rtx already cut
the
> > cwnd
> > > > rwnd advertised goes from 0 to some fairly large number, which
causes
> > the

I see the difference only in bursty load situations, under constant full
load both should perform the same.
A simulation may be helpful.

But this means also we should not try to prevent bursts more than TCP does
because this may be a disadvantage in copmetition.

Best regards

 Rainer Benedict

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 2 Mar 2002 14:45:47 -0600

Rainer,

> because this may be a disadvantage in copmetition.

Bursts are already limited by rwnd in both TCP and SCTP.  The
sender will never send more than rwnd worth of data.  In reality
there is no way for the implentation to predict the difference
between a burst and the expected maximum traffic rate permitted
on the link by rwnd.  Max.Bursts and rwnd serve the same purpose.
So who needs Max.Burst?  Are we saynig that at some points in time
the application is not permitted to send up to rwnd worth of data?
Well that is the purpose of cwnd.  But, by clamping cwnd to Max.Bursts
at some times in the life of the association, we will be effectively
clamping rwnd to the same value.

I think that the effects of even temporarily reducing cwnd to
Max.Bursts during probing would have a seriously depremental effect
on bursty signalling applications.  Placing this additional delay
is unwarranted.

The concept of congestion control for the Inetenet is ramp up slow
(linear), back off fast (exponential).  No attempt is made to avoid
packet loss, just to respond by wuickly backing off when it is
detected.  There is no reason to not let an application send at the
maximum throughput rate offerred it immediately when cwnd and rwnd
permit.

--brian

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Sat, 2 Mar 2002 16:54:05 -0500 (EST)

> maximum throughput rate offerred it immediately when cwnd and rwnd

Well, you are right... but ONLY if cwnd has a current notion of what the
network can handle. Suppose the following...

1. An existing connection over time builds its cwnd up to say 16 MTUs.
Suppose the rwnd is abitrarily large and isn't the limiting factor.

2. The application has very little to send. As a result, only 1 MTU is
sent each RTT. For sake of example, this low sending rate continues for
about 60 secs. During these 60 secs, cross traffic may increase and 16
MTUs may not be appropriate any more.

3. After the 60 secs, the application has a LOT to send and tries to send
a full cwnd worth of data (ie, 16 MTUs). As a result, a congestion induced
loss occurs.

So your argument is basically... so what? the cwnd will back off quickly
and all will be fine. Well, suppose we go with that argument. Then your
argument also supports the idea of starting a fresh connection with an
intial cwnd of 16 MTUs (or some other arbitrarily large value). So
according to your argument, slow start is a bad idea. Is that what you are
saying?

~armando

0--                                                --0

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 2 Mar 2002 18:17:12 -0600

Armando,

> MTUs may not be appropriate any more.

Or not.  That is, cross-traffic might not increase and the original cwnd
is completely applicable.

> a full cwnd worth of data (ie, 16 MTUs). As a result, a congestion induced

Good, so cwnd backs off exponentially.  That's how cwnd works.

> according to your argument, slow start is a bad idea. Is that what you are

No.  Slow start is a different situation on initial associatian setup.
It is necessary to get the congestion algorithm going and to avoid
initial transients.  Even still, it is an heuristic.  1 MTU might be
too large.

> about 60 secs.

During these 60 seconds nothing changes in the network.

3. The application has a bunch of ISUP signalling traffic to send.  But
is limited to Max.Burst for an RTT.  RTT might be rather large compared
to the expected response time on the ISUP messages.  A bunch of calls
time out for no reason.

Consider the same for backing off cwnd.

> about 60 secs.

During these 60 seconds nothing changes in the network, but according to
some new rule, cwnd is backed off.

3. The application has a bunch of ISUP signalling traffic to send.  But
is limited to cwnd for RTT until cwnd grows again.  RTT, again,  might
be rather large compared to the expected response time on the ISUP messages.
A bunch of calls time out for no reason.

Sorry, but I still see no reason to disadvantage the application because
for a short period it is sending some packets and NONE of them are getting
lost (i.e., there is NO NETWORK CONGESTION).  This is far different from
the rule that say when an application sends NO packets for a period of time
that Slows Start should be applied to the connection.

Both in the idle period and the initial establishment phases, the assocation
has *NO* indication of the newtork congestion to that desination and slow
start is applicable.  That is, the application must approach the newtork as
though it were congested because it has no information.

When an appliction is almost idle or probing it knows that there is NO
NETWORK CONGESTION because it is not experiencing packet loss (or it is
experiencing packet loss anyway and backing off cwnd).

Backing off cwnd or applying a Max.Burst is overkill and will disadvantage
SCTP from transports and other IP network users even in times where no
network congestion is occuring.

--brian

---

From: Randall Stewart <[email-masked]>
Date: Sun, 03 Mar 2002 08:52:36 -0600

> guess you meant rwnd here.
Opps.. yep sorry.. typo on my part..,

> What you say here is that Max.Burst is applied ONCE after RWND changes from

Yes and no :> It is applied ONCE yes after RWND changes from 0 to
some other value ONLY if it is a FAST-RETRANSMIT completing. At least
this is what Ivan and I were thinking. The idea of Max.Burst
comes from some of Sally Floyd's work.. if I remember right.. I
will have to dig up the paper cite.. in fact we should site it
in the implementors guide... 

It was meant to only apply to F-R..

> If bursts are such a problem, why dont apply Max.Burst always ?

Hmm...  I don't see the value of this in the case of a T-O (the
other scenario where one could get into this situation). This is
because in the time-out scenario the cwnd is set a 1 MTU so it
would be rather difficult to BURST much out :->

For Probing .. its a whole different realm.. one keeps the cwnd at
the same level... we might want to think about this here.. but I 
need to go back and dig up Sally's paper and re-read it.. a bit
hard for me at connect-a-thon.

The wording in the current implementors guide is MEANT to restrict
the max.burst to ONLY be applied at FR.. If this is unclear we
need to fix it...
> on the other hand. Both is more conservative than TCP is.

Why should we reduce cwnd.. I see no need. you are keeping the ack-clock
going with the probes... so everything is pretty much up to date.

And TCP can never reach this situation... unless SACK is in the
TCP.. and this is where max.burst comes from in Sally's paper...(again
if my early coffee starved mind remembers right :>).

> consumed our bandwith and sending a full CWND would produce massive

Again.. the restriction is meant ONLY for FR not zero window probes. If
that is unclear we will fix it in the implementors guide..

R