Subject: Re: [Tsvwg] F-RTO and SCTP (was: SCTP Timestamps)

From: "Randall R. Stewart (home)" <[email-masked]>
Date: Mon, 04 Aug 2003 07:19:06 -0500

Kacheong Poon wrote:

> Pasi Sarolahti wrote:
>
> > And after the examples shown by Sourabh, I believe detecting spurious
> > RTOs with multihoming brings interesting questions for both F-RTO and
> > Eifel that I need to study more thoroughly.
>
> I guess after all the discussions so far, SCTP implementors should
> think about not to follow what is recommended in RFC 2960, timeout
> retransmission to an alternate address.  Just a suggestion, it seems
> that at least for the first timeout, the retransmission is probably
> better sent to the original address.  And when this first retransmission
> is also lost leading to another timeout, SCTP should start considering
> using an alternate address.
>
Kacheong:

Slowly catching up on a HUGE backlog of email :-o

As to the above.. I don't agree yet.. Armando has proposed such  to me
and I have yet to be convinced....

It may be that it is better in some cases.. but when you start factoring in
real failures of a NIC you may find that your failure detection starts to
be effected in a adverse way. So far I think Armando has a good point
in that possibly FR's may be best NOT to send to an alternate .. but
T3's I think may still be best sent to an alternate...

Some of Armando's upcoming work will hopefully settle this :>

[now back to catching up on the timestamp thread... and then I will comment
 on that :> ]

R

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Caitlin Bestler" <[email-masked]>
Date: Mon, 4 Aug 2003 09:13:49 -0500

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Randall R. Stewart (home)
Sent: Monday, August 04, 2003 7:19 AM
To: Kacheong Poon
Cc: [email-masked]
Subject: Re: [Tsvwg] F-RTO and SCTP (was: SCTP Timestamps)

Kacheong:

R

--------------------------------------

I think the central question here is what is the assumed
characteristics of each leg of the path. The current SCTP
path algorithm is optimized on the assumption that a drop
represents a problem on the path attempted - congestion
or failure.

So is that a reasonable assumption? And more to the point,
if there are special characteristics of a final hop that
produce more drops, is it appropriate for them to be solved
by end-to-end retransmission?

At least some of the arguments for changing the path
selection algorithm have cited environments where the
final hop is wireless. I do not believe it is a wise
strategy to correct a higher error rate over a wireless
link with an end-to-end retransmit. Wirelss systems are
prone to burst noise. This will be interpeted by the
end-to-end flow control algorithms as severe congestion
and result in excessive rate trimming.

If each hop brings itself to a reasonable standard of
probable delivery, and then the consensus is that a single
drop probably does not represent a problem with a path,
then it might be a good idea to change the algorithm.
But I do not think that changing an end-to-end algorithm
is a good response if the real problem is confined to
a single hop.

Caitlin Bestler - http://asomi.com/

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 05 Aug 2003 13:30:03 -0700

> T3's I think may still be best sent to an alternate...

If I remember correctly, the fast retransmit case was discussed
some time ago in either tsvwg or sctp-impl.  The consensus is to
retransmit to the original address.  I thought every implementations
is already doing this...

I guess NIC failure happens very infrequently when compared to
congestion and spurious timeout.  I don't think it is a good
suggestion to design the timeout retransmission around an uncommon
case.  And it seems to me that the correct way to handle a NIC
failure is to use ASCONF to tell the peer to either remove those IP
addresses associated with the failed NIC or to change the primary
path if it is to the failed NIC.  One may ask why such mechanism
is not included in the original spec...

-- 

						K. Poon.
						[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Tue, 05 Aug 2003 16:33:30 -0500

Kacheong Poon wrote:
...
> congestion and spurious timeout.  ...

It depends :-) For example, congestion is only a remote possibility in
many (if not all) commercial telecomm networks which is THE primary use
area for SCTP at present time (e.g., SIGTRAN, 3GPP). This is due to the
over-provisioning of bandwidth in all such networks. However, in those
types of networks, one of the key system requirements is the five 9's
system availability (99.999% availability). The only way to achieve five
9's is to use fault tolerant design including redundant NICs and
cabling. But those extra cost spent on redundant hardware will only be a
waste if fault recovery is too slow. In such cases, retrans to
alternative is very effective in speeding up fault recover.

regards,
-Qiaobing

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 05 Aug 2003 14:43:30 -0700

> alternative is very effective in speeding up fault recover.

Then in this specific environment (well, SCTP is supposed to
be a general purpose transport protocol afterall), I'd suggest
those SCTP stacks to support ASCONF so that the retransmission
can automatically be made to an alternate address.  This seems
to be a better way to handle it than relying on a "guess" of NIC
failure as there is no need to guess at all.

Is this a problem to have such requirement on those stacks
operating in the telecomm world?  My guess (we are all guessing (-:)
is not...

-- 

						K. Poon.
						[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Tue, 05 Aug 2003 17:23:12 -0500

> operating in the telecomm world?  My guess (we are all guessing (-:)

K.

Are we **absolutely** sure that whatever we intended to solve in this
discussion has to include giving up this "retrans to alternative"
behavior which is so key to telecomm folks? Is there a solution that
will let those folks to co-exist in this SCTP world while we are
perfecting its congestion control here?

I am not ready to wave the white flag yet. I'd encourage the CC
perfectionists to continue work and find a solution that will satisfy
the CC behaviors while not sacrificing a very real advantage of SCTP for
telecom folks. Then we would have a WIN-WIN solution and I think we can
do it! (I know I sound a little like a promotional speaker, my apology
:)

regards,
-Qiaobing

---

From: Qiaobing Xie <[email-masked]>
Date: Tue, 05 Aug 2003 17:47:02 -0500

Kacheong Poon wrote:

> ...I'd suggest

So you want to rely on an external monitor/detector mechanism at the
peer endpoint that will tell the peer SCTP stack a NIC is bad and then
the remote stack will do a "change primary addr" through ASCONF so that
the local endpoint will start to move the traffic to an alternative dest
addr? Ok, now you have no need to guess since the remote external
detector is doing the "guessing" :-)

But this would essentially remove most of the SCTP's auto path failure
detection / recovery capabilities and take out a big chunk of SCTP's
multi-homing advantage. I think this is too costly for whatever we are
trying to gain here.

regards,
-Qiaobing

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 5 Aug 2003 23:19:55 -0600

Qiaobing,

I agree.  And please do not forget that SCTP was first designed to
meet a set of requirements for transporting telephony signalling
protocols.  If those requirements fail to be met by SCTP, yet another
transport protocol will become necessary again.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: "Randall R. Stewart (home)" <[email-masked]>
Date: Wed, 06 Aug 2003 09:12:11 -0500

> retransmit to the original address.  I thought every implementations

Wow.. I must have missed that discussion or forgotten it... Armando
and I have been haggling on this point for a while..

I think we now have definite proof that for FR's you should send to
the original address. For Time-outs however, this is not necessarily 
true. Armando
still has some more simulation to do in this vein and I have also asked him
to take into account the failover i.e. the cases where where you are
transmitting as failed.

I don't see a CC problem in all of this.. at least none that I can find. 
If you
do take a false T-3 you MAY end up retransmitting to an alternate to quickly
and when the ACK's come back credit the alternate... but is this really 
going
to happen? I think most of the times it will NOT..
aka:

Primary------cwnd=N-------------->
Alternate----cwnd=2Mtu---------->

Now assuming N is some larger number (we have been sending a while)... 
and we do
a encounter a delay spike that causes a false T3 on Primary..

You retransmit 1 MTU to Alternate and wait for a SACK.

The SACK arrives (just a bit latter) from your false T3. You then do for the
alternate

If (flight-size > cwnd) {
    if(in-ss)
         cwnd += min(1 mtu, ack-size)
    else
        pba += acksize;
        if(pba > cwnd)
               cwnd += 1 MTU
               pba -= cwnd
               if(pba <=0)
                   pba = 0
}

Now in our example guess what? flight-size = 1MTU (assuming you had
a full MTU or more to retran)... so the "if" would never be entered and
NO cwnd advance would happen.

It is hard for me to imagine a situation where you would really get the
alternate above 1MTU in flight size .... so I doubt there is a problem.. 
in fact
the real issue is the false T3 will have caused a collapse of the
primary... I guess this is what Eifel attempts to fix in TCP....

So

1) I see no CC problem with the current scheme
2) We probably need to NOT retransmit FR's to an alternate
3) It is best, I think, if we schedule a HB on the alternate after
    a retransmission.. Armando is still working on this point but
    if you have any loss on the alternate you need some way of
    collapsing the RTO so scheduling a HB can do that to help
    performance....

I am anxious for Armando to get a bit further in his research and I think
this will shed some additional light on the situation :>

> is not included in the original spec...

Well, we DID want both PR-SCTP and ADD-IP in the base spec... but in order
to not hold things up to long we decided to do them as seperate drafts.. 
and it
is a good thing too.. ADD-IP still needs some more work (in the security 
front) and
PR-SCTP is now 3 years later almost done...

Many implementations do have ASCONF working.. which can help in this
situation... but one would need (as Qiaobing pointed out) some sort of 
L2 trigger
to tell the peer "hey your nic card X failed" so that it could "know" to 
send the
"set-primary" to the alternate. And of course a "generic" trigger, as 
yet, does not
exist. Another area ripe for research...

R

---

From: Sourabh Ladha <[email-masked]>
Date: Wed, 6 Aug 2003 11:43:04 -0400 (EDT)

[snip]
> NO cwnd advance would happen.

Randall,

What you just described seems to me like a bug rather than a
"feature" (Leave alone the case of spurious timeout for now). In the
example you gave, the SCTP sender would probably need N roundtrips to recover
after a T3 and retransmission starts
off on the alternate (where N packets were in flight (consider all lost)
before you hit a T3 on the primary). This seems an over conservative
(read slow) behavior.

I must be missing something here :-O

-Sourabh

---

From: "Randall R. Stewart (home)" <[email-masked]>
Date: Wed, 06 Aug 2003 11:03:41 -0500

>I must be missing something here :-O
Consider for a moment a TCP sender....

At a T3-Timeout it sets cwnd = 1MTU
and then retransmits the lowest un-acked packet it can.

Then when the ack comes back.. it increases its cwnd and send
2 segments.. etc in slow start.

For SCTP the behavior is similar...

When a T3 is hit, you transmit only 1 MTU worth of data
to the alternate (if there is one) and sent the original destination
to 1MTU (marking all un-acked chunks for retransmission).

Now when the SACK comes back for the first retran-packet, you retransmit
up to 2 MTU's to the alternate... (lets assume for the moment you
had N in-flight chunks where  the size was N > 3 MTU)

Then you wait for a SACK... when it comes you increase the cwnd of
the alternate and send 3 MTUs of retranmssion...

In other words.. its NO DIFFERENT than TCP when a T3 hits.. even though
you are using an "alternate" path. This behavior was defined 
conservatively, yes, so
that we would in effect look the same as TCP... one really does not know
if the alternate path is diverse from the primary or not ....

Its no slower than TCP is today...

R

---

From: Sourabh Ladha <[email-masked]>
Date: Wed, 6 Aug 2003 12:41:53 -0400 (EDT)

[so now coming back to your example email]

> alternate above 1MTU in flight size .... so I doubt there is a problem..

This generalization was confusing but your latter mail eats this up.

> 1) I see no CC problem with the current scheme

So the congestion window overgrowth in your example can  occur (and
thats a term I am using because eventually an SCTP sender will credit the
alternate's cwnd for the SACK of the original).

Taking your example where the primary faced a delay spike. When the SCTP
sender hits a T3, it will retransmit 1MTU worth to alternate. Looking at
the GoBackN, as the SACKs for the orginal now start coming in, the SCTP
sender eventually starts crediting the cwnd of the alternate and pumping
more retransmissions to the alternate. This is what I had refereed to as
congestion window overgrowth due to retransmission ambuiguity in a
previous mail. The issue was really that what if the alternate path was
congested/broken (not yet detected by HB). The SCTP sender would not
realize this while doing GoBackNs and pump retransmissions to the
alternate.

As you said Eifel's purpose was to undo the primary's state after this
spurious t/o, but it may also be able to avoid this congestion window
overgrowth in some cases (a +ve side effect).

-Sourabh

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 06 Aug 2003 12:13:20 -0700

> perfecting its congestion control here?

I don't understand why we are giving up retransmission to an
alternate address.  The simple change I suggested was to do
the first timeout retransmission to the original address.  SCTP
will still send the second timeout retransmission to an alternate
address.  This is a default behavior I suggested implementors to
use.  Having said that, it does not mean that an SCTP implementation
should always follow the default behavior.  It can just be a
simple switch or an option to change the behavior to send the
first timeout retransmission to an alternate address.

> do it! (I know I sound a little like a promotional speaker, my apology

What I try to say is that if SCTP is a general purpose transport
protocol, the default behavior should not be designed to fit in
a particular kind of environment.  I think you agree with that,
right (-:

[From another email]

 > detector is doing the "guessing" :-)

I don't understand.  If the NIC you are referring to above
belongs to the endpoint, why is an external monitor/detector
needed to tell the SCTP stack that the NIC is bad?  Isn't this
the job of the device driver?

 > multi-homing advantage. I think this is too costly for whatever we are

Assuming we are referring to a NIC belonging to the endpoint where
the SCTP stack lives, I don't think using ASCONF will remove the
auto detection/recovery capabilities of SCTP.  The device driver
tells the IP stack that a NIC is bad, so the IP stack does something.
Isn't this kind of communication default in most OSes nowadays?  If
the stack also supports SCTP, then SCTP can use this info to do
something.  This is quite straightforward.

-- 

						K. Poon.
						[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Wed, 06 Aug 2003 16:51:31 -0500

Randy/Armando,

I don't know much detail and the scope of your ongoing
discussion/analysis on this issue. But one of my concerns is that if we
focus the analysis too closely to local behaviors of a single node or
association we may lose our sight to the bigger picture. 

Here is a scenario: a multi-homed SCTP association is sharing a big area
network (or just the Internet) with other TCP, UDP, SCTP, RTP traffic
and is actively passing data across the network. I understand there is
no mechanism in existence yet to guarantee path diversity for this
multi-homed SCTP association (maybe such a mechanism will be produced by
the multi-homing WG soon.. who knows). But there is nothing to prevent
path diversity from naturally occurring either. So, let's assume that
the association has path diversity across the network. 

Now at the SCTP sender suddenly gap SACKs or timeouts start to be
indicated on the primary dest address. This can mean either congestion
or path failure which we have no way to tell for sure. But it surely
indicates *some* problem with the part of the network that the primary
path crosses. Now the issue is how the sender should react. We seem to
be talking about the following options:

a) The "agile" approach - move retrans to the alternative at the first
sign of trouble; this is the current RFC2960 recommendation.

b) The "stubborn" approach - insisting on using the current addr for all
trans and retrans until nothing can get through; 

c) something in-between - e.g., if it is FR, use the same addr, if RTO,
use the alternative.

The main advantage of "stubborn" over "agile" is (to my understanding at
least) that the sender side CC algorithm is 1) less prone to generating
spurious retrans, which translates into less wasted network bandwidth,
and 2) no longer suffers from the ack ambiguity, which translates into
???... (I am still trying to understand the real benefit here).

The benefits of "agile" over "stubborn" are far clearer to me: 1) as I
mentioned in an earlier email, if the *some* problem is caused by an
actual fault in the primary path, the fault recovery speed of the
"agile" approach is much faster than the "stubborn". In fact, the
"stubborn" approach degrades SCTP to TCP equivalent in terms of path
failure detection latency. 2) if the *some* problem is congestion
induced (maybe some one started download huge movie files or a workhorse
router went down in that part of network), wouldn't it still be a very
good thing that the SCTP traffic is swiftly shifted away from the
congested area of the network (assuming path diversity exists here)?
This not only helps the congestion condition to improve quicker in the
affected area, but also lessens the impact on the data sender's
throughput. Imagine that if in the future a portion of the Internet
traffic can perform this type of "agile" multi-homing, how much it could
benefit the network congestion recovery.

IMHO, the *real* benefits of "agile" approach to both the application
and the network more than justify the occasional spurious retrans and
the whatever bad things caused by the ack ambiguity. I hope this wider
impact is factored into your simulation and analysis.

regards,
-Qiaobing

---

From: Qiaobing Xie <[email-masked]>
Date: Wed, 06 Aug 2003 17:28:48 -0500

Kacheong Poon wrote:
> address.  

This is less dramatic. Assuming the sender's RTO_min = 1 sec at the time
of the fault, this still adds 2 sec delay to the fault recovery which
could be significant in some cases. However, does this "just one more
retrans to the current addr" really help you? How?

> This is a default behavior I suggested implementors to

But how are we going to tell people when to *not* use that optional
behavior? I'd guess that having faster fault recovery is welcome by
everyone. I used telecomm as an example simply because for them recovery
speed becomes a vital requirement. But in reality, it will be a good
thing for everyone especially for those who have already gone through
the trouble of setting up multi-homing, right?

> a particular kind of environment.  I think you agree with that,

I guess I would agree if I can understand why the current design does
NOT fit general purpose (also see my other email on this topic).

> the job of the device driver?

By "external" I referred to anything outside the SCTP stack itself.
Device driver is external and we have know knowledge as to how it really
works to determine a NIC failure, for example.

regards,
-Qiaobing

---

From: "Randall R. Stewart (home)" <[email-masked]>
Date: Thu, 07 Aug 2003 06:39:11 -0500

Qiaobing:

Comments below..

>association we may lose our sight to the bigger picture. 
There is yet a LOT of analysis to be done I think :>

>c) something in-between - e.g., if it is FR, use the same addr, if RTO,
I originally was leaning toward <a> as Armando can attest too :>

But some of his work has convinced me that <c> may be a better
choice... the reason being:

1) If you are still getting Sack's that cause you to FR then there
    is no chance that the far end has failed.

2) Armando's work as shown that with even a mild loss rate on the
    alternate (due to congestion) the switched transmission is less 
beneficial
    due to I think mainly the fact that the RTO on the alternate tends to
   inflate over time... since you have very little traffic going there. My
   solution to this was to schedule a HB right away after retransmitting to
   an alternate to help bring down the RTO... this helped some (as Armandos
    simulations show) but not as much as sticking with the original 
destination. However
   Armando never included the concept of failed destination either.. 
thus some more
   work is needed...

3) After looking at all the data, I think because <1> is true in FR case 
then sticking
    with the primary will give the app the best throughput. I do believe 
that retrans
    need to go to the alternate though...

>???... (I am still trying to understand the real benefit here).
I don't see how the stubborn approach gets rid of any ack ambiguity.. you
still do not know if the SACK is from the retransmitted data or the 
original.
I think the only ambiguity you may fix is for a cwnd update.. but even that
I could imagine a really really long delay spike that finally gets you
to move to the alternate.. and presto the ambiguity is there... I just
don't see that the stubborn approach helps for the ambiguity issue :>

>impact is factored into your simulation and analysis.
We need a lot more work.. and I know Armando will get this out over
the next year or so... I really think that <c> will provide the best
throughput and at the same time the best fault recovery..

R

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 7 Aug 2003 08:53:31 -0500

> a) The "agile" approach - move retrans to the
> alternative at the first sign of trouble; this is the
> current RFC2960 recommendation.
> 
> b) The "stubborn" approach - insisting on using the
> current addr for all trans and retrans until nothing
> can get through;
> 
> c) something in-between - e.g., if it is FR, use the
> same addr, if RTO, use the alternative.

Actually I think the "stubborn" proposals are more
along the lines of "insisting on using the current addr
until it is obvious that there is really a problem
there". That is, multiple drops would indicate there
really is congestion and the path should be abandoned.
The question is whether presuming that a *single* drop
indicates a problem is appropriate.

But the "agile" versus "stubborn" distinction may be
very close to the real distinctin: "low latency" vs.
"throughput".

-   If your primary concern is low latency, rather
    than throughput, then you want to switch paths on
    the first problem. Not doing so will delay being
    able to communicate over the remaining viable
    path. That delay represents a period with loss of
    contact, which is very unacceptable for a control
    application.

-   If your primary concern is throughput then
    switching paths because of a single transient drop
    might be an overreaction. The alternate path has
    not been "exercised". Shifting to it will result
    in at least a temporary drop in throughput.
    
    On the other hand, delaying switching might be
    delaying a corrective response to a congestion
    problem. 

That's a distinction based upon usage, not on the
target network. Which is certainly more acceptable.

Just thinking out loud here, but would it be
appropriate to leave this choice to the sender? The
sender could easily know which mode it was in either by
explicit user configuration and/or just observing. (If
the pending Data Chunk queue hasn't hit empty for
several seconds you are probably not being used by a
"low latency" application).

Note: I still believe that such an algorithm has to
make sense based upon "typical" networks and should not
be used to solve special L2 problems. Is there any
hard research as to how often a packet is dropped
for reasons that are neither equipment failure or 
congestion? How high of a "false positive" rate
are we talking about here?

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 07 Aug 2003 12:51:44 -0500

> >...2) if the *some* problem is congestion
....
> throughput and at the same time the best fault recovery..

One more observation along this line about CC research - The age-old TCP
CC algorithm (as well as that of SCTP and the newer TFRC) is based on a
simple "common sense" strategy: "hit the brake when you see the road
ahead is jammed". Over the years, countless research effort has gone
into figuring out when to hit the brake and how hard to hit the brake
:-) But why sticking to this strategy? It is simply because that is the
ONLY road known to the car driver (i.e., the TCP sender) to reach the
destination! 

Now, SCTP's multi-homing support opens up a new dimension to the CC
world - now the car driver has the knowledge of (possible) alternate
routes to the same destination. Shouldn't this new dimension prompt our
driver to rethink about its strategy and take full advantage of it? Or
should we say "this new dimension does not fit into the age-old strategy
so let's take it out"?

IMHO, CC research community should start thinking about how to best
benefit from multi-homing now, instead of the other way around.

regards,
-Qiaobing

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 07 Aug 2003 13:07:18 -0700

> retrans to the current addr" really help you? How?

I believe the second timeout provides a very good indication
that the path to the original address has some issues.  The
primary concerns we have are spurious timeout, congestion,
NIC failure.  I believe the spurious timeout issue is mostly
ruled out if the the timer fires the second time.  If the
first retransmission is also dropped because of congestion,
it will be the same to use the primary and alternate path
as far as congestion control is concerned as ssthresh will
not be a factor anymore.  And the path must be very congested.
If it is a NIC failure, I guess two tries are good enough.

If SCTP retransmits to the original address in the first
timeout, I believe F-RTO will work pretty well in detecting
spurious timeouts without introducing new chunks or new
bundling rules.  To me, this is a big win for F-RTO.  We
don't want to keep changing rules, especially if it requires
both sides to agree.  F-RTO can be done on the sender side
only without changing anything in RFC 2960.  If we really
want to speed detecting NIC failure up, the already defined
ASCONF should work pretty well.  As a matter of fact, it
should work faster than the timeout retransmission.

> the trouble of setting up multi-homing, right?

You are making the assumption that the alternate address is
always better here.  I don't believe SCTP has this knowledge.
It is pretty much a guess.  So I suggest people not to use
this optional behavior unless they are pretty sure that their
network setup supports it.  For example, it may work in your
telecomm network.  And if I understand how you put it, the
telecomm network is pretty much separated from the real
Internet such that it can provide that kind of guarantees you
mentioned, no congestion.  So I don't see a problem for all
the SCTP stacks used in that environment to change the behavior.

I guess we should not provide false information to people that
the alternate address is always better than the original
address in the event of retransmission.  Or am I missing something
that you can provide such a proof that the alternate address is
always better in the case of the first timeout?

> NOT fit general purpose (also see my other email on this topic).

I believe the confusion caused by retransmission to an alternate
address is well discussed.  The current default behavior is only
guaranteed to work well in a specially designed network.  In
general, SCTP does not have this kind of knowledge about the network.
And I believe the simple change not to use alternate address in the
first timeout helps solve the majority of the confusion described with
the help of F-RTO, a mechanism which does not require new chunks or
bundling rules.  To me, this is a very strong reason not to follow
the default behavior described in RFC 2960.  Well, just my $0.02
anyway.  But I hope people will not spread the idea that using
an alternate address is always better, unless the network is designed
that way.  As I mentioned earlier, there may be a proof contrary
to this which I haven't seen.  If it can be proved, I have no issue
using the alternate address in the first timeout.

> works to determine a NIC failure, for example.

Well, I guess you think all those products, commercial and free,
which provide such kind of NIC failure detection are wrong
architecturally and do not work well...

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 07 Aug 2003 13:31:00 -0700

>     contact, which is very unacceptable for a control

I guess the issue here is whether using the alternate
address will be better.  Firstly, I believe SCTP cannot
know for sure that the paths to the two different addresses
do not cross.  If they in fact are crossed, sending to
the alternate address may have the same problem as well.
I believe all SCTP knows about the alternate address is
that it is reachable at the time SCTP receives the last
HB-ACK.  That is all.  I don't know if SCTP can infer
more info from that.  So blindly retransmitting to an
alternate address may not help in the "low latency"
goal at all.

> target network. Which is certainly more acceptable.

But the issue is if the network actually supports the
claim that the above two mechanisms work as described.

> "low latency" application).

Do you mean by the sender's SCTP code or the app code?
I don't know if it works well for SCTP to determine
the network usage of the app.

> are we talking about here?

It seems that "typical" means different things to
different people.  So such an algorithm may actually
need to work well in "all kinds of" networks...

-- 

						K. Poon.
						[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 07 Aug 2003 18:34:09 -0500

Kacheong Poon wrote:
....
> It is pretty much a guess.  

You emphasized this point several more times below (and did so in your
another email). My comment is that the enter TCP/SCTP CC algorithm is
designed around "making guesses" - or call it the "learn by experience"
principle. If one rejects this principle, nothing in the CC design will
hold. 

In our particular SCTP case here, the sender just had a "bad" experience
with the primary addr and has "no or little experience" with the
alternate addr. No one in his right mind will ever say he's sure that
the alternate address is always better. But it is simply due to this
fundamental "learn by experience" principle that the sender has a reason
to favor the alternate addr ***at that particular moment***. The
reasoning is not based on "always better", rather it is on the "no
experience + optimistism" vs. "fresh bad experience".

> So I suggest people not to use

As I said before, it's not just about telecomm. Rather, any association
that has path diversity will potentially benefit. This includes
associations across the Internet since there is nothing to prevent path
diversity naturally occurring in the Internet.

> address in the event of retransmission.  

> ... Or am I missing something

ditto.

> general, SCTP does not have this kind of knowledge about the network.

It is quite clear that if you have path diversity you will benefit from
it; But if you don't, will it hurt? This is what I wanted to understand
in my question above.

> using the alternate address in the first timeout.

ditto.

> architecturally and do not work well...

Nothing is wrong about using external detection. I think it's a matter
of design choice. In SCTP, the design decision was made very early to
make it more robust by including built-in failure detection and recovery
mechanism. Now you seem to suggest remove or disallow those built-in
mechanism and instead, to rely on external mechanism for failure
detection. Doing so, aren't you going to make SCTP users' life harder?

regards,
-Qiaobing

---

From: Pasi Sarolahti <[email-masked]>
Date: 08 Aug 2003 11:40:04 +0300

> using the alternate address in the first timeout.

If I understood it correctly, the first RTO (or T3) retransmission to
alternate address may cause problem for F-RTO and Eifel only if the RTT
of the alternate path is smaller than the RTT of the original path plus
the delay spike, so that the acknowledgement of the retransmission
arrives before the acknowledgements of the delayed original segments.

The problem with F-RTO in setup above was that a possible loss of the
originally transmitted data chunk that was later retransmitted with RTO
may not be detected, if the RTO was caused by a delay spike. So, as far
as I see it, in this case the unnecessary retransmissions can be avoided
with F-RTO, but the congestion control decision may be incorrect
(depending on what kind of congestion control is applied after detecting
a spurious timeout). Hence, with 'SCTP-retransmitting-to-alternate-addr'
the congestion control undoing decision should be made only after the
unnecessary RTO retransmission can be confirmed with a duplicate-TSN.
The benefit of F-RTO would still be that the other unnecessary
retransmissions after RTO can be avoided, regardless of the
characteristics of different paths.

This being said, I believe the issue of whether to do the first
retransmission to original or alternate address is not that important
from the F-RTO standpoint, although making first retransmission to the
original would make things a bit easier. Of course, the other aspects
discussed here are another thing.

- Pasi

---

From: "Randall R. Stewart (home)" <[email-masked]>
Date: Fri, 08 Aug 2003 07:30:21 -0500

Kacheong:

I don't think this is what Qiaobing meant... I too have
had issues with failure detection in NIC's. The problem is:

1) Some NIC support it, some don't
2) The driver generally has to do some query to see
    that status (active/failed etc)..
3) The driver generally does NOT send this information on
    outside of the driver because there is no general purpose
    way for it to do so.

I have always thought there needed to be some "standard" ethernet or IP
frame the driver could generate to report local changes to the
NIC card... this would give kernel and driver developers a standard
mechanism to pass on to transport protocols etc. And security on
this would not be an issue.. if one comes in from the outside the
driver could silently discard it...

I had a proposal once and even made a driver generate such.. but I never
did anything with it .. not enough cycles  :-0

Oh, well.. maybe someday

R

---

Randall R. Stewart