Subject: Re: quick failover in SCTP

From: Michael Tüxen <[email-masked]>
Date: Fri, 8 Oct 2010 12:16:12 +0200

Hello Nishida-san,

OK, I thought about this some time and think it would be good
to specify a way for a quick failover which can be implemented
at the sender only.

I would like to see some extensions to you suggestion:
* Introduce a threshold (call it PFMR for now).
  Then use
  if (error counter > PMR)
     the path is inactive.
  if (error counter <= PMR) && (error counter > PFMR)
     the path is potentially failed.
  Using PFMR = 0 is what you suggest, PFMR=PMR gives
  the old behavior.

* Specify that you start sending HBs when the path is
  PF. Explicitly allow PFHB.interval=0, which
  I think is a good choice. Maybe we can just remove PFHB.interval.

* Make sure that the following works: The application disabled HBs.
  When a path enters PF (or failed) HB are sent to get the path
  active again. If it is active, no HB should be send (since
  the application disables them).

* Provide a way that applications are not bothered with
  state change notification related to PF when not explicitly
  subscribed.

* Make clear what to do when all paths are PF.

* Make clear what to do when all paths are failed.

What do you think?

Best regards
Michael

On Aug 24, 2010, at 12:04 PM, Yoshifumi Nishida wrote:

> Hello Michael,
> 
> Thanks for your reply. In a nutshell, the difference between PF and PMR=0 are:
>   PF allows SCTP to use another path while SCTP is checking the
> primary path is active or inactive, but don't change the behavior of
> marking the path as inactive.
>   PMR=0 allows SCTP to mark the primary as inactive quickly.
> 
> In my feeling, this will create several differences although it looks similar
> 
> For example, if we use PMR=0, we will need to modify at least the
> following points in the RFC4960
>   1) recommended value for PMR
>   2) behavior in dormant state
>   3) relationship between PMR and AMR
>        RFC4960 states users should avoid having the value of
> 'Association.Max.Retrans' larger than the
>        summation of the 'Path.Max.Retrans'  we'll need to change this part.
> 
> Also, I think we'll need to think about the following points
>   a) Some of current applications or OS local configurations might
> already have specified PMR on their own. If they're not using PMR=0,
> their benefit might be reduced.
>   b) When you have 100Mbps and 1Mbps links and you set 100Mbps as
> primary, everytime packet loss happens on the 100Mbps link, it will
> switch over to the 1Mbps link and have to wait for HeartBeat which is
> likely less frequent (30secs or so). Or, you'll need to add special
> logic here in the spec.
> 
> If we use PF,
>   PF allows SCTP to keep PMR and AMR unchange. Hence, we don't have
> to modify 1) and 3).
>       and issue 2) will be a minor point since we don't change PMR and AMR.
>   Also, PF already has a solution for b) as described in the draft.
> 
> In my view, PMR=0 will requires several "modifications" to the spec
> which might be a bit tricky to understand for implementers while PF
> will requires explicit "addition".
> 
> Thanks,
> --
> Yoshifumi Nishida
> [email-masked]
> 
> 
> 2010/8/23 Michael Tüxen <[email-masked]>
>> 
>> On Aug 17, 2010, at 12:16 PM, Yoshifumi Nishida wrote:
>> 
>>> Hi folks,
>>> This is a follow-up from the Maastricht meeting.
>>> Preethi and I are proposing quick failover algorithm in SCTP and gave a presentation about this one.
>>> 
>>> In my feeling, the community seems to be positive in enhancing the SCTP standard to some extent to address this issue.
>>> Also, in my understanding, Michael and Randy suggested that minor updates in the current spec can have the similar effects as the PF approach can do.
>>> So,  we're going to start with investigating the alternatives for PF approach and would like to know about the detail of the suggestion.
>>> If Michael and Randy could give us some info about this, we would be grateful very much.
>> Sure.
>> One point is that RFC 4960 does not specify what to do when all paths
>> are INACTIVE. If that happens, the association is called dormant.
>> I think it should be clarified that you still send HB to get a path
>> to ACTIVE again until the association finally fails. This could be
>> handled as an errata, I think.
>> 
>> Now assume that handling of the dormant state.  If I understand PF correctly,
>> you could simply set Path.Max.Retrans to 0 to get the same behavior on the
>> wire as when using PF, or am I missing something? The only difference I see,
>> are the path state change notification sent locally to the user.
>> 
>> But maybe I have overlooked something...
>> 
>> Best regards
>> Michael
>>> Also, if someone who has comments or feedbacks for this, please let us know.
>>> 
>>> Thank you so much.
>>> --
>>> Yoshifumi Nishida
>>> [email-masked]
>>> 
>>> 
>>> 
>> 
>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Sat, 9 Oct 2010 03:04:32 -0700

Hello Michael,

Thanks for your response.

> to specify a way for a quick failover which can be implemented

Great!
Please check my comments bellow.

>  Using PFMR = 0 is what you suggest, PFMR=PMR gives

I thought similar thing and I agree with providing a way to disable PF.
I tend to agree with this idea, but one thing I'm not very sure is how
PFMR != 0 && PFMR < PMR can be useful.
If we just want a switch to disable PF, having USE_PF parameter might be enough?
May be Preethi has an opinion on this?

>  I think is a good choice. Maybe we can just remove PFHB.interval.

Hmm. Sorry. I might not quite follow this point.
Does PFHB.interval=0 mean suppress sending HB during PF state?

I agree with all of the following points. I think these are very good points.

---

From: Michael Tüxen <[email-masked]>
Date: Sat, 9 Oct 2010 14:34:31 +0200

> PFMR != 0 && PFMR < PMR can be useful.
I could image someone wanting to call a path potentially failed
after 2 consecutive timer based retransmission instead of 1.
Just being a bit more conservative. This might help deploying
such a feature in SIGTRAN networks where CMT is not deployed.
For CMT traffic, PFMR==0 is the right choice, I guess.
But I think PF is also very helpful in non-CMT scenarios.
> Does PFHB.interval=0 mean suppress sending HB during PF state?
No. I mean just to send them every RTO.
Having a specific interval allows this by setting PFHB.interval=0.
I was just thinking whether one can remove the parameter and send
the HB every RTO (and doubling it)... The same as using PFHB.interval=0.

Does this makes things clearer?

Best regards
Michael

---

From: Michael Tüxen <[email-masked]>
Date: Mon, 11 Oct 2010 19:01:30 +0200

Hi Preethi,

comments in-lne.

Best regards
Michael

On Oct 11, 2010, at 6:02 PM, Preethi Natarajan wrote:

> Michael,
> 
> Thanks for the insightful comments/suggestions. Please see my responses
> below.
> 
> 
> On 10/9/10 5:34 AM, "Michael Tüxen" <[email-masked]>
> wrote:
> 
> Preethi: Good point. What you are suggesting is a tunable PFMR threshold
> while the current draft always assumes PFMR=0, the most aggressive behavior.
> I don't see how conservative PFMR values can be helpful but it may be a good
> idea to have it just in case.
> 
> 
> Preethi: I agree. I don't think we need a PFHB.interval. In PF state, HBs
> are sent once every RTO.
> 
> 
> Preethi: Michael can you clarify this a bit? Looks like we want to confirm
> the following -- apps can control only those HBs that get sent (or not sent)
> during path failure. Apps do not have control over PF HBs; i.e., apps cannot
> enable/disable PF HBs. Is that right?
The socket API allows to configure the HB.interval and to enable/disable
HBs at all. What I suggest is the no matter if the HB are disabled, they
are sent for
* initial path verification (not within the scope of your ID, but noted
  to make things clear)
* getting a path out of potentially failed or failed.
In other words: If the application disables HBs, it only enables the
ones which do path supervision when the path is active and not potentially
failed.

Does this clarify things?
> Preethi: Good point. To clarify: when PF is on, apps will be notified only
> about PF-to-failure and failure-to-active state changes. Apps will not be
> notified about active-to-PF state change.
Yes. This means that we can enable PF system wide without apps seeing
any difference.
> Preethi: I agree with both points. We can clarify these in the draft.
> 
> Thanks,
> Preethi
> 
>

---

From: Preethi Natarajan <[email-masked]>
Date: Mon, 11 Oct 2010 09:02:44 -0700

Michael,

Thanks,
Preethi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 12 Oct 2010 03:42:53 -0700

Hello Michael,

Thanks for the clarification.

> But I think PF is also very helpful in non-CMT scenarios.

OK. I think it would be good to have tunable parameter for users. I
agree with this.

> the HB every RTO (and doubling it)... The same as using PFHB.interval=0.

I see. My main motivation here is not to use HB.interval since it's
usually rather big value.
So, using RTO is fine for me.
However, I think there might be useful case for PFHB.interval. (e.g.
all paths have mostly the same bandwidth and priority)
I prefer to keep tunable parameter here while recommended behavior is using RTO.

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: "David Lehmann" <[email-masked]>
Date: Thu, 14 Oct 2010 10:14:39 -0400

> OK. I think it would be good to have tunable parameter for users. I

Just to be clear the default would be PFMR == 0, which means use another path if the error count is non-zero.  Correct?

-David

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 14 Oct 2010 17:00:27 +0200

> Just to be clear the default would be PFMR == 0, which means use another path if the error count is non-zero.  Correct?
Correct.

Best regards
Michael

---

From: "David Lehmann" <[email-masked]>
Date: Thu, 14 Oct 2010 15:39:21 -0400

> > Just to be clear the default would be PFMR == 0, which means use another
> path if the error count is non-zero.  Correct?
> Correct.

Nice!

-David

---

From: Preethi Natarajan <[email-masked]>
Date: Thu, 14 Oct 2010 13:36:21 -0700

> Does this clarify things?

Yes, thanks.

Preethi