Subject: [Tsvwg] SCTP multi-homing association

From: devayya <[email-masked]>
Date: Fri, 07 Apr 2006 14:43:57 +0530

Hi,

I have a question on SCTP multi-homing,

My SCTP multi-homing test set up is as follows,
There are two etherenet ports with different ip address are configured 
for one endpoint(say endpont A). and similarly I have another endpoint 
(say endpoint B) with two ethernet ports with different ip addresses. 
The association between two endpoints is ESTABLISHED.

Using a test tool I transmit traffic between these two endpoints.
With less amount of traffic flowing, if one of the ethernet port IP 
address is lost, then the association between these two endpoints remain 
ACTIVE and the traffic flows on the second ethernet port.(As per 
multi-homing  requirement)
With increase in traffic flowing, if one of the ethernet port IP address 
is lost, then the association between these two endpoints goes to 
INACTIVE. and there is traffic loss. Although the SCTP association is 
still present, the ULP association goes down, with the reason "delay ACK 
timer expiry". Can anybody help find out why?

As per SCTP implemtors guide, there is mention of  updation of cwnd when 
user tries to transmit DATA, cwnd should be modified depending on the 
flightsize and a newly introduced MAX.BURST.(not present in RFC)
               if((flightsize + Max.Burst*MTU) < cwnd)
                    cwnd = flightsize +  Max.Burst*MTU

Is it really needed to implement this MAX.BURST?
Will addition of this MAX.BURST solve the earlier problem mentioned by me?

Thanks and regards,
devayya

P.s: For your reference I have attached a text file of the set up.

    ----------------                           -----------------
    |               |                          |                | 
    |               |P| <==================> |R|                | 
    |Endpoint A     |                          |  Endpoint  B   |
    |               |Q|                      |S|                |
    |               |                          |                |
    ----------------                           ------------------
Traffic is flowing between Endpoint A and endpoint B, using enthernet ports P and R

    ----------------                           -----------------
    |               |                          |                | 
    |               |P|                      |R|                | 
    |Endpoint A     |                          |  Endpoint  B   |
    |               |Q| <==================> |S|                |
    |               |                          |                |
    ----------------                           ------------------
The enternet port P is lost and now the traffic flows using ethernet port Q and S.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 7 Apr 2006 05:28:36 -0600

devayya,

Does R have a route to Q?

--brian

devayya wrote:                                                     (Fri, 07 Apr 2006 14:43:57)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: devayya <[email-masked]>
Date: Fri, 07 Apr 2006 16:59:15 +0530

There is route from P to R and Q to S. I have configured this using the 
system command
"route add -net x.x.x -iface y.y.y.y"
However I could not configure route from R to Q or S to P. Can you guide me?

thanks,
devayya

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 7 Apr 2006 05:46:15 -0600

devayya,

Yes, your problem is with your routing tables.  You have two disjoint
subnetworks.  SCTP is notorious for failing to handle this situation.
When P fails, Endpoint A loses its route to R, but some SCTP
implementations cannot detect that loss of a route to R and will attempt
to retransmit packets that time out to S to R, resulting in more
timeouts and rather quick association failure.

I cannot help you configure your routing tables on an unknown OS.  Read
the manual pages for the OS.  It might be that you have no hub/switch/
router between the subnets (e.g. direct turnover ethernet cable
connection).  In that case, install a router/switch/hub.

--brian

devayya wrote:                                                     (Fri, 07 Apr 2006 16:59:15)
> 
>    There  is  route  from P to R and Q to S. I have configured this using
>    the system command

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 7 Apr 2006 13:54:56 +0200

Hi Devayya,

'delay ACK timer expiry' is something which should result in the  
sending of a SACK, not
in the teardown of an association. This looks like a bug to me...

Best regards
Michael

On Apr 7, 2006, at 11:13 AM, devayya wrote:

> There are two etherenet ports with different ip address are  
> configured for one endpoint(say endpont A). and similarly I have  
> another endpoint (say endpoint B) with two ethernet ports with  
> different ip addresses. The association between two endpoints is  
> ESTABLISHED.
>
> address is lost, then the association between these two endpoints  
> remain ACTIVE and the traffic flows on the second ethernet port.(As  
> per multi-homing  requirement)
> With increase in traffic flowing, if one of the ethernet port IP  
> address is lost, then the association between these two endpoints  
> goes to INACTIVE. and there is traffic loss. Although the SCTP  
> association is still present, the ULP association goes down, with  
> the reason "delay ACK timer expiry". Can anybody help find out why?
>
> As per SCTP implemtors guide, there is mention of  updation of cwnd  
> when user tries to transmit DATA, cwnd should be modified depending  
> on the flightsize and a newly introduced MAX.BURST.(not present in  
> RFC)

---

From: devayya <[email-masked]>
Date: Fri, 07 Apr 2006 17:34:56 +0530

Michael,

Consider a case where Endpoint A has sent a data to endpoint B. and 
endpoint B has sent SACK for that received data. But before that SACK 
could reach endpoint A the ehernet port P of endpoint A has gone down. 
Now the traffic switches over to ethernet port Q of endpoint A. Within 
that fraction of second wher the switch over fro port P to port Q 
happens there is bound to be some message loss. There is no way that 
endpoint B will know that the SACK sent by it is not received by 
endpoint A. Or is it there?  and so the delay ack timer expiry happens, 
Correct me if I am wrong.

thanks,
devayya

Michael Tuexen wrote:

>> on the flightsize and a newly introduced MAX.BURST.(not present in  RFC)

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 7 Apr 2006 14:13:47 +0200

Devayya,

see my comments in-line.

Best regards
Michael

On Apr 7, 2006, at 2:04 PM, devayya wrote:

> endpoint B has sent SACK for that received data. But before that  
> SACK could reach endpoint A the ehernet port P of endpoint A has  
> gone down. Now the traffic switches over to
If the SACK does not arrive a Endpoint A, there will be a T3 timer  
running at Endpoint A for that DATA chunk and
it will be retransmitted when T3 expires.
> ethernet port Q of endpoint A. Within that fraction of second wher  
> the switch over fro port P to port Q happens there is bound to be  
> some message loss. There is no way that endpoint B will know that  
> the SACK sent by it is not received by endpoint A. Or is it there?   
> and so the delay ack timer expiry happens, Correct me if I am wrong.
The delay ack timer will only delay the sending of SACKs. That is  
all. When it expires, a SACK will be sent. It does
NOT supervise the successful transmission of SACKs.
>>> With less amount of traffic flowing, if one of the ethernet port  
>>> IP  address is lost, then the association between these two  
>>> endpoints  remain ACTIVE and the traffic flows on the second  
>>> ethernet port.(As  per multi-homing  requirement)
>>> address is lost, then the association between these two  
>>> endpoints  goes to INACTIVE. and there is traffic loss. Although  
>>> the SCTP  association is still present, the ULP association goes  
>>> down, with  the reason "delay ACK timer expiry". Can anybody help  
>>> find out why?
>>>
>>> As per SCTP implemtors guide, there is mention of  updation of  
>>> cwnd  when user tries to transmit DATA, cwnd should be modified  
>>> depending  on the flightsize and a newly introduced MAX.BURST. 
>>> (not present in  RFC)

---

From: <[email-masked]>
Date: Fri, 7 Apr 2006 15:15:21 +0300

Hi!

	I think you are mixing two things here: SACK delay timer and retransmission timer. When you get an SCTP datagram containing DATA chunks, you don't send the SACK chunk immediately. You either wait till you get a second datagram containing more DATA chunks or the expiration of the SACK delay timer, whatever happens earlier. And then you send back the SACK chunk. 

	That timer should be running at the receiver of the DATA chunks, and by no means should cause any problem to the association. Simply, when the SACK delay timer expires you send the SACK chunk, and that's all.

	BR Iván Arias Rodríguez

>-----Original Message-----
>From: ext devayya [mailto:[email-masked]] 
>Sent: 07 April, 2006 15:05
>To: Michael Tuexen
>Cc: tsvwg
>Subject: Re: [Tsvwg] SCTP multi-homing association
>
>Michael,
>
>Consider a case where Endpoint A has sent a data to endpoint 
>B. and endpoint B has sent SACK for that received data. But 
>before that SACK could reach endpoint A the ehernet port P of 
>endpoint A has gone down. 
>Now the traffic switches over to ethernet port Q of endpoint 
>A. Within that fraction of second wher the switch over fro 
>port P to port Q happens there is bound to be some message 
>loss. There is no way that endpoint B will know that the SACK 
>sent by it is not received by endpoint A. Or is it there?  and 
>so the delay ack timer expiry happens, Correct me if I am wrong.
>
>thanks,
>devayya
>
>Michael Tuexen wrote:
>
>> Hi Devayya,
>>
>> sending of a SACK, not in the teardown of an association. This looks 
>> like a bug to me...
>>
>> Best regards
>> Michael
>>
>>> My SCTP multi-homing test set up is as follows, There are two 
>>> etherenet ports with different ip address are configured for one 
>>> endpoint(say endpont A). and similarly I have another endpoint (say 
>>> endpoint B) with two ethernet ports with different ip 
>addresses. The 
>>> association between two endpoints is ESTABLISHED.
>>>
>>> per multi-homing  requirement) With increase in traffic flowing, if 
>>> one of the ethernet port IP address is lost, then the association 
>>> between these two endpoints goes to INACTIVE. and there is traffic 
>>> loss. Although the SCTP association is still present, the ULP 
>>> association goes down, with the reason "delay ACK timer 
>expiry". Can 
>>> anybody help find out why?
>>>
>>> Traffic is flowing between Endpoint A and endpoint B, using 
>enthernet 
>>> ports P and R
>>>
>>>
>>>
>>> The enternet port P is lost and now the traffic flows using 
> ethernet 
>>> port Q and S.
>>
>>
>>
>
>

---

From: Mark Butler <[email-masked]>
Date: Fri, 07 Apr 2006 07:58:17 -0600

devayya wrote:

> endpoint A. Or is it there?  and so the delay ack timer expiry 
> happens, Correct me if I am wrong.
>
> thanks,
> devayya

First of all, can you tell us which OS and SCTP implementation you are 
using?  If not a common one, 'internal/proprietary' is fine.   A few 
comments:

1. Normally in SCTP, associations do not go active / inactive, paths do. 
2. Delayed ACK timer expiry is a normal event that is not an indication 
of a failure,
3. If a DATA chunk is lost, either a later SACK or the T3 timer will 
eventually trigger a retransmission, which is supposed to be on an 
alternate path if available.
4.  If a DATA chunk is received but a SACK is lost, and there are no 
follow on SACKs, the DATA chunk will also be retransmitted in the same 
manner, but ignored since it was already received.
5. Max.Burst probably doesn't have anything to do with your problem.

- Mark B.

>>> enthernet ports P and R
>>> ethernet port Q and S.

---

From: devayya <[email-masked]>
Date: Sat, 08 Apr 2006 09:37:02 +0530

Hi Mark,

Please see inline comments.

Mark Butler wrote:

> using?  If not a common one, 'internal/proprietary' is fine.

I am using Lynx RTOS(version 3.0.0) and the SCTP implementation is done 
inhouse according to the  rfc2960, with some changes from SCTP 
implementors guide version 14.

>    A few comments:
>
> 1. Normally in SCTP, associations do not go active / inactive, paths 
> do. 2. Delayed ACK timer expiry is a normal event that is not an 
> indication of a failure,

Here I was talking about the Upper Layer links(M2PA) going ACTIVE and 
INACTIVE. and also the delay ACK timer is the Upper Layer delay ack 
timer(T7 timer). I had metioned in my first mail that the SCTP 
association remains in ESTABLISHED state, but the Upper Layer link(M2PA) 
goes INACTIVE, probably due to no-delivery of the message from SCTP.
I am sorry, that everybody misunderstood this as SCTP delay ack timer 
expiry. My mistake, I should have been more clear.

> alternate path if available.

I agree.

> manner, but ignored since it was already received.

This could be one of the reason.

> 5. Max.Burst probably doesn't have anything to do with your problem.

I agree.

Thanks and regards,
devayya

>>>> depending  on the flightsize and a newly introduced MAX.BURST.(not 
>>>> present in  RFC)