Subject: Watchdog logic

From: Bernard Aboba <[email-masked]>
Date: Wed, 17 Dec 2008 09:39:38 -0800

In a previous message (http://ops.ietf.org/lists/radiusext/2008/msg00857.html) Alan DeKok
pointed out:

"With TCP, there is a connection between client and server.  The
watchdog timer algorithm in RFC 3539 is defined per *connection*.  So an
ID has to be reserved, because the client can't open a new connection to
test if an existing connection is still alive."

While it's true that the watchdog timer algorithm is defined per connection,
it seems like Status-Server is about determining whether a server is up
or not.  The only way to determine whether a connection is down
is to wait for it to close (either via a RESET or timeout).  RFC 3539
attempts to detect a failure at the application layer prior to connection failure. 

I'm wondering what the implications would be of using two separate connections, 
one for Status-Server/Access-Accept and the other 
one for Access-Request/Access-Accept transactions. 

The failover logic would change, to be sure.  For example, 
a connection failure on the Request/Accept connection would probably trigger
failover, regardless of the state of the Status-Server/Accept connection. 
Also, the state of the two connections could get out of 
sync; for example, if the Request/Accept connection was quite busy,
then the Status/Accept connection might send little or no traffic,
which might cause middleboxes (e.g. a NAT) to lose connection state
on the Status/Accept connection.  In such a situation, you might just
be able to bring up another Status/Accept connection rather than 
triggering failover.

---

From: Alan DeKok <[email-masked]>
Date: Thu, 18 Dec 2008 11:38:21 +0100

> attempts to detect a failure at the application layer prior to connection failure. 

  Ok.  There are situations where a connection may be up, but the
application is unresponsive.  It would be good to use the RFC 3539
method to validate the connection.

> be able to bring up another Status/Accept connection rather than 

  I'm not sure having a separate connection for Status-Server is a good
idea.

  In addition, the algorithm in 3539 appears to be focussed on keeping
the connections up... even if that means re-opening them.  I'm not sure
this is a good idea.  It means that spikes in traffic cause a large
number of connections to be opened... which then never close, or are
continuously re-opened.  Even if there's no traffic on them.

  It may be worth adding suggestions:

- TCP connections SHOULD be kept "full".  i.e. used in a "most recently
used" fashion for normal RADIUS traffic.

- The RFC 3539 watchdog algorithm should be used to determine the status
of a *connection*.

- so long as one connection is alive, the server should be marked "alive".

- connections that haven't been used for T seconds (4 * RTT?) may be
pro-actively closed.

- at least one connection should remain open to determine application
responsiveness.

  Alan DeKok.

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/radiusext/>

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 18 Dec 2008 07:33:25 -0800

> method to validate the connection.

The watchdog is based on application-layer timescales (e.g. 6 seconds), rather than connection timescales.  So it's possible for the watchdog to cause application layer failover prior to connection failure/reset. 

>   I'm not sure having a separate connection for Status-Server is a good

I'm not sure either.   Separating the Status-Server traffic from the RADSEC
traffic breaks "fate sharing" which could introduce a number of bugs.

> continuously re-opened.  Even if there's no traffic on them.

The idea is to always have a connection "ready" for traffic, so yes, the
algorithm does keep connections up even if there is no regular traffic
(e.g. the algorithm generates watchdog traffic).  

> - The RFC 3539 watchdog algorithm should be used to determine the status of a *connection*.

Not sure that the watchdog really determines connection status so much as status at the application layer.  

> - so long as one connection is alive, the server should be marked "alive".

Agreed.  But doesn't this somewhat conflict with the previous goal?

> - connections that haven't been used for T seconds (4 * RTT?) may be

How do you know what RTT is?  Or do you assume RTTMAX?  Since routing transients can take as long as 30 seconds to resolve, T probably would need to be significant (e.g. minutes). 

> - at least one connection should remain open to determine application

Sure.

---

From: Stefan Winter <[email-masked]>
Date: Fri, 19 Dec 2008 10:09:15 +0100

Hello,

> failover, regardless of the state of the Status-Server/Accept connection. 

Separating into two connections is a very dangerous path, IMHO. The
scenario you depicted above may *wrongly* trigger failover in a roaming
scenario. The Request/Accept connection could be unresponsive due to the
server having proxied all its requests to a downstram server. I.e. it
could be up and running, waiting itself for answers. If that connection
is closed and fail-overed, actual traffic will be discarded without
reason. If, OTOH, Status-Server is sent within this connection, a
Status-Server/Accept pair can immediately enlighten the requesting
server that the other peer is indeed running.

> be able to bring up another Status/Accept connection rather than 

I am not in favour of splitting the two connections. Fate-sharing is a
desirable property, IMHO.

Greetings,

Stefan Winter

-- 
Stefan WINTER
Ingenieur de Recherche
Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche
6, rue Richard Coudenhove-Kalergi
L-1359 Luxembourg

Tel: +352 424409 1
Fax: +352 422473

---

From: Stig Venaas <[email-masked]>
Date: Fri, 19 Dec 2008 10:46:05 +0100

Stefan Winter wrote:
[...]
> desirable property, IMHO.

+1. I think fate sharing is essential here. A separate TCP connection
also adds overhead and complexity.