Subject: [Sigtran] [M3UA] Action on Receipt of ERROR

From: "Sporton, Simon, VF-Group" <[email-masked]>
Date: Wed, 16 Mar 2011 17:06:40 +0100

Hi all,

We have been looking at various vendors implementation of M3UA and seen a significant divergence in the handling of M3UA ERROR messages, from one extreme of simply logging the event, to the other, (unilaterally) changing the state of the association (e.g. ASP-ACTIVE to ASP-DOWN). The RFC doesn't seem to provide much guidance on how to act on receipt of an ERROR message (except not to respond with an ERROR!!). Is it the deliberate intent of the authors to leave the behaviour on receipt of ERROR, or have I missed a critical paragraph in the spec?

Thanks in advance for any guidance.

Simon Sporton 
Vodafone Technology, Networks

---

From: Michael Tüxen <[email-masked]>
Date: Wed, 16 Mar 2011 20:44:41 +0100

> We have been looking at various vendors implementation of M3UA and seen a significant divergence in the handling of M3UA ERROR messages, from one extreme of simply logging the event, to the other,…
Hi Simon,

I don't think you missed anything. Looking at the error causes, I could see that for some
you might to ASP-DOWN, but not for others. I think the behavior is left implementation
dependent. Could you provide some more details, for example which error causes result
in problems?

Best regards
Michael
> _______________________________________________
> Sigtran mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sigtran
>

---

From: "David Laight" <[email-masked]>
Date: Thu, 17 Mar 2011 09:21:06 -0000

> We have been looking at various vendors implementation of 
> M3UA and seen a significant divergence in the handling of 
> M3UA ERROR messages.....

The problem I had with the M3UA MGMT ERROR messages is that
they are completely 'out of band' with any requests.
We have code to detect 'Mangement blocking' as a reponse
to an activate request (an explicit negative ack to the
activate would be much better), but otherwise assume
the MGMT errors are likely to be the result of M3UA
protocol mismatches. However we can't be certain of this
so take no immediate action.
If the remote system has rejected an important message
then a guard timer expires and we disconnect - since the
far end clearly is very confused - then retry later.

This may be a rather pragmatic solution....

	David

---

From: "Sporton, Simon, VF-Group" <[email-masked]>
Date: Thu, 17 Mar 2011 21:40:31 +0100

Hi Michael,

The particular cause value we found the problem with is cause 6 (Unexpected Message). It's quite a convoluted scenario, in essence the 2 cases we see are;

  -  a particular SGP merely logs receipt of an ERROR with Cause 6 when it received in response to DATA, and continues to send DATA towards the ASP which continues to return ERROR Cause 6, effectively black-holing traffic. I'd like to think that at some point the SGP would determine there is something wrong with ASP and send traffic to another, but that doesn't seem to be part of its logic.

 - An ASP which on receipt of an ERROR Cause 6  returns itself to the ASP-DOWN state without sending any indication to the SGP (e.g. an ASP-INACTIVE or ASP-DOWN), so the SGP continue to send traffic to a black-hole. Again, I'd like to think the ASP would have some logic to say; I'm receiving multiple DATA in ASP-DOWN there must be an inconsistency in the state machines on either side, so send an ASP-INACTIVE/ASP-DOWN.

These are particular vendor issues, but the question for here is should the RFC allow such different error handling? I assume there is good reason for the RFC to leave it to vendor implementation, but I can't work out why!

Cheers,

Simon

-----Original Message-----
From: Michael Tüxen [mailto:[email-masked]] 
Sent: 16 March 2011 20:45
To: Sporton, Simon, VF-Group
Cc: [email-masked]
Subject: Re: [Sigtran] [M3UA] Action on Receipt of ERROR

Hi Simon,

I don't think you missed anything. Looking at the error causes, I could see that for some you might to ASP-DOWN, but not for others. I think the behavior is left implementation dependent. Could you provide some more details, for example which error causes result in problems?

Best regards
Michael

---

From: Chris Benson <[email-masked]>
Date: Thu, 17 Mar 2011 13:53:07 -0700 (PDT)

Simon,

I would say that there has to be a serious protocol feature 
mismatch (or serious mis-implementation) for every DATA 
message to elicit an ERROR response.  This might not be
inherent in the M3UA layer itself, but perhaps the M3UA-user
is using a feature not mutually [externally] agreed with
its peer.  In such a case, it would be the M3UA-user
that needs to stop/change the DATA message, rather than
M3UA itself.  An example of this would be using [or not
using] Routing Context values with a peer that prohibits
[requires] Routing Context values.

I understand your concern with the lack of automatic 
correction, or deterministic resolution, but the 
issue may not be solvable within M3UA, but rather 
in the use thereof.

For the ASP case, one should note that the ASP does
not maintain its own state.  That is maintained by the
peer.  An ASP can merely request to its peer to change
the ASP-state, and the peer can refuse.

With thanks, from Chris Benson.

On Thu, 17 Mar 2011, Sporton, Simon, VF-Group wrote:

>>  Date: Thu, 17 Mar 2011 21:40:31 +0100
>>  From: "Sporton, Simon, VF-Group" <[email-masked]>
>>  To: Michael Tüxen <[email-masked]>
>>  Cc:  <[email-masked]>

---

From: "Sporton, Simon, VF-Group" <[email-masked]>
Date: Thu, 17 Mar 2011 21:54:58 +0100

Hi David,

In our case the SGP included diagnostic information in it ERROR, so the
ASP could have determined that the problem was related to a DAUD it had
previously sent, but unfortunately the ASP doesn't look at diagnostic
info! 

To be honest, your pragmatic solution would probably be good enough...
it's the logic that each node needs to determine there is a miss-match
in the state machines and re-connect to get them in sync again.

Simon

-----Original Message-----
From: David Laight [mailto:[email-masked]] 
Sent: 17 March 2011 10:21
To: Sporton, Simon, VF-Group; [email-masked]
Subject: RE: [Sigtran] [M3UA] Action on Receipt of ERROR

> We have been looking at various vendors implementation of M3UA and 
> seen a significant divergence in the handling of M3UA ERROR 
> messages.....

The problem I had with the M3UA MGMT ERROR messages is that they are
completely 'out of band' with any requests.
We have code to detect 'Mangement blocking' as a reponse to an activate
request (an explicit negative ack to the activate would be much better),
but otherwise assume the MGMT errors are likely to be the result of M3UA
protocol mismatches. However we can't be certain of this so take no
immediate action.
If the remote system has rejected an important message then a guard
timer expires and we disconnect - since the far end clearly is very
confused - then retry later.

	David

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 17 Mar 2011 22:27:27 +0100

On Mar 17, 2011, at 9:40 PM, Sporton, Simon, VF-Group wrote:

Hi Simon,

see my comments in-line.

Best regards
Michael

>  -  a particular SGP merely logs receipt of an ERROR with Cause 6 when it received in response to DATA, and continues to send DATA towards the ASP which continues to return ERROR Cause 6, effective…
So why is an ASP sending an ERROR(cause=6) in response to a DATA message. For me this looks like
a state mismatch between the SGP and the ASP.
> - An ASP which on receipt of an ERROR Cause 6  returns itself to the ASP-DOWN state without sending any indication to the SGP (e.g. an ASP-INACTIVE or ASP-DOWN), so the SGP continue to send traffic…
I think the bug here is that the ASP does not send an ASP-DOWN when it decides to go into that state due
to the reception of the ERROR.
> These are particular vendor issues, but the question for here is should the RFC allow such different error handling? I assume there is good reason for the RFC to leave it to vendor implementation,…
I'm not one of the authors, so I can only guess here. My impression of the handling of ERROR messages
is that an implementation I would write would not take any serious actions when receiving an ERROR message.
The information contained in the ERROR messages is not defined in a way that I would be able to
process them. My understanding is that they might be helpful when looking at wireshark traces...

Best regards
Michael

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 17 Mar 2011 22:29:28 +0100

> previously sent, but unfortunately the ASP doesn't look at diagnostic
... which is hard to do since the diagnostic information is not defined
in a way that it can be processed automatically...

Best regards
Michael

---

From: "Sporton, Simon, VF-Group" <[email-masked]>
Date: Thu, 17 Mar 2011 22:42:15 +0100

Hi Chris,

I agree, certainly a number of serious 'mis-implementations' were discovered when we looked into this problem! These have been taken up with the vendors and are being resolved. What I'm asking here is if there is any guidance the RFC can provide to bring error handling implementations closer together... at the moment it seems to be open season for each vendor to implement the logic they think is right, which can lead to unexpected consequences when interworking with another stack. 

Our understanding was that the ASP was sending ERROR (Unexpected Message) because it was receiving DATA while in the ASP-DOWN or ASP-ACTIVE states. I agree, the RFC says the state of remote ASPs is maintained in the SGP, but I'd assume each ASP must maintain its own state even if not formally stated in the RFC. 

Simon

-----Original Message-----
From: Chris Benson [mailto:[email-masked]] 
Sent: 17 March 2011 21:53
To: Sporton, Simon, VF-Group
Cc: Michael Tüxen; [email-masked]
Subject: Re: [Sigtran] [M3UA] Action on Receipt of ERROR

Simon,

I would say that there has to be a serious protocol feature mismatch (or serious mis-implementation) for every DATA message to elicit an ERROR response.  This might not be inherent in the M3UA layer itself, but perhaps the M3UA-user is using a feature not mutually [externally] agreed with its peer.  In such a case, it would be the M3UA-user that needs to stop/change the DATA message, rather than M3UA itself.  An example of this would be using [or not using] Routing Context values with a peer that prohibits [requires] Routing Context values.

I understand your concern with the lack of automatic correction, or deterministic resolution, but the issue may not be solvable within M3UA, but rather in the use thereof.

For the ASP case, one should note that the ASP does not maintain its own state.  That is maintained by the peer.  An ASP can merely request to its peer to change the ASP-state, and the peer can refuse.

<snip>

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 17 Mar 2011 16:26:38 -0600

Simon,

I agree with Michael in this case.  The ASP is not even following
the required (MUST) ASP Down procedures.  No amount of requirements
placed on error message handling would have fixed this.  The ASP is
simply broken.

--brian

Sporton, Simon, VF-Group wrote:              (Thu, 17 Mar 2011 22:42:15)
> Hi Chris,
> 
> I agree, certainly a number of serious 'mis-implementations' were
> discovered when we looked into this problem! These have been taken up
> with the vendors and are being resolved. What I'm asking here is if
> there is any guidance the RFC can provide to bring error handling
> implementations closer together... at the moment it seems to be open
> season for each vendor to implement the logic they think is right,
> which can lead to unexpected consequences when interworking
> with another stack. 
> 
> Our understanding was that the ASP was sending ERROR (Unexpected
> Message) because it was receiving DATA while in the ASP-DOWN or
> ASP-ACTIVE states. I agree, the RFC says the state of remote ASPs is
> maintained in the SGP, but I'd assume each ASP must maintain its own
> state even if not formally stated in the RFC. 
> 
> Simon
> 

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "David Laight" <[email-masked]>
Date: Fri, 18 Mar 2011 09:25:54 -0000

> message to elicit an ERROR response.

At a guess it is absolutely easy!
If you connect connect an ASP that conforms to rfc4666 to
an SGP that conforms to rfc3332 in loadshare, then the SGP
doesn't tell the 2nd (and subsequent) ASP the AS state.
This leads to the SGP sending DATA to the ASP in the wrong
state.

I think our code actually processes the data indications
(it routes received data using the routing label info,
without looking at any routing context), however it
won't send reponses.

We have a config option that changes the AS state when we
receive an 'asptm active ack' but haven't already seen
a 'mgmt notify' of the AS state.

	David