Subject: RE: [Sigtran] recommendations for M3UA/SUA BEATs when running overSCTP

From: "Haresign Lincoln" <[email-masked]>
Date: Mon, 3 Oct 2005 13:49:47 -0400

To whom it may concern:

I believe Jeff has a valid point.  To make the argument "Either the SCTP
implementation of appplication are broken.  Fix them." is not a real
valid argument.

When you deploy in the field against other people's switches, you can
not fix their switches.  And if there is a scenario that can lead to a
stuck condition because one side is badly behaved, then we should add
protection in to the implementation.

In our implementation, we are taking down the associations if the other
side is misbehaving.  We are operating outside the specification in this
case.  But I don't think any customers will complain when we explain to
them that we are protecting against lost revenue.

I agree that we should modify the recommendations to protect against all
bad scenarios.  The ITU/ANSI specifications have undergone 25 years of
editting to improve them.  However, if you fail to get agreement, I
would operate outside the recommendation in this case.  You won't find
any interoperabilty problems and you will keep your customers happy.

Regards,
Lincoln

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Brian F. G. Bidulock
Sent: Monday, October 03, 2005 1:15 PM
To: Jeff Morriss
Cc: [email-masked]
Subject: Re: [Sigtran] recommendations for M3UA/SUA BEATs when running
overSCTP

Jeff,

Jeff Morriss wrote:
(Mon, 03 Oct 2005 10:29:40)
> > 
> > 
> > Because the application is stuck.
> 
> No, the application is fine.  But (one of) the SCTP assocs it is using

> is stuck (due to a bug or whatever).

Splitting hairs.  Either the SCTP implementation of appplication are
broken.  Fix them.

> 
> > 
> > 
> > A similar bug in SS7 would cause problems too.  
> 
> Hmmm, I'm not so sure...
> 
> MTP2 has T7 whose expiry will kill the link for excessive delay of ACK

> and T6 which will kill the link due to excessive congestion.  SCTP has

> neither, though it does, as you say, have an optional (and barely
> documented) data lifetime.  (I'll have to look at how many SCTPs 
> implement that...)

No.  T7 is equivalent to the SCTP RTO.  In your situation the messages
have passed SCTP and been delivered to the user (receive buffer).  The
user is not collecting them.

If an SS7 link acknowledges messages and places them in the RB and the
MTP3 does not collect them and local processor outage is not signalled,
the same thing will occur.  There would be no T7 to expire.  The
implementation might SIB at some point, but that would be implementation
dependent.

Careful and correct design (and, yes, of course, accurate realization)
ensures that this cannot happen.  SS7 Level 2 does not acknowledge
message until they have actually been delivered to MTP3, not just the
receive buffer.

If you need such an acknowegement, use CorrelationId/CorrelationId Ack
in M3UA.

> 
> MTP3 has (optional) periodic SLTMs in case those two MTP2 safeguards 
> fail.

SLTM is not used in this way.

> M3UA has (optional) BEATs but doesn't recommend them; neither does it 
> recommend using SCTP's lifetime feature.

BEATs are more useful for flushing data, like COO/COA, CBD/CBA.
See the corrid draft.

> 
> These differences mean, to me, that M3UA (as specified) has a big hole

> in it (through which someone could drive several hundred thousand call

> failures).

I disagree.  I also agree with Michael.  BEAT should not be used in this
fashion.

> 
>  > For SS7, validation and
> > interworking testing is performed to ensure that the protocol stacks

> > are not so poorly designed.
> 
> No amount of testing and validation will uncover every single bug in 
> the system.  (And I do hope the above was a typo and you do you know 
> the difference between a bug and a design problem.)

If you know the difference between one kind of defect and another,
perhaps you know the difference between validation testing and ad hoc
testing.

> > 
> > So abort it.  Set a lifetime or a buffer threshold to abort.  But 
> > your still going to loose all buffered messages or risk duplicating 
> > them if you don't follow the corrid procedures.
> 
> I'll look into that.  But should the RFCs be updated to have some kind

> of recommendation?  I'd hate for other people/implementations to run 
> into the same problem (isn't that part of what recommendations are
for?).

That's what this mailing list is for.  I also doubt that anyone else has
your application sticky bug problem.

> 
> >>As per above, we're not talking about congestion nor queuing delay. 
> >>We're talking about a "stuck" assoc.  Those BEATs/SLTMs will _never_

> >>be responded to; in MTP3 2 such failures will cause the link to be 
> >>failed
> >>(T1.111.7 section 2.2).
> > 
> > 
> > No.  Not while in service, only during activation.  It is optional 
> > to even send SLTM while in service.
> 
> Agreed (that's why I specified that I was talking about periodioc 
> SLTMs).  But MTP3 can afford to make that optional since MTP2 has T6
and T7.

Yes, and SCTP provids M3UA with many ways to monitor the health and
performance of an SCTP association.  You can even retrieve the SRTT,
current RTO, congestion window and others that MTP3 cannot do from MTP2.

At what level of degradation of the association the application wishes
to abort it is an operational consideration that will vary from
application to application, network to network, and operator to
operator.

If it is USSD e-mail that is being transferred, I might not care if the
application sticks until Tuesday.  If it is an ISUP call setup, I might
care if it sticks for more than two seconds.

Set your SCTP lifetimes accordingly.

> 
> > I remember in the early days of SS7 the switch line modules used to 
> > nicely send an echo SLTMs even though MTP on the front-end was dead.
> > Sending heartbeats does not cure anything.  Proper design of the 
> > application does.
> 
> Sounds like that MTP was poorly designed; there's no point in having 
> health checks if they're responded to by the wrong layer.

You do not understand the SLTM.  It is not intended to be used to check
the health of MTP3.  It is intended to be used to check for digital
cross connect and loopback problems.

If you ever worked with SS7 you might know that an MTP Level 2 link will
align nicely against itself if you loop back transmit onto receive.  The
mistaken presence of a loopback on an SS7 circuit could endanger an STP
or SP aligning links, particularly during emergencies where a
significant level of traffic would otherwise be applied to the link.

The SLTM detects loopbacks on links and is performed just after
alignment and will fail the link if a loopback is detected.

Another common cross-connect problem is attaching the transmit of one
SS7 link to the receive of the other.  Another is connecting the wrong
links in a linkset to each other (that will cause problems with a
changeover, changeback or other link management is performed).  SLTM
detects these cross-connect problems too.

STLM could be performed periodically as well to detect cross-connect
problems that occurred that did not cause the link to fail, but, because
it is very unlikely that a cross-connect change would leave the SS7 link
standing, taking the link out of service due to a periodic SLTM is not
done.  It is more likely that the SLTM just got delayed.  If a
cross-connect change is made while the link is in service, it will fail
and the cross-connect problem will be discovered by the SLTM if the link
can still successfully align.  Most switches only generate a minor alarm
condition if a periodic SLTM fails.

Your insistence on BEAT is similar.  If your M3UA data messages are
stuck between M3UA and the application, BEAT will not detect that,
because it will be responded to below the blockage.  If the blockage is
below that, the BEAT will also be delayed.

Use SCTP lifetimes or the other mechanisms afforded by SCTP to monitor
that that grade of service of the association meets your expectations.

> 
> So that there are never, ever, sustained call failures?  Or at least 
> not avoidable ones?

What you consider an avoidable failure by resetting an association, a
correct implementation sees as an unnecessary failure (of the
association).

> 
> If SIGTRAN hopes to be as reliable as SS7, "the other side is broke, 
> it's their problem" is the wrong attitude to have.

But that is SS7's attitude.  Attitude and reliability it seems have
nothing to do with each other.

> > 
> > Which is precisely why congestion should be signalled, so the rest 
> > of the network can avoid sending to the overloaded component.
> 
> But it still misses the point that the application itself is _not_ 
> congested.

Yes it is.  Due to a bug maybe, but it is congested.

> > 
> > So set lifetime and abort the assocation.  But that is surely not 
> > only implementation dependent but is an operational consideration 
> > that has not place in the protocol specification.
> 
> Hmmm, I disagree.  Reading the specs would lead one to believe that 
> one is reasonably safe in letting SCTP manage the viability of the 
> assocs (and M3UA's ability to send data on them).  This could lead 
> implementors to ignore this potential problem until they run into it.

Read the SCTP Applicability RFC.

--brian

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

______________________________________________________________________
  This email message has been scanned by PineApp Mail-Secure and has
been found clean.

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 3 Oct 2005 20:17:04 +0200

Hi Lincoln,

see my comments in-line.

Best regards
Michael

> implementation of appplication are broken.  Fix them." is not a real
But if there is a bug, it is better to fix it.
> them that we are protecting against lost revenue.
If you think your peer is misbehaving, you can take down the 
association.
I would consider it valid behaviour.
The RFCs usually do not tell you what to do if your peer is broken...
> any interoperabilty problems and you will keep your customers happy.
As I said in an earlier mail: If you want to send BEAT messages, you
can do so. Every M3UA endpoint is required to send back BEAT-ACK.
It is just that I think it is of limited use, but you can have a 
different
opinion and the spec allows you what you want to do.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 3 Oct 2005 12:29:15 -0600

Lincoln,

After you abort the association, how are you handling the resulting
message loss?

--brian

Haresign Lincoln wrote:                       (Mon, 03 Oct 2005 13:49:47)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Jeff Morriss <[email-masked]>
Date: Mon, 03 Oct 2005 18:31:07 -0400

[Resending from address subscribed to the mailing list.]

Hi Lincoln,

Thank you, very nice summary.

Having been in Customer Support for, what, 7 years now, I do admit to
(in my laziness) preferring to quote spec or recommendations than having
to explain, time and again, why we are doing something outside said
specs or recommendations.

Add to that the/my idea that the recommendations should contain (most of
or at least a lot of) the knowledge built up over the years about a
particular protocol--rather than having that knowledge locked away in
each implementation or engineers' heads--and you've got why I think
there should be recommendations about this.

Regards,
-Jeff

---

From: "Tolga Asveren" <[email-masked]>
Date: Mon, 3 Oct 2005 18:25:09 -0400

> them that we are protecting against lost revenue.
[TOLGA]Actually, this can be considered as not really against the
specification(?). Afterall, an association can be brought time any time for
administrative reasons -normally one would first inactivate RKs but this is
besides the point-. OTOH, mentioning about this issue in the specification
could be useful for future implementors. Something like:
"If an anomaly on an SCTP association is detected, association MAY be torn
down. The means of detecting such situations is implementation dependent and
outside of the scope of this recommendation."

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 4 Oct 2005 03:56:27 -0600

Jeff,

Jeff Morriss wrote:                                                                   (Mon, 03 Oct 2005 18:12:28)
> there should be recommendations about this.

In IETF that is a BCP document.  M3UA is only a "Proposed Standard".  It
need much more implementation experience before it can move to
"Standard".  And then it one could write a BCP describing recommended
implementation practices outside the scope of the standard (and the
functional protocol).

I think you need to ask yourself why you are replacing an almost
perfectly functioning SS7 network with a wet behind the ears IETF
"Proposed Standard".

Sounds like a "Customer Support" nightmare to me.  But maybe you like
stomping out forest fires every day. ;)

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 4 Oct 2005 03:57:25 -0600

Tolga,

Tolga Asveren wrote:                                                                 (Mon, 03 Oct 2005 18:25:09)
> outside of the scope of this recommendation."

Cruft.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Jeff Morriss <[email-masked]>
Date: Tue, 04 Oct 2005 08:55:36 -0400

Hi Brian,

> implementation practices outside the scope of the standard (and the

Okay, point taken.

> perfectly functioning SS7 network with a wet behind the ears IETF

I do it because our customers do it; they do it because their customers 
want it.  I don't really get exposure to the questions of "why."  Cheap 
bandwidth and cheap interface cards, I guess.  (Though I do think there 
is a decent amount of misunderstanding among operators and OEMs as to 
the status of M3UA as a standard.  See Lincoln's examples.)

> stomping out forest fires every day. ;)

Welcome to my life.  :-)

(It does keep it interesting, though.)

Regards,
-Jeff