Subject: RE: [mpls] Re: Reg: Fast Reroute Timings in RFC 4090

From: [email-masked]
Date: Thu, 11 Aug 2005 08:10:28 +0100

Pranjal,

Total node failures (i.e. a node completely dies) will be detected by all directly connected peers as the links will go down at the physical layer. If you want to detect IP/MPLS layer failures, the IETF's protocol for doing this is BFD (see: http://www.ietf.org/html.charters/bfd-charter.html). You can run BFD on a per LSP basis (in the data plane) and also at the IP forwarding level (essentially in the control plane) and the detection time is user configurable.

However, I think its unlikely that you will be able to achieve reliable failure detection much faster than 500ms-1000ms. If the nodes supported it, you could adjust the timers to detect failures quicker than this but the network would be highly susceptible to false failures due to things like network congestion and short error bursts. Also, running multiple BFD sessions with very fast detection rates will consume a significant amount of processing resources. 

In general, when considering failure protection there is a tradeoff between network reliability and fast switchover. IMO, optimising a large SP network to protect against specific failures (i.e. MPLS node forwarding plane failures) faster than 500ms-1000ms (at the expense of processing resources and network reliability) is unlikely to be technically or commercially viable.

You mentioned packet loss for voice/video traffic in your original email, what do you think the impact of a 500ms-1000ms loss will have at the application layer? Can you give some examples?

Regards,
Richard

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]]On
> Behalf Of Dutta, Pranjal
> Sent: 11 August 2005 05:39
> To: David Charlap
> Cc: [email-masked]
> Subject: [mpls] Re: Reg: Fast Reroute Timings in RFC 4090
> 
> 
> 
> Hi,
>              I think most of the vendors implement min 1 seconds as
> hello-intervals as lesser than that might lead to scalability 
> issues (I
> am not sure if that is the reason why vendors implement at 
> least 1 sec).
> Hardware level detection will be possible only when physical layer has
> end-to-end signaling mechanism for failure. In the case where 
> end-to-end
> signaling mechanism is not possible, the PLR will still see 
> that link is
> up and keep forwarding traffic to downstream LSR, but 
> actually the link
> is failed. I am basically looking for some mechanism that can detect
> "Node Failure" as early as possible and PLR will initiate node
> protection accordingly. I think some kind of OAM mechanism might help.
> 
> Thanks,
> Pranjal
> 
> 
> 
> Date: Wed, 10 Aug 2005 10:02:20 -0400
> From: David Charlap <[email-masked]>
> Subject: Re: [mpls] Reg: Fast Reroute Timings in RFC 4090
> To: [email-masked]
> Message-ID: <[email-masked]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Dutta, Pranjal wrote:
> > 
> >                As per RFC 4090- Fast Reroute Extensions to 
> RSVP-TE for
> 
> > LSP Tunnels, it takes 10 ms to reroute the LSP at PLR either by
> facility 
> > back up or one-to-one back up method. The re-route timing is
> definitely 
> > lesser when compared to time taken for global rerouting (at Ingress 
> > LSR). My concern is PLR will reroute the LSP locally only 
> when it has 
> > the fault notification (after missing hellos from the downstream 
> > neighbor). Assuming that min 1 second of RSVP hello 
> interval is being 
> > used, it will take at least 1 second in total to re-route 
> the traffic 
> > locally which will lead to loss of voice/video packets and affects 
> > traffic performance. I would like to know if there is some effort
> going 
> > on in this group to address this issue. Any pointers will 
> be helpful.
> 
> First of all, the Hello interval may be faster than 1s, 
> depending on the
> 
> implementation.  The RFC actually recommends 5ms intervals, 
> but I don't 
> know if anyone actually implements that rate.
> 
> Second, a switch may detect a link failure by means other 
> than loss of 
> RSVP-Hello.  For instance, the hardware may detect a loss of 
> carrier on 
> the interface.  (As a matter of fact, I would expect hardware 
> detection 
> of link failure to be the only way that can reroute traffic 
> fast enough 
> for real-time applications.)
> 
> If a switch loses RSVP Hellos, but does not lose carrier, 
> then the link 
> is still physically active.  It is probably still forwarding 
> data-plane 
> traffic as well (even though the control plane has probably 
> failed.)  So
> 
> actual losses may not be nearly as severe as you are assuming.
> 
> -- David
> 
> 
> 
> ------------------------------
> 
> _______________________________________________
> mpls mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/mpls
> 
> 
> End of mpls Digest, Vol 16, Issue 7
> ***********************************
> 
> _______________________________________________
> mpls mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/mpls
> 

_______________________________________________
mpls mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mpls

---

From: [email-masked]
Date: Thu, 11 Aug 2005 09:23:37 +0100

Richard,  Of course I agree with these observations.  1 further
point.....

BFD only works on arch valid trail constructs, ie p2p and p2mp.  If you
violate the co mode by merging (ie LDP/PHP) then the
traffic/fault/performance management problems become rather difficult
(understatement)...adding ECMP to brew just makes it significantly
worse.  Check out the complexity of LSP-Ping....N years on, still a
draft (rev 9)....this fact alone should tell you something is not quite
right here.

I'd also advise that if you really want to see what simple always-on
defect detection/handling OAM SHOULD look like for the co-ps mode you
should have a look at Y.1711.  BFD is a sort-of Y.1711 look-alike (has
to be, the nature of the functional problem being addressed is the
same....so the solution has to be similar).  However, Y.1711 is complete
of specification.  Not sure how many folks implement it as nothing works
well with LDP/PHP/ECMP stuff and many folks already have lots of that.
Further, given its an ITU Rec and is written with a HW implementation in
mind then I suspect many router vendors don't like it.  However, the
functionality specified in here is correct, complete and
minimalist......nothing else exists which comes close to this.

If operators want to (i) get opex down (ii) have more customer-oriented
proactive network/service management which includes (iii) getting on the
availability/performance SLA page one day, they are going to need
something like this as a base from which to work from....else these
objectives will not be achieveable in any simple and consistently
meaningful way.

regards, Neil

> [mailto:[email-masked]] On Behalf Of 
> [email-masked]
> Sent: 11 August 2005 08:10
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: RE: [mpls] Re: Reg: Fast Reroute Timings in RFC 4090
> 
> 
> Pranjal,
> 
> Total node failures (i.e. a node completely dies) will be
> detected by all directly connected peers as the links will go 
> down at the physical layer. If you want to detect IP/MPLS 
> layer failures, the IETF's protocol for doing this is BFD 
> (see: http://www.ietf.org/html.charters/bfd-charter.html). 
> You can run BFD on a per LSP basis (in the data plane) and 
> also at the IP forwarding level (essentially in the control 
> plane) and the detection time is user configurable.
> 
> However, I think its unlikely that you will be able to
> achieve reliable failure detection much faster than 
> 500ms-1000ms. If the nodes supported it, you could adjust the 
> timers to detect failures quicker than this but the network 
> would be highly susceptible to false failures due to things 
> like network congestion and short error bursts. Also, running 
> multiple BFD sessions with very fast detection rates will 
> consume a significant amount of processing resources. 
> 
> In general, when considering failure protection there is a
> tradeoff between network reliability and fast switchover. 
> IMO, optimising a large SP network to protect against 
> specific failures (i.e. MPLS node forwarding plane failures) 
> faster than 500ms-1000ms (at the expense of processing 
> resources and network reliability) is unlikely to be 
> technically or commercially viable.
> 
> You mentioned packet loss for voice/video traffic in your
> original email, what do you think the impact of a 
> 500ms-1000ms loss will have at the application layer? Can you 
> give some examples?
> 
> Regards,
> Richard
> 
> > hello-intervals as lesser than that might lead to 
> scalability issues
> > (I am not sure if that is the reason why vendors implement at least 
> > 1 sec). Hardware level detection will be possible only when
> physical layer has

> > First of all, the Hello interval may be faster than 1s, depending on

> > the
> > 
> > implementation.  The RFC actually recommends 5ms intervals, but I 
> > don't know if anyone actually implements that rate.
> > 
> > Second, a switch may detect a link failure by means other than loss 
> > of RSVP-Hello.  For instance, the hardware may detect a loss of
> > carrier on 
> > If a switch loses RSVP Hellos, but does not lose carrier, then the 
> > link is still physically active.  It is probably still forwarding
> > data-plane 
> > [email-masked] https://www1.ietf.org/mailman/listinfo/mpls
> > 
> > 
> > [email-masked] https://www1.ietf.org/mailman/listinfo/mpls
> > 
> 
> [email-masked] https://www1.ietf.org/mailman/listinfo/mpls
>

---

From: "Dutta, Pranjal" <[email-masked]>
Date: Thu, 11 Aug 2005 19:35:55 +0530

Hi Richard,
              Please refer inline.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Thursday, 11 August, 2005 12:40 PM
To: Dutta, Pranjal; [email-masked]
Cc: [email-masked]
Subject: RE: [mpls] Re: Reg: Fast Reroute Timings in RFC 4090

Pranjal,

Total node failures (i.e. a node completely dies) will be detected by
all directly connected peers as the links will go down at the physical
layer. If you want to detect IP/MPLS layer failures, the IETF's protocol
for doing this is BFD (see:
http://www.ietf.org/html.charters/bfd-charter.html). You can run BFD on
a per LSP basis (in the data plane) and also at the IP forwarding level
(essentially in the control plane) and the detection time is user
configurable.

Pranjal: Yes, I completely agree with this. Right now it seems we are
having BFD as the most viable option.

However, I think its unlikely that you will be able to achieve reliable
failure detection much faster than 500ms-1000ms. If the nodes supported
it, you could adjust the timers to detect failures quicker than this but
the network would be highly susceptible to false failures due to things
like network congestion and short error bursts. Also, running multiple
BFD sessions with very fast detection rates will consume a significant
amount of processing resources. 

Pranjal: Along with the point you have raised I do see another thread
here.
5 ms of hello timers might solve the problem but it has to be done at
hardware layer for accurate timings and scalabilty. But to reduce hello
interval to milliseconds level, it would raise false failures due to
short congestion/error bursts when hellos are missed. We might see
instability at PLR due to this as there would be frequent switchover
between paths.

In general, when considering failure protection there is a tradeoff
between network reliability and fast switchover. IMO, optimising a large
SP network to protect against specific failures (i.e. MPLS node
forwarding plane failures) faster than 500ms-1000ms (at the expense of
processing resources and network reliability) is unlikely to be
technically or commercially viable.

Pranjal: Since we have OAM mechanisms to tackle re-route at physical
layer (like APS in SONET etc. etc), we can live with link failures to a
certain extent. The problem will be more prominent in Node failure case.
Yes,any such fast protection might not carry the OPEX/Commercial
viability. 

You mentioned packet loss for voice/video traffic in your original
email, what do you think the impact of a 500ms-1000ms loss will have at
the application layer? Can you give some examples?

Pranjal: I was pointing to some applications like MPEG-2 SD/HD video
frames which will carry at least 30 frames (I+P+B frames) a second.In
that case we see the affect in video traffic.

Thanks,
Pranjal

Regards,
Richard

---

From: [email-masked]
Date: Thu, 11 Aug 2005 15:20:04 +0100

Thanks JP,

who wrote 11 August 2005 12:00 <snipped>

> > However...and having said all that.....I am always puzzled why some 
> > folks think really rapid restoration MUST be done for 
> voice.  I can't 
> > believe anyone would clear down a call after 10s, 100s or 
> maybe even a 
> > few 1000 ms of loss.  I know there is some macho 'must equal/beat
> > SDH's
> > 50ms' mindset at work here.....but this is nuts.  I know 
> the folks who
> > set the 50ms defect detection threshold in SDH and this was NOT  
> > done to
> > accomodate voice, or any application for that matter.  It was simply
> > down to the physics and ring topologies being considered.
> >
> 
> That really depends on your targeted SLA ... If indeed the 
> goal is to  
> avoid a drop call, you're absolutely right, 1-2s is probably good  
> enough depending on the VoIP GTW. On the other hand, if you want to  
> avoid any voice quality degradation (in other words, no failure  
> detection by the user), then your rerouting time objective will  
> probably be on the order of a few tens of ms. Is there a golden  
> number one should never exceed ? Certainly not .... but a few 
> tens of  
> ms is undoubtedly a good target reasonably achievable in the vast  
> majority of the failure cases (provided a fast failure detection at  
> the lower level). Many tests have been conducted during the last few  
> years that showed a significant MOS degradation when exceeding a few  
> hundreds of ms.
> 
> JP.
NH=> If we could protect instantly that would be the best.
Unfortunately you can trigger unnecessary protection events by going too
quickly, eg an error event causing one lost OAM pkt, and thus cause more
harm than good......especially if you have to make choice on *which*
traffic to restore if there is a resource shortfall following a failure.
Voice traffic may not be the most important traffic (and BTW there is no
relationship between an application's QoS requirements and its
survivability importance).

So you need some integration period over which you make sure this is a
real failure event and not some transient error event.  And I as I noted
earlier it all depends which layer network you are considering the
failures reside in...the rule (of thumb) is as I stated before: closer
to the duct the faster, closer to the application the slower.  Further,
if you have no sense of harmonisation/alignment across the layer
networks then you can be invoking protection unnecessarily at the wrong
layer network.  This is not a good thing either.

However, I have a question.  What MPLS OAM mechanisms are you assuming
are being used here?  I am not interested in defects in the co-cs mode
layer networks (like SDH, I know about these) but I am particularly
interested to understand what MPLS layer defects you are referring to
(because these cannot be detected by lower layer networks), and what the
entry/exit criteria and consequent actions are that you would use here
and where these are defined.

Note - I note you mention SLAs...and I agree these are very important.
Now an SLA has 2 related (but quite distinguishable) parts:  A part that
defines the amount of unavailability (ie down-time) that is allowed, and
a part that defines the network transfer performance impairments that
are allowed when available (ie up-time).  So in order to define SLAs,
and please note this also includes apportioning impairment allowances to
different operators domains when interworking, then its rather important
to have some standardised metrics and their objectives agreed here for
these 2 parts of the SLA.  However, to define the Network Performance
SLA part you must know whether the trail is up or down....which means
you must have defined the entry/exit criteria for unavailability (as Net
Perf metric collection for SLA purposes must be suspended when
unavailable).....and in order to specify the availability SLA part you
must 1st have clearly defined defects with entry/exit criteria and
consequent actions.  Hence, to even talk about SLAs we must start with
knowing where the defects are defined....hence my question above.

regards, Neil