Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 1 Dec 2009 12:53:44 +0200

Adrian and all,
I've read the  draft, and I must admit that IMHO it does not resolve the problems some people (myself included) 
raised with regard to the concept during the  Hiroshima discussions. In addition, it raises new problems.

I shall try to present my problems both with the concept of separating MIPs into in- and out-MIPs
and with the proposals in the draft in an ordered way in hope that this will make the discussion more productive.

1. Which specific operational problems are addressed by separation of in- and out-MIPs?
   - The reference to the  MPLS-TP OAM Framework draft is not a good substitution
      to an explicit definition of an operational problem
   - The problem in question should not depend on specific implementations (e.g., existence of
     Fabric, ability to inspect the packets at ingress or egress (beyond inspection that is an integral
     part of the MPLS-TP data plane etc.)
   - An explicit definition of an operational problem would also facilitate comparison of a solution based on separation 
     of in- and out-MIPs with alternative solutions (e.g., sending OAM packets to the per-NE MIP to the
     next downstream NE...)

2. What happens to an out-MIP when its supporting interface is down?
    (It is pretty clear that an in-MIP disappears when its supporting interface 
    goes down, since no OAM packets would reach it.)

3. It seems that all the proposals in the draft require certain extensions to the MPLS-TP data plane,
    and that these extensions are only used for delivery of OAM packets to out-MIPs. 
    - Is this understanding correct?
    - If the answer to the previous question is positive, can we claim that OAM packets addressed to out-MIPs
      are fate-sharing with the traffic?

My 2c,
     Sasha

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Adrian Farrel
> Sent: Sunday, November 29, 2009 8:22 PM
> To: [email-masked]
> Subject: [mpls-tp] Sending OAM messages to outgoing MIPs
> 
> Hi,
> 
> We had a bit of discussion about this in Hiroshima.
> 
> The draft below is a stake in the ground to record some 
> ideas. If they are 
> useful, please pick them up and include them in whichever draft is 
> appropriate.
> 
> Thanks,
> Adrian
> ----- Original Message ----- 
> From: <[email-masked]>
> To: <[email-masked]>
> Sent: Sunday, November 29, 2009 4:45 PM
> Subject: I-D Action:draft-farrel-mpls-tp-mip-mep-map-00.txt
> 
> 
> >A New Internet-Draft is available from the on-line Internet-Drafts 
> >directories.
> >
> > Title           : Handling MPLS-TP OAM Packets Targeted at 
> Internal MIPs
> > Author(s)       : A. Farrel
> > Filename        : draft-farrel-mpls-tp-mip-mep-map-00.txt
> > Pages           : 12
> > Date            : 2009-11-29
> >
> > The Framework for Operations, Administration and Maintenance (OAM)
> > within the MPLS Transport Profile (MPLS-TP) describes how
> > Maintenance Intermediate Points (MIPs) may be situated 
> within network
> > nodes at the incoming and outgoing interfaces.
> >
> > This document describes a way of forming OAM messages so that they
> > can be targeted at incoming MIPs and outgoing MIPs, forwarded
> > correctly through the "switch fabric", and handled efficiently in
> > optimized node implementations where there is no distinction between
> > the incoming and outgoing MIP.
> >
> > The material in this document is provided for discussion within the
> > MPLS-TP community in the expectation that this idea or some similar
> > mechanism will be subsumed into a more general MPLS-TP OAM document.
> >
> > This document is a product of a joint Internet Engineering 
> Task Force
> > (IETF) / International Telecommunication Union Telecommunication
> > Standardization Sector (ITU-T) effort to include an MPLS Transport
> > Profile within the IETF MPLS and PWE3 architectures to support the
> > capabilities and functionalities of a packet transport network.
> >
> > A URL for this Internet-Draft is:
> > 
> http://www.ietf.org/internet-drafts/draft-farrel-mpls-tp-mip-m
ep-map-00.txt 

_______________________________________________
mpls-tp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mpls-tp

---

From: Adrian Farrel <[email-masked]>
Date: Tue, 01 Dec 2009 12:34:38 +0000

Hi Sasha,

> I've read the  draft, and I must admit that IMHO it does not resolve the 
> problems
> some people (myself included) raised with regard to the concept during the
> Hiroshima discussions. In addition, it raises new problems.

Excellent.
Then it has served its purpose to advance the discussion.

> I shall try to present my problems both with the concept of separating 
> MIPs into in-
> and out-MIPs and with the proposals in the draft in an ordered way in hope 
> that
> this will make the discussion more productive.
>

Well, I guess I am (willfully) a little ahead of the next revision of the 
framework draft.
In Hiroshima, Yoshinori presented his thoughts (from 
draft-koike-ietf-mpls-tp-oam-maintenance-points).

AFAIKS, these amounted to (in no particular order)
- trace the outgoing interface
  (i.e. discover which outgoing interface the data is passing through)
- checkcross-connect function
  (i.e. find out how the XC is forwarding)
- check conintinuity to the outgoing interface
  (distinguish between XC failure, outgoing i/f failure, outgoing link 
failure)
- measure XC performance
  (packet drop ratio between in and out i/f, delay across node)

>   - The problem in question should not depend on specific implementations 
> (e.g.,
>      existence of Fabric, ability to inspect the packets at ingress or 
> egress (beyond
>      inspection that is an integral part of the MPLS-TP data plane etc.)

I hear you, but I disagree. I am not suggesting that this function is 
mandatory to support in an implementation. But I am suggesting that some 
implementations will be able to provide this function, and in that case we 
should enable it.

>   - An explicit definition of an operational problem would also facilitate 
> comparison
>     of a solution based on separation of in- and out-MIPs with alternative 
> solutions
>     (e.g., sending OAM packets to the per-NE MIP to the next downstream 
> NE...)

I agree. A problem statement is critical (without it, any evaluation of a 
solution is pointless).
As I understand it, Yoshinori is working with the editors of 
draft-ietf-mpls-tp-oam-framework to include the functional elements that he 
wants to see (extracted from his draft).

>    goes down, since no OAM packets would reach it.)

If it is clear that an in-MIP disappears when the interface is down, why 
wouldn't an out-MIP also disappear when its interace is down?

> 3. It seems that all the proposals in the draft require certain extensions 
> to the
> MPLS-TP data plane, and that these extensions are only used for delivery 
> of
> OAM packets to out-MIPs.

Yes.

>    - If the answer to the previous question is positive, can we claim that 
> OAM
>       packets addressed to out-MIPs are fate-sharing with the traffic?

We can claim that to be true to exactly (an no more!) the extent that we 
claim that OAM packets addressed to the in-MIP are fate-sharing with the 
traffic.

[Personally, I think the fate-sharing of OAM messages with traffic is 
historically a myth! For example, TDM OAM messages are not fate sharing with 
traffic, they are actually in-fiber out-of-band. In MPLS-TP we are 
attempting to get closer to in-band OAM, but the very fact that the OAM 
message is not a data packet and that we need to be able to separate OAM 
from data means that we can never get perfect fate-sharing.]

Cheers,
Adrian

---

From: George Newsome <[email-masked]>
Date: Tue, 01 Dec 2009 12:02:57 -0500

Adrian,

Interesting thread, and you raise several interesting points, especially 
around what might be historic myth!

See inline anyway.

Adrian Farrel wrote:
> Hi Sasha,
> 
>> I've read the draft, and I must admit that IMHO it does not resolve
>> the problems some people (myself included) raised with regard to
>> the concept during the Hiroshima discussions. In addition, it
>> raises new problems.
> 
> Excellent.
>> I shall try to present my problems both with the concept of
>> separating MIPs into in- and out-MIPs and with the proposals in the
>> draft in an ordered way in hope that this will make the discussion
>> more productive.
>>
> Well, I guess I am (willfully) a little ahead of the next revision of 
> the framework draft.
You only need the MIP in the model if you are going to report them 
against an LSP. It seems that most of these are actually things you 
would report against the node or the interface, so wether you have a MIP 
or not is a bit irrelevant.

The MIP seems critical if you need to do per lsp loopbacks, and it is 
only that which gives rise to the addressing problem.

>>   - The problem in question should not depend on specific 
>> implementations (e.g.,
> implementations will be able to provide this function, and in that case 
> we should enable it.

I agree with Sasha. What makes OAM useful is uniformity. If there is an 
operational procedure that varies depending on what mix of equipment is 
present, then you have created a problem. If you have a procedure that 
does not need outgoing mips, then you do not need outgoing MIPS anyway.

Talking of historical left overs, outgoing monitors stem from 
technologies that (used to) have a very significant per channel failure 
rate, and loop backs were the only method of measuring across  switch. 
Todays switches are almost all massively shared computers, so the chance 
of a fault affecting a single LSP is close to zero. That leaves 
configuration errors to be thought about, and non intrusive monitors are 
quite adequate to detect the node that got it wrong.

Replicating OAM architecture designed for one set of failure modes into 
a technology that does ot have those modes seems to me to be counter 
productive.

> 
>>   - An explicit definition of an operational problem would also 
>> facilitate comparison
>>     of a solution based on separation of in- and out-MIPs with 
>> alternative solutions
>>     (e.g., sending OAM packets to the per-NE MIP to the next 
>> downstream NE...)
> 
> I agree. A problem statement is critical (without it, any evaluation of 
> a solution is pointless).
> draft-ietf-mpls-tp-oam-framework to include the functional elements that 
> he wants to see (extracted from his draft).
>
BTW, do not forget that operational cost includes both the cost of 
finding and fining a fault together with the probability of the fault. 
Low probability faults do not add significantly to operational cost 
however tiresome they may be to detect.
This is why adding up front cost and OAM complexity might be the wrong 
thing to do even if there is a demonstrable scenario that works much 
better when loopbacks are available.

>> 3. It seems that all the proposals in the draft require certain 
>> extensions to the
>> MPLS-TP data plane, and that these extensions are only used for 
>> delivery of
>> OAM packets to out-MIPs.
>>    - If the answer to the previous question is positive, can we claim 
>> that OAM
See below.

> historically a myth! For example, TDM OAM messages are not fate sharing 
> with traffic, they are actually in-fiber out-of-band. In MPLS-TP we are 
The systems that gave rise to the OAM architecture that is being 
replicated did not have OAM messages. They had fields in the traffic 
that were devoted to OAM measurements, and it is this aspect of OAM that 
MUST fate share with traffic. Technically, OAM messages in those systems 
were out of band OAM reporting, and they must NOT fate share with 
traffic or you find it hard to observe the OAM results just when you 
need them most.

The confusion is that packet technologies have messages for everything, 
and we have not been too good at separating the OAM measurement messages 
(which MUS fate share) from the AOM reporting messages which should not 
fate share. What a downstream MIP is measuring MUST fate share, but 
control messages for loopback need not fate share.

Regards,

	George

---

From: Maarten Vissers <[email-masked]>
Date: Tue, 01 Dec 2009 18:17:35 +0100

Adrian,

The file name of this draft suggests that it addresses both MIP and MEP
functions. Reading the draft I only find a description of how to address OAM
frames to MIPs. Is it the intention to extend this draft with similar
considerations for MEP functions? Would there be a similar problem for UP
MEP functions?

As you know, MEP functions are present in a NNI port in two directions.
These MEP functions consist of two parts: pro-active OAM terminating
function and a on-demand OAM terminating function. The latter function
communicates with the MIP functions.

In Ethernet OAM those two sets of MEP functions are referred to as DOWN MEP
and UP MEP functions. A UNI-N port also contains UP MEP functions. I have
added those UP MEP (U.MEP) and DOWN MEP (D.MEP) functions in your Figure 1.
In Figure 2 I have indicated the OAM flows to the U.MEP, MIP and D.MEP
functions on the Out i/f port.

             ------------------------------------------------------
            |                                                      |
            |--------------------              --------------------|
            |D.MEP - MIP - U.MEP |            | U.MEP - MIP - D.MEP|
            |                    |    ----    |                    |
     ----->-|         In         |->-| XC |->-|        Out         |->----
            |         i/f        |    ----    |        i/f         |
            |---------------------              -------------------|
            |                                                      |
             ------------------------------------------------------

      Figure 1 : Abstract Functional Representation of an MPLS-TP Node

             ------------------------------------------------------
            |                                                      |
            |--------------------              --------------------|
            |D.MEP - MIP - U.MEP |            | U.MEP - MIP - D.MEP|
            |                    |    ----    |                    |
     ----->-|         In         |->-| XC |->-|        Out         |->----
            |         i/f        |    ----    |        i/f         |
            |---------------------              -------------------|
            |                                                      |
             ------------------------------------------------------

                             |<-OAM from U.MEP to MIP--->|
     --------------------OAM to MIP--------------------->|
     ------------OAM to U.MEP-------------------->|             |-OAM from->
                                                                    D.MEP

                  Figure 2 : MPLS-TP OAM support in Out i/f ports

In Figure 3 there is only a MIP active on the In i/f port; on the Out i/f
port the U.MEP, MIP and D.MEP functions are active. The U.MEP terminates
e.g. segment LSP A and the MIP on the In i/f port is part of this segment
LSP A. The MIP on the Out i/f port is part of a higher level segment LSP B
which has no OAM access on the In i/f port.

             ------------------------------------------------------
            |                                                      |
            |--------------------              --------------------|
            |d.mep - MIP - u.mep |            | U.MEP - MIP - D.MEP|
            |                    |    ----    |                    |
     ----->-|         In         |->-| XC |->-|        Out         |->----
            |         i/f        |    ----    |        i/f         |
            |---------------------              -------------------|
            |                                                      |
             ------------------------------------------------------

     --------------------OAM to MIP-------------------->|

     ---OAM to MIP--->|
     ------------OAM to U.MEP-------------------->|             |-OAM from->
                                                                    D.MEP

                                Figure 3

Will e.g. a segment LSP A loopback OAM packet arriving at the In i/f port,
but targetted for the U.MEP at the Out i/f port experience the same problem
as a similar loopback OAM packet in your example?

Furthermore, how would a segment LSP B lookback OAM packet arriving at the
In i/f port be reaching the MIP at the Out i/f port? Is there in this case
no problem as there is no MIP on the In i/f port for this segment LSP B?

Regards,
Maarten

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Adrian Farrel
Sent: zondag 29 november 2009 19:22
To: [email-masked]
Subject: [mpls-tp] Sending OAM messages to outgoing MIPs

Hi,

The draft below is a stake in the ground to record some ideas. If they are
useful, please pick them up and include them in whichever draft is
appropriate.

Thanks,
Adrian
----- Original Message -----
From: <[email-masked]>
To: <[email-masked]>
Sent: Sunday, November 29, 2009 4:45 PM
Subject: I-D Action:draft-farrel-mpls-tp-mip-mep-map-00.txt

> Title           : Handling MPLS-TP OAM Packets Targeted at Internal MIPs
> Author(s)       : A. Farrel
> Maintenance Intermediate Points (MIPs) may be situated within network
> This document is a product of a joint Internet Engineering Task Force
http://www.ietf.org/internet-drafts/draft-farrel-mpls-tp-mip-mep-map-00.txt

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Dec 2009 17:27:37 -0000

> functions.

It's only a file name. It means nothing. You have to read the document 
title.

I just couldn't resist mep-mip-map.
I am considering the need to clean up the whole issue with a 
mep-mip-map-mop.
Maybe a muppet could work on it?

> Reading the draft I only find a description of how to address OAM

I have no plans to address that space as I think it is an architectural 
dodge to avoid admitting that the definition of MIP is deficient.

But other people are free to write any Internet-draft that they like.

Cheers,
Adrian

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 2 Dec 2009 00:21:40 -0000

Hello George,

> You only need the MIP in the model if you are going to report them against 
> an LSP.

Agree completely.

> It seems that most of these are actually things you would report against 
> the node or the interface, so wether you have a MIP or not is a bit 
> irrelevant.

No, I don't think these things are reported against a node. I think they are 
per-LSP things and i would expect them to (potentially) be different for 
different LSPs.

> The MIP seems critical if you need to do per lsp loopbacks, and it is only 
> that which gives rise to the addressing problem.

The "addressing problem" being not a problem of "how to address a MIP" but 
how to "deliver to the correct MIP"

>>> implementations (e.g., existence of Fabric, ability to inspect the 
>>> packets at ingress or egress (beyond

True, but...

If you have to choose between having a feature that you can sometimes use 
and a feature you can never use, what will you select?

If you have to choose between buying and deploying equipment optimized for 
one feature or optimized for another feature, what will you choose?

> Talking of historical left overs, outgoing monitors stem from technologies 
> that (used to) have a very significant per channel failure rate, and loop 
> backs were the only method of measuring across switch. Todays switches are 
> almost all massively shared computers, so the chance of a fault affecting 
> a single LSP is close to zero.

Ha!

Yes. I believe some of the early MPLS LSRs had some "interesting" faults 
such that the LFIB performed the wrong actions. Although these "features" 
may now have been fixed, the memory lives on (somewhat in folk myth).

> That leaves configuration errors to be thought about, and non intrusive 
> monitors are quite adequate to detect the node that
> got it wrong.

As I understand it carriers are telling us that inspection of the control 
plane or management state to determine what the forwarding plane is doing is 
not adequate. It is certainly true that *if* there is a bug such that the 
management plane sets an LFIB entry incorrectly, it is likely that there is 
an equivalent bug that causes it to report the state of the LFIB 
incorrectly.

> Replicating OAM architecture designed for one set of failure modes into a 
> technology that does ot have those modes seems to me to be counter 
> productive.

Hooray!

I could not agree more.
Let's paint that one red and put it up over our desks.

[snip]

> BTW, do not forget that operational cost includes both the cost of finding 
> and fining a fault together with the probability of the fault. Low 
> probability faults do not add significantly to operational cost however 
> tiresome they may be to detect.

This is important, and it is why overly complex OAM schemes are doomed. We a 
build them, but they will never be used.

OAM has to be simple.

On the other hand, the people to tell us what OAM function to build are 
clearly the carriers. They know what issues they want to investigate, what 
faults they are frightened on, and what OAM they are prepared to operate.

[snip]

> replicated did not have OAM messages. They had fields in the traffic that 
> were devoted to OAM measurements, and it is this aspect of OAM that MUST 
> fate share with traffic. Technically, OAM messages in those systems were 
> out of band OAM reporting, and they must NOT fate share with traffic or 
> you find it hard to observe the OAM results just when you need them most.

:-)

The grin is because you say...
"did not have OAM messages" and "OAM messages in those systems"
...which could be taken to be a small contradiction.

We are off in the weeds, and I suggest this is a conversation between two 
old duffers and not anything anyone else wants to read. But, I would make 
some observations...
1. "fields in the traffic" is an excellent way to handle in-band OAM
    except that setting those fields changes the content of the message
    and therefore changes the way the message is handled (if only 
slightlly).
    Had OAM been engineered into MPLS from day one, this would be easy.
    Arguably, we could include it in all data packets in MPLS-TP, but the
    concern is that every data packet would need to be inspected to 
determine
    the OAM it carries.
2. "fields in the traffic" carry information between sender and consumer. 
They
    constitute messages.
3. Overhead bytes are not "fields in traffic" IMHO.

> control messages for loopback need not fate share.

Indeed, and this is noted in noted in section 2.1 of the draft.

At the start of the Hiroshima meeting, we believed the only function 
requested was control of data loopback. In that case, the in-MIP could have 
controlled (through implementation-specific means) the data loopback at the 
out-interface.

In Hiroshima we were told that other features were needed that demanded 
fate-sharing through the fabric. Hence my draft. But if "we" agree that the 
features are not required, then we can drop the whole thing.

Thus the discussion of the requirements is key.

Best,
Adrian

---

From: Maarten Vissers <[email-masked]>
Date: Wed, 02 Dec 2009 10:10:49 +0100

There is an alternative solution to this MIP problem which you will notice
when you look at the complete functionality of the NNI ports on PE and P
nodes. Let me present this functionality in the figure 1 below:

---------------------------------------------------------------
|                             XC                              |
---------------------------------------------------------------
      |        |                  |        |
------|--------|------      ------|--------|------
|     |        |     |      |     |        |     |
|  E.U.MEP  M.U.MEP  |      |  E.U.MEP  M.U.MEP  |
|     |        |     |      |     |        |     |
|   E.MIP    M.MIP   |      |   E.MIP    M.MIP   |
|     |        |     |      |     |        |     |
|  E.D.MEP     |     |      |  E.D.MEP     |     |
|     |        |     |      |     |        |     |
|  M.D.MEP     |     |      |  M.D.MEP     |     |
|     |--------|     |      |     |--------|     |
|  M.D.MEP     |     |      |  M.D.MEP     |     |
|     |--------|     |      |     |--------|     |
|  (D)MUX      |     |      |  (D)MUX      |     |
|     |        |     |      |     |        |     |
|  M.D.MEP     |     |      |  M.D.MEP     |     |
|     |--------      |      |     |--------      |
|  (D)MUX            |      |  (D)MUX            |
|     |              |      |     |              |
|   D.MEP            |      |   D.MEP            |
|    |        In i/f |      |     |      Out i/f |
-----|----------------      ------|---------------
     ^                            |
     |                            V

    FIGURE 1 - MPLS-TP i/f port MEP, MIP, MUX functionality

The physical interfaces are in figure 1 located at the bottom. Starting at
the bottom and going to the XC there is the following functionality present:
1) Section DOWN MEP (D.MEP)
2) Mux/Demux function (D)MUX to multiplex/demultiplex tunnel LSPs 
3) MPLS-TP DOWN MEP (M.D.MEP) terminating a tunnel LSP
4) Mux/Demux function to multiplex/demultiplex PWs and or service LSPs
5) MPLS-TP DOWN MEP (M.D.MEP) terminating a segment LSP
6) MPLS-TP DOWN MEP (M.D>MEP) terminating an Ethernet carrying PW
7) Ethernet DOWN MEP (E.D.MEP) terminating an Ethernet tandem connection
8A) Ethernet MIP (E.MIP)
8B) MPLS-TP PW/LSP MIP (M.MIP)
9A) Ethernet UP MEP (E.U.MEP)
9B) MPLS-TP UP MEP (M.U.MEP).

The Ethernet MEP and MIP functions are present as part of the VPLS
functionality in PE nodes. As the MPLS-TP MIP function is in parallel to the
Ethernet MIP function, the interface port will support e.g. Ethernet
Loopback OAM. This OAM includes an EUI-48 value to identify the specific
E.MIP or E.MEP that has to respond to the loopback OAM. The MPLS-TP
interface port is as such capable to read this EUI-48 value, compare it with
the local EUI-48 value and then decides if the loopback frame should be
passed through or extracted. 

This same capability can be reused in MPLS-TP OAM to identify the MPLS-TP
MIP function that has to respond to e.g. the MPLS-TP loopback OAM.

Regards,
Maarten

-----Original Message-----

Hi,

Thanks,
Adrian

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 2 Dec 2009 11:13:51 +0200

Adrian hi,

Lots of thanks for a prompt response.

I will try to analyze specific problems you've listed as being relevant for splitting the MIPs into in- and out-MIPs.

In this analysis I will assume (looks like you agree with me) that an out-MIP will die when its supporting interface goes down.

I will also assume that

 *   All nodes in the network have unique identifiers ("addresses")
 *   All interfaces of a given node have some identifiers (maybe local to the supporting node)
 *   OAM messages sent to the originating MEP contain at least identification of the replying node and of the ingress interface
 *   The operator is aware of topology between the nodes (i.e., knows which node is connected to which, and via which interfaces.)

These assumptions look OK to me if we are speaking about a single administrative domain.

The validity of the last assumption may be problematic in the case when an LSP crosses multiple administrative domains with peering at the edges.  It is not clear to me whether inter-domain peering (as opposed to tunneling through a foreign domain) is a valid MPLS-TP situation.

Trace the outgoing interface of an LSP

The simplest way to achieve that would be to send OAM messages to the MIP in node of interest and to the MIP in the next downstream node. Note that if you your OAM message with TTL=x will be handled by the MIP in the node of interest, the same message with TTL=(x+1) will be handled by the MIP in the next downstream node. Do I miss something?

Check the X-connect (Fabric) block in the node of interest

My knowledge of the actual Fabric implementations is limited, but I suspect that most of them do not distinguish between different LSPs (just as George has suggested). (In fact, the modern fabrics usually deliver chunks of packets, not entire packets). So I believe that per-LSP (by definition) MIPs and MEPs are not suitable for these checks. The same applies to measurements of the Fabric performance. (George has actually told you that already.) I also doubt if these checks (applicable to specific LSR architectures should be a subject of standardization at IETF. They should be part of the equipment-specific diagnostic tools.

Distinguish between the XC failure and the outgoing link failure

IMHO the split we're talking about will not help you, because when the outgoing link fails, all its out-MIPs disappear.

Hence both the XC failure and the outgoing link failure will look exactly the same to the originator (no reply received).

Distinguish between the failure of an outgoing link and failure of the next downstream node

If the outgoing link is a P2P interface, failure of the next downstream node will most probably result in failure of the interface connecting the node of interest to the next downstream node. Hence you will not be able to see the difference. The situation may be different for a LAN interface (i.e., if you have a "Layer 2" switch between a pair of LSRs). My attempts to find out whether these interfaces are allowed in MPLS-TP did not yet yield any results - this way or another.

Measure the intra-node delay

I am not sure if you can really do that using in- and out-MIPs. Please note that in any case the treatment of the OAM messages addressed to the out-MIP is different from that of the "data packets". If this treatment adds delay, you will be measuring something that may be completely irrelevant for the customer traffic. On the other hand, if the out-MIP is located before the egress queue, you may miss delay added by this queue.

My summary (FWIW) is that, of the problems mentioned so far, some can be solved without splitting the MIPs while others cannot be solved even with such split. If this is indeed the case, what we really need (as you've mentioned in your message to Maarten) is a big mop.

My 2c,

     Sasha

________________________________________
From: Adrian Farrel [[email-masked]]
Sent: Tuesday, December 01, 2009 2:34 PM
To: Alexander Vainshtein
Cc: [email-masked]
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Hi Sasha,

Excellent.

Yes.

Cheers,
Adrian

---

From: <[email-masked]>
Date: Wed, 2 Dec 2009 10:02:52 -0000

Some wise words in here from both George and Adrian.  A couple of
comments in-line, snipped to the points: 

Adrian wrote 02 December 2009 00:22

<snipped>

GN=> I agree with Sasha. What makes OAM useful is uniformity.

NH=> Exactly so, especially for the defect detection/handling OAM,
because one needs consistent CI in a layer network and OAM is a key part
of this.  So if traffic-units/OAM-messages arrive at some arbitrary
trail termination point that trail termination point needs to be able
parse/understand the OAM in a consistent way.  However, "uniformity"
does not means that the OAM for each of the 3 network modes has to be
the same, there are some important differences....which nicely brings me
to this remark:

<snipped>

GN=> Replicating OAM architecture designed for one set of failure modes 
> > into a technology that does ot have those modes seems to me to be 
> > counter productive.
> 
AF=> Hooray!
> Let's paint that one red and put it up over our desks.

NH=> Yes please!

<snipped>

GN=> BTW, do not forget that operational cost includes both the cost of 
> > better when loopbacks are available.
AF=> This is important, and it is why overly complex OAM schemes 
> are doomed. We a build them, but they will never be used.
> 
> OAM has to be simple.
> 
> On the other hand, the people to tell us what OAM function to 
> build are clearly the carriers. They know what issues they 
> want to investigate, what faults they are frightened on, and 
> what OAM they are prepared to operate.

NH=> Absolutely right wrt 'OAM has to be simple'.  And it's worse than
Operational folks simply turning this stuff off (which I know they do).
If one has a network with some intrinsic reliability figure X say, then
when we add OAM we should not notice any change to X.  Further, we
should not introduce complex OAM that requires lots of configuration to
cater for rare defect/diagnostic cases.....we are far more likely to
make matters worse doing this. 

All we really need in a well-designed and intrinsically reliable co-ps
mode network is an end-end CV and BDI OAM flow for most defect cases.
If we also have an ability to add a simple PM flow (and LB for those who
like this) on-demand, and an ability to tap-off an end-end CV/PM flow at
intermediate nodes also on-demand, then I think we would have a fairly
optimal OAM system we could be proud of.  Note we do not require any
FDI/AIS here.  And folks should think more about simply using just CV
and BDI for prot-sw....some quite simple schemes can be defined here
that would meet many operators requirements IMO.

I say the above with some conviction because Andy Reid and I looked hard
at these issues a few years ago wrt how we would guide suppliers on the
specification of OAM/prot-sw in PBB-TE....and since MPLS-TP is also a
co-ps mode technology the general thrust of the solutions we considered
as 'optimum' should be largely the same. 

regards, Neil
<snipped to end>

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 2 Dec 2009 13:14:33 +0200

Maarten,
Could you please explain to which problem you refer as "this problem"?

I'd also like to remind you that interaction between Ethernet OAM and MPLS-TP OAM has been explicitly defined by the IESG as out of bounds for the MPLS-TP effort at IETF.

Regards,
     Sasha

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Maarten Vissers [[email-masked]]
Sent: Wednesday, December 02, 2009 11:10 AM
To: 'Adrian Farrel'; [email-masked]
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Regards,
Maarten

-----Original Message-----

Hi,

Thanks,
Adrian

---

From: Yoshinori KOIKE <[email-masked]>
Date: Wed, 02 Dec 2009 22:06:03 +0900

Hello Adrian and Sasha,

Thank you very much for the drafting and analyzing on OAM maintenance
points requirements.

Firstly, I would like to distinguish this matter between purpose and 
means. I know there are some operators who don't need this feature 
because this can make some impacts on existing implementation. However, 
MPLS C-plane/D-plane has one problem for operations that it doesn't 
clearly specify the receiving and generating point of C-plane/D-plane 
message within one NE. Considering this ambiguous ambiguity, OAM 
function cannot play an effective role, as I took up in my I-D. For 
example, an operator might not be able to judge the cause of some fault 
between our network and customer's network and the delay measurement 
result might be useless if it is measured only at the ingress IF of the 
last NE which doesn't include forwarding engine.

Unfortunately I haven't updated my I-D, but, regarding at least 
on-demand CV(loopback in my ID) and delay measurement(in my presentation 
in Hiroshima), I believe that almost all operators would like to have 
clear definition on the location of OAM maintenance points within one NE 
and both ingress&egress maintenance points, if there is no impact on 
other functions of existing systems. We can have valuable tools for 
maintenance and keep one of the most important characteristics in 
transport networks which definitely help providing reliable services to 
our customers and reduce a lot of burden in operation phase,

I think that performance of processors are rapidly developing and how 
far the additional OAM functions influence the performance of existing 
system depends on scalability and how many functions are enabled.

Therefore, I believe that including this new feature as an option is 
meaningful and necessary, when we consider the intrinsic functional 
validity.

Now "means" of introducing new feature on maintenance points are 
discussed. I fully understand that requirements are important and it is 
possible to add the necessity of requirements on every OAM function. 
However, IMHO, we can discuss the forwarding methodology itself for 
egress maintenance points before we move on to the usage of egress MP on 
other OAM functions.

See in-line plase,

Alexander Vainshtein wrote:
> I will try to analyze specific problems you've listed as being relevant 
> for splitting the MIPs into in- and out-MIPs.
> 
> In this analysis I will assume (looks like you agree with me) that an 
> out-MIP will die when its supporting interface goes down.
> 
>  
> 
> I will also assume that
> 
>     * OAM messages sent to the originating MEP contain at least
>       identification of the replying node and of the ingress interface
>     * The operator is aware of topology between the nodes (i.e., knows
>       which node is connected to which, and via which interfaces.)
> 
> The validity of the last assumption may be problematic in the case when 
> an LSP crosses multiple administrative domains with peering at the 
> edges.  It is not clear to me whether /inter-domain peering/ (as opposed 
> to /tunneling through a foreign domain/) is a valid MPLS-TP situation.

I'm sorry I don't understand your point. I think inter-domain peering 
can be both inter-vendors domain within one operator domain and 
inter-operators domain. Is this an appropriate answer to your question?

> *Trace the outgoing interface of an LSP*
> 
>     The simplest way to achieve that would be to send OAM messages to
>     the MIP in node of interest and to the MIP in the next downstream
>     node. Note that if you your OAM message with TTL=x will be handled
>     by the MIP in the node of interest, the same message with TTL=(x+1)
>     will be handled by the MIP in the next downstream node. Do I miss
>     something?

This problem is similar to the situation in on-demand CV, because from 
the operational perspective, traceroute can be realized by plural 
on-demand CV. There is no way to judge if Egress MIP has generated OAM 
packet. This can leads some wasteful work, for example on Egress 
interface card.

> *Check the X-connect (Fabric) block in the node of interest*
> 
>     My knowledge of the actual Fabric implementations is limited, but I
>     suspect that most of them do not distinguish between different LSPs
>     (just as George has suggested). (In fact, the modern fabrics usually
>     deliver chunks of packets, not entire packets). So I believe that
>     per-LSP (by definition) MIPs and MEPs are not suitable for these
>     checks. The same applies to measurements of the Fabric performance.
>     (George has actually told you that already.) I also doubt if these
>     checks (applicable to specific LSR architectures should be a subject
>     of standardization at IETF. They should be part of the
>     equipment-specific diagnostic tools.
> 

I agree. Switch fabric structure is a matter of implementation, 
therefore the diagnostic tool will/might be prepared by each vendor. 
However, the point is differentiation between equipment part(forwarding 
engine) and communication part(mapping/de-mapping and transmission)

> *Distinguish between the XC failure and the outgoing link failure*
> 
>     IMHO the split we're talking about will not help you, because when
>     the outgoing link fails, all its out-MIPs disappear.
> 
>     Hence both the XC failure and the outgoing link failure will look
>     exactly the same to the originator (no reply received).

This scenario seems to be limited to physical link failure case. There 
are other causes or problems which lead loss of CV. Moreover, there 
seems to be a implementation assumption that different layers have 
relationship in case of link fault, like between MPLS-TP LSP and MPLS-TP 
section layer.

> *Distinguish between the failure of an outgoing link and failure of the 
> next downstream node*
> 
>     If the outgoing link is a P2P interface, failure of the next
>     downstream node will most probably result in failure of the
>     interface connecting the node of interest to the next downstream
>     node. Hence you will not be able to see the difference. The
>     situation may be different for a LAN interface (i.e., if you have a
>     "Layer 2" switch between a pair of LSRs). My attempts to find out
>     whether these interfaces are allowed in MPLS-TP did not yet yield
>     any results - this way or another.

My humble comment on this observation is the same as previous one.

> *Measure the intra-node delay*
> 
>     I am not sure if you can really do that using in- and out-MIPs.
>     Please note that in any case the treatment of the OAM messages
>     addressed to the out-MIP is different from that of the "data
>     packets". If this treatment adds delay, you will be measuring
>     something that may be completely irrelevant for the customer
>     traffic. On the other hand, if the out-MIP is located before the
>     egress queue, you may miss delay added by this queue.

Does this mean that forwarding of OAM packets exactly as data packets is 
impossible at out-MIPs from the perspective of MPLS-TP architecture? If 
so, an egress maintenance point goes meaningless. I think this is the 
point we are analyzing. Considering existing implementations, there are 
at least two implementations of originating/receiving C-plane/D-plane 
message. First one is at ingress and the other one is at 
center(forwarding part). This way of thinking might lead a possibility 
to set TTL decrement point at egress from architectural perspective. In 
this case, OAM packet seems to follow the requirement "OAM packets 
exactly as data packets" because those things seem to be implementation 
matters. Can this solution solve the problem?

> My summary (FWIW) is that, /of the problems mentioned so far, some can 
> be solved without splitting the MIPs while others cannot be solved even 
> with such split/. If this is indeed the case, what we really need (as 
> you've mentioned in your message to Maarten) is a big mop.
> 
>  
> 
> My 2c,
> 
>      Sasha
> 

Best regards,

Yoshinori

> facilitate
> alternative
> extensions
> ------------------------------------------------------------------------
> 

-- 
**************************************
Yoshinori Koike
Optical Transmission Systems Development Project
First Promotion Project
NTT Network Service Systems Laboratories
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
Telephone: +81 422 59 6723
Facsimile: +81 422 59 3494
Email: [email-masked]
**************************************

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 2 Dec 2009 16:29:34 +0200

Dear Mr. Yoshinori,

I shall try to clarify the points in my message that you did not understand and, in my turn, add some questions in order to clarify your  position.

I shall snip the parts of the email that do not deal with the points that you (or I) see as requiring clarification to make the results more readable.

Hopefully this is OK with you.

My clarifications and questions shall be inserted inline and marked with magenta italics.

Regards,

     Sasha

________________________________________
From: Yoshinori KOIKE [[email-masked]]
Sent: Wednesday, December 02, 2009 3:06 PM
To: Alexander Vainshtein; Adrian Farrel
Cc: [email-masked]; [email-masked]
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Hello Adrian and Sasha,

Thank you very much for the drafting and analyzing on OAM maintenance points requirements.

Firstly, I would like to distinguish this matter between purpose and means. I know there are some operators who don't need this feature because this can make some impacts on existing implementation. However, MPLS C-plane/D-plane has one problem for operations that it doesn't clearly specify the receiving and generating point of C-plane/D-plane message within one NE.

[[Sasha]] Can you please explain why C-plane is mentioned here?

--- snipped ---
> The validity of the last assumption may be problematic in the case when an LSP crosses multiple administrative domains with peering at the
> edges.  It is not clear to me whether /inter-domain peering/ (as opposed to /tunneling through a foreign domain/) is a valid MPLS-TP situation.

I'm sorry I don't understand your point. I think inter-domain peering can be both inter-vendors domain within one operator domain and
inter-operators domain. Is this an appropriate answer to your question?

[[Sasha]] First of all, do you agree that the first 3 assumptions do not pose a problem for you?

As for the last assumption, I have been speaking solely about administrative domains crossed by a given LSP. I have assumed that within a single administrative domain the operator knows all about the network topology (i.e., which NE is connected to which, and using which interfaces).

Boundaries between sub-domains composed of NEs from different vendors do not matter as long as we deal with a single administration.

What happens when an LSP crosses a border between two administrative domains is not clear to me. I see at least two options:

 1.  Tunneling (Carriers' Carrier model). In this model, the operator that has originated the LSP cannot send OAM messages to the MIPs in the nodes in the other operator domain, because MPLS-TP only works with Pipe (or Short Pipe) models but not with the Uniform model.  As a consequence,
    *   You can find out that your LSP does not cross the foreign domain.
    *   You cannot localize the problem within the foreign domain
    *   This does not look problematic to me because in any case repair would be the responsibility of the foreign operator.
 2.  Peering at the edge: In this model:
    *   It is possible to trace the LSP thru a foreign domain and to send OAM messages to the nodes in this domain
       *   This may present security issues
    *   Usefulness of information provided in this way is somewhat problematic since:
       *   Identification of nodes and interfaces used by the foreign operator is not necessary disclosed to other operators
       *   Even if you can localize the problem, you cannot repair it.

[[Sasha]] Why do you need an egress MIP in this situation?

Please remember that if the egress link goes down, all associated egress MIPs die with it.

So if you have got a loopback reply from both the node of interest and the next downstream node, everything is OK.

If  you have got a  loopback reply from the node of interest but not from the next downstream node - you have a problem between these two nodes, and egress MIP does not help you to localize it.

[[Sasha]] Sorry, I cannot understand that. And, again, please remember that if you have a transmission problem (your egress interface goes down), all its associated egress MIPs will disappear.
>     exactly the same to the originator (no reply received).

This scenario seems to be limited to physical link failure case. There are other causes or problems which lead loss of CV.

[[Sasha]] Can you please provide an example? And please note (this has been said by many people on this list, e.g., Neil), that adding OAM functionality that covers very rare faults does not really pay.

Moreover, there seems to be a implementation assumption that different layers have relationship in case of link fault, like between MPLS-TP LSP and MPLS-TP section layer.

[[Sasha]] Please look at the ingress MIP. If its supporting ingress interface goes down, you will not receive any replies to your OAM messages because the OAM messages will not be received in the first place.

--- snipped ---

>     I am not sure if you can really do that using in- and out-MIPs. Please note that in any case the treatment of the OAM messages addressed to the out-

>    MIP is different from that of the "data packets". If this treatment adds delay, you will be measuring something that may be completely irrelevant for the

>    customer traffic. On the other hand, if the out-MIP is located before the egress queue, you may miss delay added by this queue.

Does this mean that forwarding of OAM packets exactly as data packets is impossible at out-MIPs from the perspective of MPLS-TP architecture? If
so, an egress maintenance point goes meaningless.

[[Sasha]] At least Adrian's model assumes that all decisions regarding disposition of packets is taken at ingress.

AFAIK, most implementations do it this way today. If we keep this assumption, differentiation between an OAM message addressed to an in-MIP and one that is addressed to an out-MIP requires dedicated changes in the forwarding mechanism that will not be used for forwarding data packets on the same LSP.

I think this is the point we are analyzing. Considering existing implementations, there are at least two implementations of originating/receiving C-plane/D-plane
message. First one is at ingress and the other one is at center(forwarding part). This way of thinking might lead a possibility to set TTL decrement point at egress from architectural perspective. In this case, OAM packet seems to follow the requirement "OAM packets exactly as data packets" because those things seem to be implementation matters. Can this solution solve the problem?

[[Sasha]] IMHO and FWIW it does not really help you. First of all, if you look at RFC 3443, it differentiates between "oTTL determination and check" and actual placing of oTTL in the packet. The former typically happens at ingress (as all the other checks that affect the disposition of a packet), the former can happen at ingress or at egress, but is not really important (since if the oTTL check fails, the packet will not be forwarded).

But even if you could move oTTL check to egress, you would create a new problem: you would not be able to send OAM messages to in-MIPs.

This would be probably much, much worse than not having out-MIPs.

Best regards,

Yoshinori

---

From: Maarten Vissers <[email-masked]>
Date: Wed, 02 Dec 2009 20:21:56 +0100

Sasha,

The problem to address OAM frames to the egress MIP is the problem I refer
to.

In my proposal there is no interaction between Ethernet OAM and MPLS-TP OAM.
I only note that the interface port on an MPLS-TP equipment includes
Ethernet OAM processing fucntions as part of the VPLS capability. This
implies that such interface port is capable to read a field inside a packet
which contains a 48-bit EUI-48 value and compare this value with the local
EUI-48 value. That capability to read and compare with the local EUI-48
value can be reused also when the packet contains a MPLS-TP OAM packet. As
the EUI-48 value of the In i/f port will be different from the EUI-48 value
of the Out i/f port, MPLS-TP loopback packet containing the EUI-48 value of
the In i/f port will be processed in the ingress MIP, while such packet with
the EUI-48 value of the Out i/f port will be processed in the egress MIP.

Regards,
Maarten

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]] 
Sent: woensdag 2 december 2009 12:15
To: Maarten Vissers
Cc: 'Adrian Farrel'; [email-masked]
Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

Maarten,

I'd also like to remind you that interaction between Ethernet OAM and
MPLS-TP OAM has been explicitly defined by the IESG as out of bounds for the
MPLS-TP effort at IETF.

Regards,
     Sasha

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of
Maarten Vissers [[email-masked]]
Sent: Wednesday, December 02, 2009 11:10 AM
To: 'Adrian Farrel'; [email-masked]
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Regards,
Maarten

-----Original Message-----

Hi,

Thanks,
Adrian
----- Original Message -----
From: <[email-masked]>
To: <[email-masked]>
Sent: Sunday, November 29, 2009 4:45 PM
Subject: I-D Action:draft-farrel-mpls-tp-mip-mep-map-00.txt

> within the MPLS Transport Profile (MPLS-TP) describes how Maintenance 
> Intermediate Points (MIPs) may be situated within network nodes at the 
> incoming and outgoing interfaces.
>
> This document describes a way of forming OAM messages so that they can 
> be targeted at incoming MIPs and outgoing MIPs, forwarded correctly 
> through the "switch fabric", and handled efficiently in optimized node 
> implementations where there is no distinction between the incoming and 
> outgoing MIP.
>

_______________________________________________
mpls-tp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mpls-tp=

---

From: Adrian Farrel <[email-masked]>
Date: Wed, 02 Dec 2009 19:52:35 +0000

Hey Sasha,

> out-MIP will die when its supporting interface goes down.

I agree.

>      (maybe local to the supporting node)

Combining these. All interfaces have globally unique identifiers (but these 
might not be globally known).

>      identification of the replying node and of the ingress interface

I don't see why the ingress interface ID is required or assumed.
(but I don't object to it being present)

> *   The operator is aware of topology between the nodes (i.e.,
>      knows which node is connected to which, and via which
>      interfaces.)

Disagree.
It may be semantic fiddling, but I believe the operator might be aware of 
what they believe to be the topology between nodes. However, in the absence 
of a control plane discovery and routing protocol combination providing 
information back to the operator, this information can easily be wrong 
and/or out of date, or simply missing altogether.

> These assumptions look OK to me if we are speaking about a single

We will see where our disagreement on this assumption takes us...

> an LSP crosses multiple administrative domains with peering at the edges.
> It is not clear to me whether inter-domain peering (as opposed to 
> tunneling
> through a foreign domain) is a valid MPLS-TP situation.

I think it is valid, but I also think you put a MEP at the domain boundary. 
So that particular issue would go away.

> The simplest way to achieve that would be to send OAM messages
> to the MIP in node of interest and to the MIP in the next downstream

OK, we have crashed into our disagreement on the assumption at day one. I do 
not necessarily know which interface on node n is connected to node n+1. In 
fact, I didn't expect node n+1 to be on the path. That must mean I have a 
misfiber or an XC screw-up. How do I find out through which interface the 
LSP left node n?

> My knowledge of the actual Fabric implementations is limited,

That is good. We should not be talking about implementation. This should be 
abstract.

> but I suspect that most of them do not distinguish between different
> LSPs (just as George has suggested).

I am not sure I follow this point.
A packet arrives carrying a specific label (i.e. associated with a specific 
LSP).
If the fabric does not distinguish between LSPs, how will it forward the 
packet differently from another packet with a different label?

> (In fact, the modern fabrics usually deliver chunks of packets,
> not entire packets).

And this is particularly weird. In order to correctly "deliver", there is 
going to have to be a delivery destination selection procedure. That 
procedure has to be keyed off something. That something is ultimately the 
label in the packet.

I suspect we are very far off in the weeds of implementation. What you are 
saying is that the label look up and swap is done at the incoming interface 
and that the outgoing interface is chosen at that time. From then you are 
saying that it is the fabric that delivers the packet from the incoming 
interface to the outgoing interface.

I think it is helpful to step back from implementation and describe the 
functional units. The switching is actually taking place (in this 
description) where the label lookup is performed, the label swap is done, 
and the outgoing interface is chose. The mechanism for moving the data from 
one place to another (on a backplane or whatever) is a separate issue.

So, you are describing what I (in my draft) called the "optimized 
implementation".

The point at which this *does* become important is that if *everyone* tells 
us that they are implementing the optimized approach such that all 
"switching" takes place on the incoming interface.

But please note that even in this case, we can have an in-MIP and an 
out-MIP.
The in-MIP is present only on the incoming interface.
The out-MIP is present on and distributed across all incoming (sic) 
interfaces.
The out-MIP may also be present on the outgoing interface.

> So I believe that per-LSP (by definition) MIPs and MEPs are
> not suitable for these checks.

You are saying that "because I build my box like this, other people cannot 
build their boxes differently"!

> The same applies to measurements of the Fabric performance.
> checks (applicable to specific LSR architectures should be a
> subject of standardization at IETF. They should be part of the

If it was the case that the only work was applicable inside an 
implemenation, I would agree.
If, however, there is a need to make some change to the OAM messages outside 
the box, then there is reason to standardise.

> the outgoing link fails, all its out-MIPs disappear.

No, you said that the out-MIP goes when the interface goes, not when the 
link goes.

The MIP is still there on the outgoing interface when the link has failed.

> Hence both the XC failure and the outgoing link failure will look exactly
> the same to the originator (no reply received).

> If the outgoing link is a P2P interface, failure of the next downstream
> node will most probably result in failure of the interface connecting the
> node of interest to the next downstream node. Hence you will not be
> able to see the difference.

"Most probably"?
Well, that's OK.
"Hi, Mr. Customer, I can most probably tell you whether or not you have a 
fault."

> The situation may be different for a LAN interface (i.e., if you have a

It is surely the case that the operation of MPLS-TP is independent of the 
layer over which it runs. I believe this is captured in RFC5654.

> I am not sure if you can really do that using in- and out-MIPs.

Well, surely we can do as much (or as little) as we can with OAM between any 
MEP and a pair of MIPs.

> addressed to the out-MIP is different from that of the "data packets".
> If this treatment adds delay, you will be measuring something that may
> be completely irrelevant for the customer traffic.

I think this is also an implementation issue. It is true that some 
additional checks/changes may be required on OAM packets being handled at 
local MIPs. But these checks/changes could be implemented on the fast path 
and are not significantly different from a label swap.

Frankly, if the difference in handling of packets in these two cases is 
actually measurable, then there is a clear implementation problem.

> On the other hand, if the out-MIP is located before the egress queue, you
> may miss delay added by this queue.

This is an important point and tells us where the out-MIP has to be located.

> My summary (FWIW) is that, of the problems mentioned so far, some
> can be solved without splitting the MIPs while others cannot be solved
> even with such split.

Agreed that in some implementation models this cannot be solved.
And the question is whether we therefore rule out all posibility of solving 
it.

> If this is indeed the case, what we really need (as you've mentioned in
> your message to Maarten) is a big mop.

Going to need a bucket as well.

Adrian

My 2c,

     Sasha

Hi Sasha,

Excellent.

Yes.

Cheers,
Adrian

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 3 Dec 2009 09:15:01 +0200

Adrian,

I think that you are raising an interesting problem:

What happens if the operator's view of  the network topology does not match reality?

A naive (but, probably, still correct) answer to that is "Complete mess".

I agree with you that these situations do happen, and that the operators need some OAM tools in order o verify that their understanding of the network topology is correct.  However, I think (and whatever experience I have with the traditional transport networks seems to confirm that), that the operators try to get rid of this kind of problems (fiber mis-connect, lambda mis-connect etc.) before they create the first trail/LSP in their networks.

If we are speaking about fiber connections, the connectivity is pretty much static. If we are speaking about lambda connections using ROADM, connectivity may change dynamically, and you need proactive monitoring. But in any case these problems occur (and must be resolved) at what MPLS-TP calls the Section level, not at the level of a specific LSP and/or PW.

Hence, IMHO, the real problem that must be addressed by the MPLS-TP OAM toolset is verification of the Section level connectivity in the network. And its solution does not require any MIPs at all, because, to the best of my understanding, you cannot have MIPs at the Section level. Is this correct?

Another problem you are raising refers to the situation when a labeled packet is sent to a wrong next hop node.

This problem is IMHO orthogonal to the previous one (i.e. it may happen even if the operator's understanding of the network topology at the Section level is absolutely correct). It is aggravated by the fact that the wrong node probably will not know what to do with the label it has received. Section 3.18 of RFC 3031 claims that such a packet MUST be discarded. To the best of my understanding this problem has been resolved with LSP Ping in IP/MPLS, because the wrong node will inspect the packet with the invalid label and in send back an error response to the originator.

In MPLS-TP we have been going round the bush for quite some time now, because there are plenty of situations (unidirectional LSPs, bidirectional LSPs that are not co-routed) when the there is no way to send replies to OAM messages back to the originator in a way that is somehow related to the original monitored LSP, even if the LSP is OK. As you probably remember, I have been saying for quite some time that only the "forward" OAM messages have to be fate-sharing with the LSP in question, replies can be delivered in any way. I believe that George in his email has expressed the same position. But it seems that some people still do not accept the fact that the MPLS data plane is inherently unidirectional, and all the attempts to make it look differently only result in additional problems. So maybe we should simply stop pretending, accept the hard facts of life, and look for solutions that match the reality?

My 2c,

     Sasha

________________________________________
From: Adrian Farrel [[email-masked]]
Sent: Wednesday, December 02, 2009 9:52 PM
To: Alexander Vainshtein
Cc: [email-masked]; George Newsome; Maarten Vissers
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Hey Sasha,

I agree.

Disagree.

OK, we have crashed into our disagreement on the assumption at day one. I do
not necessarily know which interface on node n is connected to node n+1. In
fact, I didn't expect node n+1 to be on the path. That must mean I have a
 misfiberor an XC screw-up. How do I find out through which interface the
LSP left node n?

"Most probably"?
Well, that's OK.

Adrian

My 2c,

     Sasha

Hi Sasha,

Excellent.

Yes.

Cheers,
Adrian

---

From: <[email-masked]>
Date: Thu, 3 Dec 2009 09:41:19 -0000

Sasha,
 
I see some important points here that I would like to make a few
comments on.......please see in-line (I've left the text as-is and not
snipped anything):

________________________________

	From: [email-masked] [mailto:[email-masked]]
On Behalf Of Alexander Vainshtein
	Sent: 03 December 2009 07:15
	To: Adrian Farrel
	Cc: [email-masked]
	Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

	Adrian, 

		What happens if the operator's view of  the network
topology does not match reality? 

	A naive (but, probably, still correct) answer to that is
"Complete mess".

	I agree with you that these situations do happen, and that the
operators need some OAM tools in order o verify that their understanding
of the network topology is correct.  However, I think (and whatever
experience I have with the traditional transport networks seems to
confirm that), that the operators try to get rid of this kind of
problems (fiber mis-connect, lambda mis-connect etc.) before they create
the first trail/LSP in their networks.  

	NH=> Agree.  But this comes back to a more fundamental point in
that we invariably have several *independent* layer networks between the
(always MUST be present) (i) top-of-stack network (ie external
aplication interfacing, eg IP) and the (ii) bottom-of-stack network (ie
where we modulate an EM wave on metallic/optical/radio media).  And
these MUST be independent layer networks for a load of
practical/commercial reasons (let alone technical/arch ones) , eg each
layer is often owned by different parties, each layer usually covers
different geographic spans, regulatory considerations force layer
independence, rapdily increasing time-constants for connection holding
as we approach the duct (else stack becomes unstable => implies
independent routing processes, and more generally 'independent
configuration', in each layer network).

	As more general observation on this problem (which has very real
capex/opex impact in practice) we should strive to reduce the number of
layer networks betwen the TOS and BOS cases.  Which is one reason why I
would consider we had failed in our work if there was a MS-PW layer
above MPLS-TP.

	It's also (somewhat more subtley) why we don't need S bit
subayering in MPLS-TP, but if it stays then it can't be left as
arbitrary....see further remarks below on misconnectivity. 

	If we are speaking about fiber connections, the connectivity is
pretty much static. If we are speaking about lambda connections using
ROADM, connectivity may change dynamically, and you need proactive
monitoring. But in any case these problems occur (and must be resolved)
at what MPLS-TP calls the Section level, not at the level of a specific
LSP and/or PW.

	Hence, IMHO, the real problem that must be addressed by the
MPLS-TP OAM toolset is verification of the Section level connectivity in
the network. And its solution does not require any MIPs at all, because,
to the best of my understanding, you cannot have MIPs at the Section
level. Is this correct?

	Another problem you are raising refers to the situation when a
labeled packet is sent to a wrong next hop node.

	This problem is IMHO orthogonal to the previous one (i.e. it may
happen even if the operator's understanding of the network topology at
the Section level is absolutely correct). 

	NH=>This is the problem of (i) inconsistent CI (here wrt to
label semantic) and (ii) having a digital wrapper (S bit) behaviour wrt
arbitrary stacking of LSPs that I keep banging-on about.  It is also why
I said that even if these somewhat broken behaviours are carried-over
into MPLS-TP we MUST have consistent OAM so that if/when LSP
misconnection occurs the receiving LSP trail termination point does know
how to correctly process the OAM whilst it is incorrectly processing the
traffic units.

	  It is aggravated by the fact that the wrong node probably will
not know what to do with the label it has received. 

	NH=> Quite....

	  Section 3.18 of RFC 3031 claims that such a packet MUST be
discarded.  

	NH=> This (discarding) will only happen if the label is not in
use where the misconnectivity has happened...if the label is in use the
traffioc will be mis-forwarded.  The probability of which of these
actions happnes is a function of (i) what % of the label space is in use
on a given link connection and (ii) how label values are selected from
the available pool on each link connection in the network.

	 To the best of my understanding this problem has been resolved
with LSP Ping in IP/MPLS, because the wrong node will inspect the packet
with the invalid label and in send back an error response to the
originator. 

	In MPLS-TP we have been going round the bush for quite some time
now, because there are plenty of situations (unidirectional LSPs,
bidirectional LSPs that are not co-routed) when the there is no way to
send replies to OAM messages back to the originator in a way that is
somehow related to the original monitored LSP, even if the LSP is OK.  

	NH=>You may have forgotten, but this is why I have always stated
that the defect detection/handling OAM in co-ps mode networks MUST be
unidirectional and NOT of a request/response nature.....there are
several reasons for this....you mention one but another is the p2mp case
and another case is what we have just discused above wrt
misconnectivity, ie traffic/OAM may quite readily leak *from* a
connection but there may be no returm path if the misconnectivity defect
is unidirectional.

	 As you probably remember, I have been saying for quite some
time that only the "forward" OAM messages have to be fate-sharing with
the LSP in question, 

	NH=> Correct...this reinforces my point above, ie why defect
detection/handling OAM MUST be unidirectional in a co-ps mode
network...its not like the cl-ps mode case we have with IP say.

	too in a different way.

	  replies can be delivered in any way. I believe that George in
his email has expressed the same position. But it seems that some people
still do not accept the fact that the MPLS data plane is inherently
unidirectional, and all the attempts to make it look differently only
result in additional problems. So maybe we should simply stop
pretending, accept the hard facts of life, and look for solutions that
match the reality? 

	NH=> Well....I keep trying to help folks understand we cannot
have a common treatment of all 3 modes (more than just OAM of
course).....maybe one day we'll accept this!  Can but hope.

	regards, Neil 

	My 2c,

	     Sasha

	

	Hey Sasha,

	I agree.
	
	>      (maybe local to the supporting node)
	
	Combining these. All interfaces have globally unique identifiers
(but these
	might not be globally known).

	
	>      knows which node is connected to which, and via which
	
	Disagree.
	It may be semantic fiddling, but I believe the operator might be
aware of
	what they believe to be the topology between nodes. However, in
the absence
	of a control plane discovery and routing protocol combination
providing
	information back to the operator, this information can easily be
wrong
	and/or out of date, or simply missing altogether.
	
	> These assumptions look OK to me if we are speaking about a
single
	> administrative domain.
	
	We will see where our disagreement on this assumption takes
us...
	
	> through a foreign domain) is a valid MPLS-TP situation.
	
	I think it is valid, but I also think you put a MEP at the
domain boundary.
	So that particular issue would go away.
	
	> will be handled by the MIP in the next downstream node. Do I
miss
	
	OK, we have crashed into our disagreement on the assumption at
day one. I do
	not necessarily know which interface on node n is connected to
node n+1. In
	fact, I didn't expect node n+1 to be on the path. That must mean
I have a
	 misfiberor an XC screw-up. How do I find out through which
interface the
	LSP left node n?
	
	> My knowledge of the actual Fabric implementations is limited,
	
	That is good. We should not be talking about implementation.
This should be
	abstract.

	I am not sure I follow this point.
	A packet arrives carrying a specific label (i.e. associated with
a specific
	LSP).
	If the fabric does not distinguish between LSPs, how will it
forward the
	packet differently from another packet with a different label?
	
	> (In fact, the modern fabrics usually deliver chunks of
packets,
	
	And this is particularly weird. In order to correctly "deliver",
there is
	going to have to be a delivery destination selection procedure.
That
	procedure has to be keyed off something. That something is
ultimately the
	label in the packet.
	
	I suspect we are very far off in the weeds of implementation.
What you are
	saying is that the label look up and swap is done at the
incoming interface
	and that the outgoing interface is chosen at that time. From
then you are
	saying that it is the fabric that delivers the packet from the
incoming
	interface to the outgoing interface.
	
	I think it is helpful to step back from implementation and
describe the
	functional units. The switching is actually taking place (in
this
	description) where the label lookup is performed, the label swap
is done,
	and the outgoing interface is chose. The mechanism for moving
the data from
	one place to another (on a backplane or whatever) is a separate
issue.
	
	So, you are describing what I (in my draft) called the
"optimized
	implementation".
	
	The point at which this *does* become important is that if
*everyone* tells
	us that they are implementing the optimized approach such that
all
	"switching" takes place on the incoming interface.
	
	But please note that even in this case, we can have an in-MIP
and an
	out-MIP.
	The in-MIP is present only on the incoming interface.
	The out-MIP is present on and distributed across all incoming
(sic)
	interfaces.
	The out-MIP may also be present on the outgoing interface.
	
	> not suitable for these checks.
	
	You are saying that "because I build my box like this, other
people cannot
	build their boxes differently"!

	If it was the case that the only work was applicable inside an
	implemenation, I would agree.
	If, however, there is a need to make some change to the OAM
messages outside
	the box, then there is reason to standardise.
	
	> the outgoing link fails, all its out-MIPs disappear.
	
	No, you said that the out-MIP goes when the interface goes, not
when the
	link goes.
	
	The MIP is still there on the outgoing interface when the link
has failed.

	> Distinguish between the failure of an outgoing link and
failure of the
	> next downstream node
downstream
	> able to see the difference.
	
	"Most probably"?
	Well, that's OK.
	"Hi, Mr. Customer, I can most probably tell you whether or not
you have a
	fault."
	
	> any results - this way or another.
	
	It is surely the case that the operation of MPLS-TP is
independent of the
	layer over which it runs. I believe this is captured in RFC5654.
	
	> I am not sure if you can really do that using in- and
out-MIPs.
	
	Well, surely we can do as much (or as little) as we can with OAM
between any
	MEP and a pair of MIPs.
	
	> addressed to the out-MIP is different from that of the "data
packets".
	> be completely irrelevant for the customer traffic.
	
	I think this is also an implementation issue. It is true that
some
	additional checks/changes may be required on OAM packets being
handled at
	local MIPs. But these checks/changes could be implemented on the
fast path
	and are not significantly different from a label swap.
	
	Frankly, if the difference in handling of packets in these two
cases is
	actually measurable, then there is a clear implementation
problem.
	
	> may miss delay added by this queue.
	
	This is an important point and tells us where the out-MIP has to
be located.

	Agreed that in some implementation models this cannot be solved.
	And the question is whether we therefore rule out all posibility
of solving
	it.

	
	Adrian

	My 2c,
	
	     Sasha

	
	
	Hi Sasha,

	Excellent.
	
	> I shall try to present my problems both with the concept of
separating
	> 1. Which specific operational problems are addressed by
separation of in-
	> and out-MIPs?
	>   - The reference to the  MPLS-TP OAM Framework draft is not a
good
	> substitution
	
	Well, I guess I am (willfully) a little ahead of the next
revision of the
	framework draft.
	In Hiroshima, Yoshinori presented his thoughts (from
	draft-koike-ietf-mpls-tp-oam-maintenance-points).
	
	AFAIKS, these amounted to (in no particular order)
	- trace the outgoing interface
	  (i.e. discover which outgoing interface the data is passing
through)
	- checkcross-connect function
	  (i.e. find out how the XC is forwarding)
	- check conintinuity to the outgoing interface
	  (distinguish between XC failure, outgoing i/f failure,
outgoing link
	failure)
	- measure XC performance
	  (packet drop ratio between in and out i/f, delay across node)
	
	>   - The problem in question should not depend on specific
implementations
	>      inspection that is an integral part of the MPLS-TP data
plane etc.)
	
	I hear you, but I disagree. I am not suggesting that this
function is
	mandatory to support in an implementation. But I am suggesting
that some
	implementations will be able to provide this function, and in
that case we
	should enable it.
	
alternative
	>     (e.g., sending OAM packets to the per-NE MIP to the next
downstream
	> NE...)
	
	I agree. A problem statement is critical (without it, any
evaluation of a
	solution is pointless).
	As I understand it, Yoshinori is working with the editors of
	draft-ietf-mpls-tp-oam-framework to include the functional
elements that he
	wants to see (extracted from his draft).
	
	>    (It is pretty clear that an in-MIP disappears when its
supporting
	> interface
	
	If it is clear that an in-MIP disappears when the interface is
down, why
	wouldn't an out-MIP also disappear when its interace is down?

	Yes.
	
	>       packets addressed to out-MIPs are fate-sharing with the
traffic?
	
	We can claim that to be true to exactly (an no more!) the extent
that we
	claim that OAM packets addressed to the in-MIP are fate-sharing
with the
	traffic.
	
	[Personally, I think the fate-sharing of OAM messages with
traffic is
	historically a myth! For example, TDM OAM messages are not fate
sharing with
	traffic, they are actually in-fiber out-of-band. In MPLS-TP we
are
	attempting to get closer to in-band OAM, but the very fact that
the OAM
	message is not a data packet and that we need to be able to
separate OAM
	from data means that we can never get perfect fate-sharing.]
	
	Cheers,
	Adrian

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 3 Dec 2009 15:43:04 +0200

Neil,
Lots of thanks for a prompt and highly informative response.

I just wanted to clarify one minor point where, it seems, you and I are not exactly on the same wavelength.

I have said (in my email to Adrian and the list) that if the wrong node receives a labeled packet, it may consider the top label invalid and hence will discard the packet.

You have noted that discard only will happen if the label is indeed invalid; if, however, the wrong node does not recognize this label as invalid (because it treats it as an incoming  label for a different LSP), it will mis-forward the packet.

You are right, of course, in the generic case.

However (and I probably should clarify that from the very beginning), I have been only considering the case when a wrong next hop node receives an OAM message intended to the MIP in the right next hop node, i.e., the incoming TTL of the received packet is 1. In this case the wrong node will not (mis-)forward the OAM packet (either due to TTL expiration if the label was valid, or because the label was invalid). Instead it will(hopefully) inspect the suspicious packet. Such inspection could identify the packet as the OAM one, and some  indication of a problem (a report to the originator or to the management system) will be (hopefully) generated.
Such a report would (again, hopefully), contain all the relevant information about the mis-behavior packet (originating LSR, reporting LSR, ingress interface of the reporting LSR, received top label, identifier of the LSP that has been monitored using this OAM packet etc.) to help the operator to understand the nature of the problem.

Hopefully these notes will be useful.

Regards,
     Sasha
________________________________
From: [email-masked] [[email-masked]]
Sent: Thursday, December 03, 2009 11:41 AM
To: Alexander Vainshtein; [email-masked]
Cc: [email-masked]
Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

Sasha,

I see some important points here that I would like to make a few comments on.......please see in-line (I've left the text as-is and not snipped anything):

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Alexander Vainshtein
Sent: 03 December 2009 07:15
To: Adrian Farrel
Cc: [email-masked]
Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Adrian,

NH=> Agree.  But this comes back to a more fundamental point in that we invariably have several *independent* layer networks between the (always MUST be present) (i) top-of-stack network (ie external aplication interfacing, eg IP) and the (ii) bottom-of-stack network (ie where we modulate an EM wave on metallic/optical/radio media).  And these MUST be independent layer networks for a load of practical/commercial reasons (let alone technical/arch ones) , eg each layer is often owned by different parties, each layer usually covers different geographic spans, regulatory considerations force layer independence, rapdily increasing time-constants for connection holding as we approach the duct (else stack becomes unstable => implies independent routing processes, and more generally 'independent configuration', in each layer network).

As more general observation on this problem (which has very real capex/opex impact in practice) we should strive to reduce the number of layer networks betwen the TOS and BOS cases.  Which is one reason why I would consider we had failed in our work if there was a MS-PW layer above MPLS-TP.

It's also (somewhat more subtley) why we don't need S bit subayering in MPLS-TP, but if it stays then it can't be left as arbitrary....see further remarks below on misconnectivity.

This problem is IMHO orthogonal to the previous one (i.e. it may happen even if the operator's understanding of the network topology at the Section level is absolutely correct).

NH=>This is the problem of (i) inconsistent CI (here wrt to label semantic) and (ii) having a digital wrapper (S bit) behaviour wrt arbitrary stacking of LSPs that I keep banging-on about.  It is also why I said that even if these somewhat broken behaviours are carried-over into MPLS-TP we MUST have consistent OAM so that if/when LSP misconnection occurs the receiving LSP trail termination point does know how to correctly process the OAM whilst it is incorrectly processing the traffic units.

  It is aggravated by the fact that the wrong node probably will not know what to do with the label it has received.

NH=> Quite....

  Section 3.18 of RFC 3031 claims that such a packet MUST be discarded.

NH=> This (discarding) will only happen if the label is not in use where the misconnectivity has happened...if the label is in use the traffioc will be mis-forwarded.  The probability of which of these actions happnes is a function of (i) what % of the label space is in use on a given link connection and (ii) how label values are selected from the available pool on each link connection in the network.

 To the best of my understanding this problem has been resolved with LSP Ping in IP/MPLS, because the wrong node will inspect the packet with the invalid label and in send back an error response to the originator.

In MPLS-TP we have been going round the bush for quite some time now, because there are plenty of situations (unidirectional LSPs, bidirectional LSPs that are not co-routed) when the there is no way to send replies to OAM messages back to the originator in a way that is somehow related to the original monitored LSP, even if the LSP is OK.

NH=>You may have forgotten, but this is why I have always stated that the defect detection/handling OAM in co-ps mode networks MUST be unidirectional and NOT of a request/response nature.....there are several reasons for this....you mention one but another is the p2mp case and another case is what we have just discused above wrt misconnectivity, ie traffic/OAM may quite readily leak *from* a connection but there may be no returm path if the misconnectivity defect is unidirectional.

 As you probably remember, I have been saying for quite some time that only the "forward" OAM messages have to be fate-sharing with the LSP in question,

NH=> Correct...this reinforces my point above, ie why defect detection/handling OAM MUST be unidirectional in a co-ps mode network...its not like the cl-ps mode case we have with IP say.

too in a different way.

  replies can be delivered in any way. I believe that George in his email has expressed the same position. But it seems that some people still do not accept the fact that the MPLS data plane is inherently unidirectional, and all the attempts to make it look differently only result in additional problems. So maybe we should simply stop pretending, accept the hard facts of life, and look for solutions that match the reality?

NH=> Well....I keep trying to help folks understand we cannot have a common treatment of all 3 modes (more than just OAM of course).....maybe one day we'll accept this!  Can but hope.

regards, Neil

My 2c,

     Sasha

Hey Sasha,

I agree.

Disagree.

"Most probably"?
Well, that's OK.

Adrian

My 2c,

     Sasha

Hi Sasha,

Excellent.

Yes.

Cheers,
Adrian

---

From: <[email-masked]>
Date: Thu, 3 Dec 2009 14:01:51 -0000

Thanks Sasha...OK, I understand your specific scenario and agree (but
personally I think TTL in a co mode network is rather silly, and just
finding cases to justify it with OAM is not a great idea IMO).
 
Aside=> If I had a *rare* query regarding the forwarding intergrity of
the fabric of some node X that was on the path of some LSP Y, then I am
not sure why simply tapping-off an end-end CV/PM flow on LSP Y at
ingress and egress ports of node X would not be sufficient to diaganose
this.  
 
regards, Neil

	From: Alexander Vainshtein
[mailto:[email-masked]] 
	Sent: 03 December 2009 13:43
	To: Harrison,N,Neil,DPF2 R; [email-masked]
	Cc: [email-masked]
	Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

	Neil,
	 
	I just wanted to clarify one minor point where, it seems, you
and I are not exactly on the same wavelength.
	 
	I have said (in my email to Adrian and the list) that if the
wrong node receives a labeled packet, it may consider the top label
invalid and hence will discard the packet. 
	 
	You have noted that discard only will happen if the label is
indeed invalid; if, however, the wrong node does not recognize this
label as invalid (because it treats it as an incoming  label for a
different LSP), it will mis-forward the packet.

	However (and I probably should clarify that from the very
beginning), I have been only considering the case when a wrong next hop
node receives an OAM message intended to the MIP in the right next hop
node, i.e., the incoming TTL of the received packet is 1. In this case
the wrong node will not (mis-)forward the OAM packet (either due to TTL
expiration if the label was valid, or because the label was invalid).
Instead it will(hopefully) inspect the suspicious packet. Such
inspection could identify the packet as the OAM one, and some
indication of a problem (a report to the originator or to the management
system) will be (hopefully) generated.
	Such a report would (again, hopefully), contain all the relevant
information about the mis-behavior packet (originating LSR, reporting
LSR, ingress interface of the reporting LSR, received top label,
identifier of the LSP that has been monitored using this OAM packet
etc.) to help the operator to understand the nature of the problem.

	Regards,
	     Sasha 

	Sasha,

		From: [email-masked]
[mailto:[email-masked]] On Behalf Of Alexander Vainshtein
		Sent: 03 December 2009 07:15
		To: Adrian Farrel
		Cc: [email-masked]
		Subject: Re: [mpls-tp] Sending OAM messages to outgoing
MIPs

		Adrian, 

			What happens if the operator's view of  the
network topology does not match reality? 

		I agree with you that these situations do happen, and
that the operators need some OAM tools in order o verify that their
understanding of the network topology is correct.  However, I think (and
whatever experience I have with the traditional transport networks seems
to confirm that), that the operators try to get rid of this kind of
problems (fiber mis-connect, lambda mis-connect etc.) before they create
the first trail/LSP in their networks.  

		NH=> Agree.  But this comes back to a more fundamental
point in that we invariably have several *independent* layer networks
between the (always MUST be present) (i) top-of-stack network (ie
external aplication interfacing, eg IP) and the (ii) bottom-of-stack
network (ie where we modulate an EM wave on metallic/optical/radio
media).  And these MUST be independent layer networks for a load of
practical/commercial reasons (let alone technical/arch ones) , eg each
layer is often owned by different parties, each layer usually covers
different geographic spans, regulatory considerations force layer
independence, rapdily increasing time-constants for connection holding
as we approach the duct (else stack becomes unstable => implies
independent routing processes, and more generally 'independent
configuration', in each layer network).

		As more general observation on this problem (which has
very real capex/opex impact in practice) we should strive to reduce the
number of layer networks betwen the TOS and BOS cases.  Which is one
reason why I would consider we had failed in our work if there was a
MS-PW layer above MPLS-TP.

		It's also (somewhat more subtley) why we don't need S
bit subayering in MPLS-TP, but if it stays then it can't be left as
arbitrary....see further remarks below on misconnectivity. 

		If we are speaking about fiber connections, the
connectivity is pretty much static. If we are speaking about lambda
connections using ROADM, connectivity may change dynamically, and you
need proactive monitoring. But in any case these problems occur (and
must be resolved) at what MPLS-TP calls the Section level, not at the
level of a specific LSP and/or PW.

		Hence, IMHO, the real problem that must be addressed by
the MPLS-TP OAM toolset is verification of the Section level
connectivity in the network. And its solution does not require any MIPs
at all, because, to the best of my understanding, you cannot have MIPs
at the Section level. Is this correct?

		Another problem you are raising refers to the situation
when a labeled packet is sent to a wrong next hop node.

		This problem is IMHO orthogonal to the previous one
(i.e. it may happen even if the operator's understanding of the network
topology at the Section level is absolutely correct). 

		NH=>This is the problem of (i) inconsistent CI (here wrt
to label semantic) and (ii) having a digital wrapper (S bit) behaviour
wrt arbitrary stacking of LSPs that I keep banging-on about.  It is also
why I said that even if these somewhat broken behaviours are
carried-over into MPLS-TP we MUST have consistent OAM so that if/when
LSP misconnection occurs the receiving LSP trail termination point does
know how to correctly process the OAM whilst it is incorrectly
processing the traffic units.

		  It is aggravated by the fact that the wrong node
probably will not know what to do with the label it has received. 

		NH=> Quite....

		  Section 3.18 of RFC 3031 claims that such a packet
MUST be discarded.  

		NH=> This (discarding) will only happen if the label is
not in use where the misconnectivity has happened...if the label is in
use the traffioc will be mis-forwarded.  The probability of which of
these actions happnes is a function of (i) what % of the label space is
in use on a given link connection and (ii) how label values are selected
from the available pool on each link connection in the network.

		 To the best of my understanding this problem has been
resolved with LSP Ping in IP/MPLS, because the wrong node will inspect
the packet with the invalid label and in send back an error response to
the originator. 

		In MPLS-TP we have been going round the bush for quite
some time now, because there are plenty of situations (unidirectional
LSPs, bidirectional LSPs that are not co-routed) when the there is no
way to send replies to OAM messages back to the originator in a way that
is somehow related to the original monitored LSP, even if the LSP is OK.

		NH=>You may have forgotten, but this is why I have
always stated that the defect detection/handling OAM in co-ps mode
networks MUST be unidirectional and NOT of a request/response
nature.....there are several reasons for this....you mention one but
another is the p2mp case and another case is what we have just discused
above wrt misconnectivity, ie traffic/OAM may quite readily leak *from*
a connection but there may be no returm path if the misconnectivity
defect is unidirectional.

		 As you probably remember, I have been saying for quite
some time that only the "forward" OAM messages have to be fate-sharing
with the LSP in question, 

		NH=> Correct...this reinforces my point above, ie why
defect detection/handling OAM MUST be unidirectional in a co-ps mode
network...its not like the cl-ps mode case we have with IP say.

		too in a different way.

		  replies can be delivered in any way. I believe that
George in his email has expressed the same position. But it seems that
some people still do not accept the fact that the MPLS data plane is
inherently unidirectional, and all the attempts to make it look
differently only result in additional problems. So maybe we should
simply stop pretending, accept the hard facts of life, and look for
solutions that match the reality? 

		NH=> Well....I keep trying to help folks understand we
cannot have a common treatment of all 3 modes (more than just OAM of
course).....maybe one day we'll accept this!  Can but hope.

		regards, Neil 

		My 2c,

		     Sasha

		
		
		Subject: Re: [mpls-tp] Sending OAM messages to outgoing
MIPs
		
		Hey Sasha,

		I agree.
		
		> *   All interfaces of a given node have some
identifiers
		>      (maybe local to the supporting node)
		
		Combining these. All interfaces have globally unique
identifiers (but these
		might not be globally known).
		
		>      identification of the replying node and of the
ingress interface
		
		I don't see why the ingress interface ID is required or
assumed.
		(but I don't object to it being present)
		
		>      knows which node is connected to which, and via
which
		
		Disagree.
		It may be semantic fiddling, but I believe the operator
might be aware of
		what they believe to be the topology between nodes.
However, in the absence
		of a control plane discovery and routing protocol
combination providing
		information back to the operator, this information can
easily be wrong
		and/or out of date, or simply missing altogether.
		
		> These assumptions look OK to me if we are speaking
about a single
		> administrative domain.
		
		We will see where our disagreement on this assumption
takes us...
		
		> through a foreign domain) is a valid MPLS-TP
situation.
		
		I think it is valid, but I also think you put a MEP at
the domain boundary.
		So that particular issue would go away.
		
		> will be handled by the MIP in the next downstream
node. Do I miss
		
		OK, we have crashed into our disagreement on the
assumption at day one. I do
		not necessarily know which interface on node n is
connected to node n+1. In
		fact, I didn't expect node n+1 to be on the path. That
must mean I have a
		 misfiberor an XC screw-up. How do I find out through
which interface the
		LSP left node n?
		
		> My knowledge of the actual Fabric implementations is
limited,
		
		That is good. We should not be talking about
implementation. This should be
		abstract.

		I am not sure I follow this point.
		A packet arrives carrying a specific label (i.e.
associated with a specific
		LSP).
		If the fabric does not distinguish between LSPs, how
will it forward the
		packet differently from another packet with a different
label?
		
		> (In fact, the modern fabrics usually deliver chunks of
packets,
		
		And this is particularly weird. In order to correctly
"deliver", there is
		going to have to be a delivery destination selection
procedure. That
		procedure has to be keyed off something. That something
is ultimately the
		label in the packet.
		
		I suspect we are very far off in the weeds of
implementation. What you are
		saying is that the label look up and swap is done at the
incoming interface
		and that the outgoing interface is chosen at that time.
>From then you are
		saying that it is the fabric that delivers the packet
from the incoming
		interface to the outgoing interface.
		
		I think it is helpful to step back from implementation
and describe the
		functional units. The switching is actually taking place
(in this
		description) where the label lookup is performed, the
label swap is done,
		and the outgoing interface is chose. The mechanism for
moving the data from
		one place to another (on a backplane or whatever) is a
separate issue.

		The point at which this *does* become important is that
if *everyone* tells
		us that they are implementing the optimized approach
such that all
		"switching" takes place on the incoming interface.
		
		But please note that even in this case, we can have an
in-MIP and an
		out-MIP.
		The in-MIP is present only on the incoming interface.
		The out-MIP is present on and distributed across all
incoming (sic)
		interfaces.
		The out-MIP may also be present on the outgoing
interface.
		
		> not suitable for these checks.
		
		You are saying that "because I build my box like this,
other people cannot
		build their boxes differently"!
		
		> equipment-specific diagnostic tools.
		
		If it was the case that the only work was applicable
inside an
		implemenation, I would agree.
		If, however, there is a need to make some change to the
OAM messages outside
		the box, then there is reason to standardise.
		
		> the outgoing link fails, all its out-MIPs disappear.
		
		No, you said that the out-MIP goes when the interface
goes, not when the
		link goes.
		
		The MIP is still there on the outgoing interface when
the link has failed.

		> Distinguish between the failure of an outgoing link
and failure of the
		> able to see the difference.
		
		"Most probably"?
		Well, that's OK.
		"Hi, Mr. Customer, I can most probably tell you whether
or not you have a
		fault."

		It is surely the case that the operation of MPLS-TP is
independent of the
		layer over which it runs. I believe this is captured in
RFC5654.
		
		> I am not sure if you can really do that using in- and
out-MIPs.
		
		Well, surely we can do as much (or as little) as we can
with OAM between any
		MEP and a pair of MIPs.
		
		> be completely irrelevant for the customer traffic.
		
		I think this is also an implementation issue. It is true
that some
		additional checks/changes may be required on OAM packets
being handled at
		local MIPs. But these checks/changes could be
implemented on the fast path
		and are not significantly different from a label swap.
		
		Frankly, if the difference in handling of packets in
these two cases is
		actually measurable, then there is a clear
implementation problem.
		
		> may miss delay added by this queue.
		
		This is an important point and tells us where the
out-MIP has to be located.
		
		> can be solved without splitting the MIPs while others
cannot be solved
		
		Agreed that in some implementation models this cannot be
solved.
		And the question is whether we therefore rule out all
posibility of solving
		it.

		
		Adrian

		My 2c,
		
		     Sasha

		
		________________________________________
		From: Adrian Farrel [[email-masked]]
		Sent: Tuesday, December 01, 2009 2:34 PM
		To: Alexander Vainshtein
		Cc: [email-masked]
		Subject: Re: [mpls-tp] Sending OAM messages to outgoing
MIPs
		
		Hi Sasha,
		
		> Hiroshima discussions. In addition, it raises new
problems.
		
		Excellent.
		Then it has served its purpose to advance the
discussion.
		
		> 1. Which specific operational problems are addressed
by separation of in-
		> and out-MIPs?
		>   - The reference to the  MPLS-TP OAM Framework draft
is not a good
		>      to an explicit definition of an operational
problem
		
		Well, I guess I am (willfully) a little ahead of the
next revision of the
		framework draft.
		In Hiroshima, Yoshinori presented his thoughts (from
		draft-koike-ietf-mpls-tp-oam-maintenance-points).
		
		AFAIKS, these amounted to (in no particular order)
		- trace the outgoing interface
		  (i.e. discover which outgoing interface the data is
passing through)
		- checkcross-connect function
		  (i.e. find out how the XC is forwarding)
		- check conintinuity to the outgoing interface
		  (distinguish between XC failure, outgoing i/f failure,
outgoing link
		failure)
		- measure XC performance
		  (packet drop ratio between in and out i/f, delay
across node)
		
		>      inspection that is an integral part of the
MPLS-TP data plane etc.)
		
		I hear you, but I disagree. I am not suggesting that
this function is
		mandatory to support in an implementation. But I am
suggesting that some
		implementations will be able to provide this function,
and in that case we
		should enable it.
		
		>     (e.g., sending OAM packets to the per-NE MIP to
the next downstream
		
		I agree. A problem statement is critical (without it,
any evaluation of a
		solution is pointless).
		As I understand it, Yoshinori is working with the
editors of
		draft-ietf-mpls-tp-oam-framework to include the
functional elements that he
		wants to see (extracted from his draft).
		
		>    (It is pretty clear that an in-MIP disappears when
its supporting
		>    goes down, since no OAM packets would reach it.)
		
		If it is clear that an in-MIP disappears when the
interface is down, why
		wouldn't an out-MIP also disappear when its interace is
down?

		Yes.
		
		>       packets addressed to out-MIPs are fate-sharing
with the traffic?
		
		We can claim that to be true to exactly (an no more!)
the extent that we
		claim that OAM packets addressed to the in-MIP are
fate-sharing with the
		traffic.
		
		[Personally, I think the fate-sharing of OAM messages
with traffic is
		historically a myth! For example, TDM OAM messages are
not fate sharing with
		traffic, they are actually in-fiber out-of-band. In
MPLS-TP we are
		attempting to get closer to in-band OAM, but the very
fact that the OAM
		message is not a data packet and that we need to be able
to separate OAM
		from data means that we can never get perfect
fate-sharing.]
		
		Cheers,
		Adrian

---

From: "BUSI ITALO" <[email-masked]>
Date: Wed, 9 Dec 2009 22:54:54 +0100

Adrian,

> The MIP is still there on the outgoing interface when the 

I am not sure the subtle difference between an interface going down and
a link going down is crystal clear. Could you provide more information?

Reading the mail thread my gut feeling is that an interface goes down
when there is an equipment failure while a link goes down when there is
a transmission failure.

Thanks, Italo

P.S. Please note my new email address, [email-masked],
effective since 1 November 2009.

> Sent: Wednesday, December 02, 2009 8:53 PM
> To: Alexander Vainshtein
> Cc: [email-masked]

---

From: "BUSI ITALO" <[email-masked]>
Date: Wed, 9 Dec 2009 23:24:11 +0100

Sasha,
 
please find my comments in line marked with [ib]
 
Italo

 

	From: [email-masked] [mailto:[email-masked]]
On Behalf Of Alexander Vainshtein
	Sent: Wednesday, December 02, 2009 3:30 PM
	To: Yoshinori KOIKE
	Cc: Adrian Farrel; [email-masked]
	Subject: Re: [mpls-tp] Sending OAM messages to outgoing MIPs

	Dear Mr. Yoshinori, 

	I shall try to clarify the points in my message that you did not
understand and, in my turn, add some questions in order to clarify your
position.

	I shall snip the parts of the email that do not deal with the
points that you (or I) see as requiring clarification to make the
results more readable.

	My clarifications and questions shall be inserted inline and
marked with magenta italics.

	Regards,

	     Sasha

	
	Hello Adrian and Sasha,
	
	Thank you very much for the drafting and analyzing on OAM
maintenance points requirements.
	
	Firstly, I would like to distinguish this matter between purpose
and means. I know there are some operators who don't need this feature
because this can make some impacts on existing implementation. However,
MPLS C-plane/D-plane has one problem for operations that it doesn't
clearly specify the receiving and generating point of C-plane/D-plane
message within one NE. 

	--- snipped ---
	>     * OAM messages sent to the originating MEP contain at
least identification of the replying node and of the ingress interface
	>     * The operator is aware of topology between the nodes
(i.e., knows which node is connected to which, and via which
interfaces.)
	> These assumptions look OK to me if we are speaking about a
single  administrative domain.
	>
	> The validity of the last assumption may be problematic in the
case when an LSP crosses multiple administrative domains with peering at
the
	> edges.  It is not clear to me whether /inter-domain peering/
(as opposed to /tunneling through a foreign domain/) is a valid MPLS-TP
situation.
	
	I'm sorry I don't understand your point. I think inter-domain
peering can be both inter-vendors domain within one operator domain and
	inter-operators domain. Is this an appropriate answer to your
question? 

	[[Sasha]] First of all, do you agree that the first 3
assumptions do not pose a problem for you?  

[ib] I think that the OAM message sent back to the originating MEP
should contain the identification of the replying MIP.

	As for the last assumption, I have been speaking solely about
administrative domains crossed by a given LSP. I have assumed that
within a single administrative domain the operator knows all about the
network topology (i.e., which NE is connected to which, and using which
interfaces).

	Boundaries between sub-domains composed of NEs from different
vendors do not matter as long as we deal with a single administration. 

	What happens when an LSP crosses a border between two
administrative domains is not clear to me. I see at least two options:

	1.	Tunneling (Carriers' Carrier model). In this model, the
operator that has originated the LSP cannot send OAM messages to the
MIPs in the nodes in the other operator domain, because MPLS-TP only
works with Pipe (or Short Pipe) models but not with the Uniform model.
As a consequence, 

		*	You can find out that your LSP does not cross
the foreign domain. 
		*	You cannot localize the problem within the
foreign domain 
		*	This does not look problematic to me because in
any case repair would be the responsibility of the foreign operator. 
			  
			[ib]  In the carrier's carrier model you still
need to know if the problem resides on the handoff link between yourself
and the foreign operator or just within the foreign operator. 
			
			[ib] This is actually another use case for the
egress MIP function. If the problem lays within the switching fabric of
the foreign operator, you should diagnose it as a problem within the
foreigh domain and NOT as the problem on the handoff link. 

		*	It is possible to trace the LSP thru a foreign
domain and to send OAM messages to the nodes in this domain 

		*	Usefulness of information provided in this way
is somewhat problematic since: 

			*	Identification of nodes and interfaces
used by the foreign operator is not necessary disclosed to other
operators 
			*	Even if you can localize the problem,
you cannot repair it.

[ib] All these issues are easily solved by instantiating a TCM within
the boundaries of the foreigner domain. All the OAM considerations
discussed above will now apply between TCMs.
The TCM allows the foreigner domain to properly manage failures within
its domain. E2E (or an higher TCM level) OAM allows you to detect
whether the problem is within the handoff links or within the foreigner
operator.

	>     will be handled by the MIP in the next downstream node. Do
I miss
	
	This problem is similar to the situation in on-demand CV,
because from
	the operational perspective, traceroute can be realized by
plural
	on-demand CV. There is no way to judge if Egress MIP has
generated OAM
	packet. This can leads some wasteful work, for example on Egress
	interface card.

	Please remember that if the egress link goes down, all
associated egress MIPs die with it.

	So if you have got a loopback reply from both the node of
interest and the next downstream node, everything is OK.

	If  you have got a  loopback reply from the node of interest but
not from the next downstream node - you have a problem between these two
nodes, and egress MIP does not help you to localize it.   

[ib] I think  the issue comes with the fact that when the egress link
goes down, the egress MIP does not die.  If this is not the case, I
agree with you that we defeat the value of egress MIP support. The
egress MIP is needed to distinguish between transmission failures and
equipment failures so it cannot die in case of transmission failures.

	
	I agree. Switch fabric structure is a matter of implementation,
	therefore the diagnostic tool will/might be prepared by each
vendor.
	However, the point is differentiation between equipment
part(forwarding
	engine) and communication part(mapping/de-mapping and
transmission)

	[[Sasha]] Sorry, I cannot understand that. And, again, please
remember that if you have a transmission problem (your egress interface
goes down), all its associated egress MIPs will disappear.  

[ib]  See above.

This scenario seems to be limited to physical link failure case. There
are other causes or problems which lead loss of CV. 

	[[Sasha]] Can you please provide an example? And please note
(this has been said by many people on this list, e.g., Neil), that
adding OAM functionality that covers very rare faults does not really
pay.

	Moreover, there seems to be a implementation assumption that
different layers have relationship in case of link fault, like between
MPLS-TP LSP and MPLS-TP section layer. 

	[[Sasha]] Please look at the ingress MIP. If its supporting
ingress interface goes down, you will not receive any replies to your
OAM messages because the OAM messages will not be received in the first
place.   

[ib] I think  that
http://tools.ietf.org/html/draft-koike-ietf-mpls-tp-oam-maintenance-poin
ts-00 clearly describes the issue we are trying to solve.
If the ingress MIP on node (n+1) does not reply, the operator cannot
understand whether this is due to a failure in the link between node (n)
and node (n+1) or to a failure within node (n).
 
Adding the support of egress MIP allows the operator to disambiguate the
case.
 
If the egress MIP on node (n) replies, then there is a failure on the
link between node (n) and node (n+1). Otherwise, egress MIP on node (n)
does not reply, there is a failure within node (n).

	--- snipped ---

	>     I am not sure if you can really do that using in- and
out-MIPs. Please note that in any case the treatment of the OAM messages
addressed to the out-

	>    MIP is different from that of the "data packets". If this
treatment adds delay, you will be measuring something that may be
completely irrelevant for the 

	>    customer traffic. On the other hand, if the out-MIP is
located before the egress queue, you may miss delay added by this queue.

	Does this mean that forwarding of OAM packets exactly as data
packets is impossible at out-MIPs from the perspective of MPLS-TP
architecture? If
	so, an egress maintenance point goes meaningless. 

	[[Sasha]] At least Adrian's model assumes that all decisions
regarding disposition of packets is taken at ingress.

	AFAIK, most implementations do it this way today. If we keep
this assumption, differentiation between an OAM message addressed to an
in-MIP and one that is addressed to an out-MIP requires dedicated
changes in the forwarding mechanism that will not be used for forwarding
data packets on the same LSP.  

	I think this is the point we are analyzing. Considering existing
implementations, there are at least two implementations of
originating/receiving C-plane/D-plane
	message. First one is at ingress and the other one is at
center(forwarding part). This way of thinking might lead a possibility
to set TTL decrement point at egress from architectural perspective. In
this case, OAM packet seems to follow the requirement "OAM packets
exactly as data packets" because those things seem to be implementation
matters. Can this solution solve the problem?

	[[Sasha]] IMHO and FWIW it does not really help you. First of
all, if you look at RFC 3443, it differentiates between "oTTL
determination and check" and actual placing of oTTL in the packet. The
former typically happens at ingress (as all the other checks that affect
the disposition of a packet), the former can happen at ingress or at
egress, but is not really important (since if the oTTL check fails, the
packet will not be forwarded).

	But even if you could move oTTL check to egress, you would
create a new problem: you would not be able to send OAM messages to
in-MIPs.

	This would be probably much, much worse than not having
out-MIPs.

[ib] I do not think the solution outlined by Adrian in section 3 of his
draft suffers from this issue.

	
	
	Best regards,
	
	Yoshinori
	
	>  > some people (myself included) raised with regard to the
concept
	> during the
separating
	>  > MIPs into in-
	>  > and out-MIPs and with the proposals in the draft in an
ordered way in
	> hope
separation of in-
	>  > and out-MIPs?
	>  >   - The reference to the  MPLS-TP OAM Framework draft is
not a good
revision of the
through)
outgoing link
implementations
function is
that some
that case we
evaluation of a
elements that he
supporting
down, why
fate-sharing with the
traffic is
are
separate OAM

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 10 Dec 2009 08:13:07 +0200

Italo, Adrian and all,

I wanted to ask the same question (i.e. what is the difference between interface going down vs. link going down). However, my interpretation is different from Italo's.

IMHO and FWIW, the difference is relevant when/if you deal with an interface that is comprised of multiple physical links using some form of inverse multiplexing (like ML-PPP, IMA or LAG). It is not clear to me whether such interfaces are allowed in MPLS-TP, and under which restrictions.

However, for an interface that represents a single physical link both transmission failure and equipment failure result in exactly the same behavior for an in-MIP (since the OAM packets will not be received from a broken link even if its equipment component is alive). Hence my expectation is that the same should apply to an out-MIP - provided such a beast exists. Personally I doubt it does, because you would have to modify the MPLS data plane in a specific way in order to deliver the packets to an out-MIP.

My 2c,

     Sasha

________________________________________
From: BUSI ITALO [[email-masked]]
Sent: Wednesday, December 09, 2009 11:54 PM
To: Adrian Farrel; Alexander Vainshtein
Cc: [email-masked]
Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

Adrian,

Thanks, Italo

> concept during the
> ordered way in hope

---

From: "BUSI ITALO" <[email-masked]>
Date: Thu, 10 Dec 2009 09:15:09 +0100

Sasha,
 
our different interpretation is affecting the required behavior when the
outgoing interface goes down. Let's park this point until we have a
clarification from Adrian and focuse on the required behavior when the
outgoing link goes down.
 
In order to address the problem identified in
http://tools.ietf.org/html/draft-koike-ietf-mpls-tp-oam-maintenance-poin
ts-00 it is important that the egress MIP is alive in case of a failure
on the outgoing link.
 
The fact that the egress MIP replies and the ingress MIP on the next hop
does not is a clear indication to the operator that the failure is
located on the link between the two nodes and not within the node.
 
Link failures affect replies from ingress MIPs. Node failures affect
replies from egress MIPs.
 
Italo

 

	From: Alexander Vainshtein
[mailto:[email-masked]] 
	Sent: Thursday, December 10, 2009 7:13 AM
	To: BUSI ITALO; Adrian Farrel
	Cc: [email-masked]
	Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

	Italo, Adrian and all,

	I wanted to ask the same question (i.e. what is the difference
between interface going down vs. link going down). However, my
interpretation is different from Italo's.

	IMHO and FWIW, the difference is relevant when/if you deal with
an interface that is comprised of multiple physical links using some
form of inverse multiplexing (like ML-PPP, IMA or LAG). It is not clear
to me whether such interfaces are allowed in MPLS-TP, and under which
restrictions.

	However, for an interface that represents a single physical link
both transmission failure and equipment failure result in exactly the
same behavior for an in-MIP (since the OAM packets will not be received
from a broken link even if its equipment component is alive). Hence my
expectation is that the same should apply to an out-MIP - provided such
a beast exists. Personally I doubt it does, because you would have to
modify the MPLS data plane in a specific way in order to deliver the
packets to an out-MIP. 

	My 2c,

	     Sasha

	
	
	Adrian,

	> The MIP is still there on the outgoing interface when the
	
	I am not sure the subtle difference between an interface going
down and
	a link going down is crystal clear. Could you provide more
information?
	
	Reading the mail thread my gut feeling is that an interface goes
down
	when there is an equipment failure while a link goes down when
there is
	a transmission failure.
	
	Thanks, Italo
	
	P.S. Please note my new email address,
[email-masked],
	effective since 1 November 2009.

	> > *   OAM messages sent to the originating MEP contain at
least
assumed.
	> > *   The operator is aware of topology between the nodes
(i.e.,
single
us...
procedure. That
this
separate issue.
"optimized
all
and an
(sic)
RFC5654.
out-MIPs.
some
problem.
solved.
	> >   - The reference to the  MPLS-TP OAM Framework draft is not
a good
through)
outgoing link
function is
supporting
traffic is
are

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 10 Dec 2009 10:38:08 +0200

Italo,
Lots of thanks for a prompt response and clarification of your approach.

I think I understand the problem you try to solve, but I do not think that your approach is correct.
I shall expand on this subject later if you do not mind.

Regards,
     Sasha

________________________________
From: BUSI ITALO [mailto:[email-masked]]
Sent: Thursday, December 10, 2009 10:15 AM
To: Alexander Vainshtein; Adrian Farrel
Cc: [email-masked]
Subject: RE: [mpls-tp] Sending OAM messages to outgoing MIPs

Sasha,

our different interpretation is affecting the required behavior when the outgoing interface goes down. Let's park this point until we have a clarification from Adrian and focuse on the required behavior when the outgoing link goes down.

In order to address the problem identified in http://tools.ietf.org/html/draft-koike-ietf-mpls-tp-oam-maintenance-points-00 it is important that the egress MIP is alive in case of a failure on the outgoing link.

The fact that the egress MIP replies and the ingress MIP on the next hop does not is a clear indication to the operator that the failure is located on the link between the two nodes and not within the node.

Link failures affect replies from ingress MIPs. Node failures affect replies from egress MIPs.

Italo

P.S. Please note my new email address, [email-masked], effective since 1 November 2009.

Italo, Adrian and all,

My 2c,

     Sasha

Adrian,

Thanks, Italo

---

From: [email-masked]
Date: Thu, 10 Dec 2009 19:44:25 +0800

hi, everyone:
 IMO, is it necessary to solve the problem? if it is the same way for the 
operator to deal with the problem of the link is going down and in-mip in 
the next node is going down, why can we cost too time to identify and 
individe the two failures conditions?

 may my understanding be wrong ? thank you!

best regards
liu

Alexander Vainshtein <[email-masked]> 
发件人:  [email-masked]
2009-12-10 16:38

收件人
BUSI ITALO <[email-masked]>
抄送
Adrian Farrel <[email-masked]>, "[email-masked]" 
<[email-masked]>
主题
Re: [mpls-tp] Sending OAM messages to outgoing MIPs

Italo,
 
I think I understand the problem you try to solve, but I do not think that 
your approach is correct. 
I shall expand on this subject later if you do not mind.
 
Regards,
     Sasha

Sasha,

In order to address the problem identified in 
http://tools.ietf.org/html/draft-koike-ietf-mpls-tp-oam-maintenance-points-00 
it is important that the egress MIP is alive in case of a failure on the 
outgoing link.
 
The fact that the egress MIP replies and the ingress MIP on the next hop 
does not is a clear indication to the operator that the failure is located 
on the link between the two nodes and not within the node.

Italo

Italo, Adrian and all,
I wanted to ask the same question (i.e. what is the difference between 
interface going down vs. link going down). However, my interpretation is 
different from Italo's.
 
IMHO and FWIW, the difference is relevant when/if you deal with an 
interface that is comprised of multiple physical links using some form of 
inverse multiplexing (like ML-PPP, IMA or LAG). It is not clear to me 
whether such interfaces are allowed in MPLS-TP, and under which 
restrictions.
 
However, for an interface that represents a single physical link both 
transmission failure and equipment failure result in exactly the same 
behavior for an in-MIP (since the OAM packets will not be received from a 
broken link even if its equipment component is alive). Hence my 
expectation is that the same should apply to an out-MIP - provided such a 
beast exists. Personally I doubt it does, because you would have to modify 
the MPLS data plane in a specific way in order to deliver the packets to 
an out-MIP. 
 
My 2c,
     Sasha

Adrian,

Thanks, Italo

mpls-tp mailing list
[email-masked]

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail is solely property of the sender's organization. This mail communication is confidential. Recipients named above are obligated to maintain secrecy and are not permitted to disclose the contents of this communication to others.
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the originator of the message. Any views expressed in this message are those of the individual sender.
This message has been scanned for viruses and Spam by ZTE Anti-Spam system.

---

From: <[email-masked]>
Date: Mon, 14 Dec 2009 09:01:01 +0000

On 10/12/2009 08:15, "BUSI ITALO" <[email-masked]> wrote:
> The fact that the egress MIP replies and the ingress MIP on the next hop does
> not is a clear indication to the operator that the failure is located on the
> link between the two nodes and not within the node.

In reality how specific does the OAM really need to be? Automatic
identification down to functional component level in a piece of equipment
seems like overkill to me. All these MIPs need managing & that in itself
causes operations overhead. If the proactive OAM can identify a fault
location down to one of either a link/node, operations personnel can then
use on demand OAM & other tools to further narrow down the specific
functional element (or physical element) causing the fault.

I am fearful that if we are not careful we will end up building pretty
complex solutions (which in my mind equates to additional operational cost &
equipment complexity/cost) to solve problems that can be solved in simpler
ways using a combination of human intelligence & on demand operations tools.

Ben

---

From: <[email-masked]>
Date: Mon, 14 Dec 2009 09:29:06 -0000

Ben wrote on 14 December 2009 09:01:

> > our different interpretation is affecting the required 
> behavior when 
> > the outgoing interface goes down. Let's park this point 
> until we have 
> > a clarification from Adrian and focuse on the required 
> behavior when 
> > the outgoing link goes down.
> > 
> http://tools.ietf.org/html/draft-koike-ietf-mpls-tp-oam-maintenance-po
> > ints-00 it is important that the egress MIP is alive in case of a 
> > failure on the outgoing link.
> > 
> > The fact that the egress MIP replies and the ingress MIP on 
> the next 
> > hop does not is a clear indication to the operator that the 
> failure is 
> In reality how specific does the OAM really need to be? 
> Automatic identification down to functional component level 
> in a piece of equipment seems like overkill to me. All these 
> MIPs need managing & that in itself causes operations 
> overhead. If the proactive OAM can identify a fault location 
> down to one of either a link/node, operations personnel can 
> then use on demand OAM & other tools to further narrow down 
> the specific functional element (or physical element) causing 
> the fault.
> 
> I am fearful that if we are not careful we will end up 
> building pretty complex solutions (which in my mind equates 
> to additional operational cost & equipment complexity/cost) 
> to solve problems that can be solved in simpler ways using a 
> combination of human intelligence & on demand operations tools.

NH=> Well, as you know Ben, I totally agree with you on this point.  And
this is also the position Andy Reid and I came to many years ago when we
considered what would be the ideal OAM for the co-ps mode....our view
has not changed since then. I have been consistently saying for many
years in both IETF and ITU that the *defect detection/handling* OAM
needs to be as simple/sparse as possible for the network mode in
question because it must be uber reliable.....far more reliable than the
network the OAM operates in.

If a system (eg a network) MUST be intrinsically highly reliable to be
fit-for-purpose in the first place, then one does not need to overload
that system with lots of always-on defect detecting/handling OAM 'just
in case'.  Not only is there an unnecessary capex/opex cost associated
with providing all this unnecessary instrumentation, but there is a very
real danger (I'd wager almost certainty) of making the system less
reliable. 

Given I can always detect a problem with an end-end OAM flow (ie between
LSP trail termination points) then I can use this information to look
deeper into the network on the **rare** occasions that this is required,
eg I don't see why the ability to do an on-demand tap-off of a CV/PM
flow at ingress and egress ports of node is not good enough to diagnose
a problem down to a specific node....can anyone explain why this is NOT
true?

regards, Neil

---

From: Alexander Vainshtein <[email-masked]>
Date: Mon, 14 Dec 2009 11:51:08 +0200

Neil, Ben
I fully agree with your statement that OAM must be very reliable - and hence as simple as possible - to be useful.

IMHO there is a plethora of examples of over-designed OAM mechanisms that eventually result in increased equipment costs while not being deployed (in order to avoid operational complexity and associated operational costs). AFAIK, TCM in SONET/SDH is one of the prominent ones. 

Since I've already raised this point once on this list, I've looked up the latest - October-09 - issue of GR-253 for any requirements related to Tandem Connections, and it does not define any use for them, even if it mentions the function. To me this seems to indicate that North American transport operators still do reasonably well without TCM...)

Should we really add more of the same?

My 2c,
     Sasha

> From: [email-masked] [mailto:[email-masked]] 
> Sent: Monday, December 14, 2009 11:29 AM
> To: [email-masked]; 
> [email-masked]; Alexander Vainshtein; 
> [email-masked]
> Cc: [email-masked]