Subject: [mpls-tp] comments on draft-ietf-mpls-tp-oam-framework-05

From: Curtis Villamizar <[email-masked]>
Date: Wed, 31 Mar 2010 18:24:56 -0400

Eric, etc al,

First of all, thank you to the WG chairs and others for extending the
last calls on many important internet-drafts.

I had hoped to have comments completed by this weekend but was unable
to do so.  I appologize for any inconvenience it may cause by
submitting comments so close to the end of the extended last call.

I am proposing addtions to draft-ietf-mpls-tp-oam-framework-05 mostly
to cover the issues that arrise due to multipath techniques such as
link aggregation or a common use of link bundling.

My comments are in the form of a context diff against the text version
of the latest draft.  All but one change is an addition to the draft.
One is a minor wording correction.

Curtis

--- draft-ietf-mpls-tp-oam-framework-05.txt	Fri Mar  5 20:24:00 2010
+++ draft-ietf-mpls-tp-oam-framework-05.txt.modified	Wed Mar 31 18:14:04 2010
@@ -124,41 +124,50 @@
       3.6. Configuration Considerations............................15
       3.7. P2MP considerations.....................................15
    4. Reference Model..............................................16
       4.1. MPLS-TP Section Monitoring (SME)........................18
       4.2. MPLS-TP LSP End-to-End Monitoring (LME).................19
       4.3. MPLS-TP LSP Path Segment Tunnel Monitoring (LPSTME).....19
       4.4. MPLS-TP PW Monitoring (PME).............................21
       4.5. MPLS-TP MS-PW Path Segment Tunnel Monitoring (PPSTME)...21
+      4.6  Fate sharing considerations for multipath
    5. OAM Functions for proactive monitoring.......................22
       5.1. Continuity Check and Connectivity Verification..........23
          5.1.1. Defects identified by CC-V.........................25
          5.1.2. Consequent action..................................26
          5.1.3. Configuration considerations.......................27
+	 5.1.4  Multipath considerations
       5.2. Remote Defect Indication................................28
          5.2.1. Configuration considerations.......................29
       5.3. Alarm Reporting.........................................29
       5.4. Lock Reporting..........................................30
       5.5. Packet Loss Measurement.................................31
          5.5.1. Configuration considerations.......................32
+	 5.5.2  Sampling skew
+	 5.5.3  Multipath issues
       5.6. Client Failure Indication...............................32
          5.6.1. Configuration considerations.......................32
       5.7. Packet Delay Measurement................................33
          5.7.1. Configuration considerations.......................33
    6. OAM Functions for on-demand monitoring.......................33
       6.1. Connectivity Verification...............................34
          6.1.1. Configuration considerations.......................35
+	 6.1.2  Multipath considerations
       6.2. Packet Loss Measurement.................................35
          6.2.1. Configuration considerations.......................36
+	 6.2.2  Sampling skew
+	 6.2.3  Multipath issues
       6.3. Diagnostic Tests........................................36
          6.3.1. Throughput Estimation..............................36
          6.3.2. Data plane Loopback................................37
+	 6.3.3.  Multipath considerations
       6.4. Route Tracing...........................................37
          6.4.1. Configuration considerations.......................38
+	 6.4.2  Multipath issues

 Busi et al.           Expires September 5, 2010               [Page 3]
 
 Internet-Draft          MPLS-TP OAM Framework               March 2010

       6.5. Packet Delay Measurement...............................38
@@ -1162,16 +1171,54 @@
    Domain 1 (PW13 PPSTME), and 2) a PPSTME monitoring the PWXZ MS-PW
    Segment on Domain Z with (PWXZ PPSTME).

    It is worth noticing that PPSTMEs can coexist with the PME monitoring
    the end-to-end MS-PW and that PPSTME MEPs and PME MEPs can be
    coincident in the same node (e.g. TPE1 node supports both the PW1Z
    PME MEP and the PW13 PPSTME MEP).

+4.6  Fate sharing considerations for multipath
+
+   Multipath techniques are in use today and are expected to continue
+   to be used in future deployments.  These techniques include
+   Ethernet Link Aggregations [IEEE 802.1AX], the use of Link Bundling
+   for MPLS [RFC4201] where the option to spead traffic over component
+   links is supported and enabled, and the use of ECMP.  Other
+   techniques may emerge in the future.  These techniques share in
+   common the characteristic that an LSP may be spread over a set of
+   component links and therefore be reordered but no flow within the
+   LSP is reordered (except when very infrequent and minimally
+   disruptive load rebalancing occurs).
+
+   The common method of identifying flows within an LSP makes use of
+   the entire MPLS label stack.  For LSP carrying IPv4 or IPv6
+   payload, the IP header information may also be used.  For most
+   implementations of multipath the exact path of the traffic taken by
+   a particular flow cannot be replicated by an OAM flow unless the
+   entire MPLS stack is identical and the payload of the LSP is not
+   IPv4 or IPv6.  Duplicating the exact path is possible only for the
+   innermost flow when pseudowires are used.  In other cases it is not
+   possible.  The entire set of component links can be exercised if an
+   additional label is added beneath the PW label for pseudowire or
+   beneath the GAL for LSP.  This additional label is added for the
+   purpose of providing exercising all component links, though the
+   ability to exercise any particular component link with any given
+   OAM packet is not available.
+
+   The use of multipath techniques may be prohibited or permitted in
+   any particular deployment.  If multipath techniques are used, the
+   deplyment can be considered to be only partially MPLS-TP compliant,
+   however this is unlikely to prevent its use.  Reduced OAM LM
+   functionality is documented in sections 5.2.3 and 5.2.3.
+   Considerations which will improve OAM functionality for CC-V OAM
+   are documented in 5.1.4 and 6.1.2.  Considerations which will
+   improve OAM functionality for throughput estimation are documented
+   in 6.3.1.2.
+
 5. OAM Functions for proactive monitoring

    In this document, proactive monitoring refers to OAM operations that
    are either configured to be carried out periodically and continuously
    or preconfigured to act on certain events such as alarm signals.

    Proactive monitoring is frequently "in service" monitoring. The
    control and measurement implications are:
@@ -1465,16 +1512,32 @@

    For statically provisioned transport paths the above information are
    statically configured; for dynamically established transport paths
    the configuration information are signaled via the control plane.

    The operator SHOULD be able to enable/disable some of the consequent
    actions defined in section 5.1.2.

+5.1.4  Multipath considerations
+
+   If an ability is provided to include an additional label on the
+   MPLS stack below the GAL, then the entire set of component links
+   can be exercised by CC-V OAM.  If an uncorrected fault exists on
+   one of the component links, the result will be some loss of CC-V
+   OAM packets.  While detection will be slower than for the case
+   where multipath is not used, the use of an drop rate threshhold on
+   CC-V can detect failure of a component.  The consequent action of
+   the detected failure SHOULD be a signal fail condition declared by
+   the MEP.
+
+   In the absense of an ability to include an additional label on the
+   MPLS stack below the GAL, CC-V reliability cannot be assured for
+   LSP.
+
 5.2. Remote Defect Indication

    The Remote Defect Indication (RDI) function, as required in section
    2.2.9 of [12], is an indicator that is transmitted by a MEP to
    communicate to its peer MEPs that a signal fail condition exists.
    RDI is only used for bidirectional connections and is associated with
    proactive CC-V activation. The RDI indicator is piggy-backed onto the
    CC-V packet.
@@ -1679,16 +1742,37 @@
    In order to support proactive LM, the transmission rate and PHB
    associated with the LM OAM packets originating from a MEP need be
    configured as part of the LM provisioning procedures. LM OAM packets
    should be transmitted with the same PHB class that the LM is intended
    to measure. If that PHB is not an ordered aggregate where the
    ordering constraint is all packets with the PHB being delivered in
    order, LM can produce inconsistent results.

+5.5.2. Sampling skew
+
+   If an implementations makes use of a hardware forwarding path which
+   operates in parallel with an OAM processing path, whether hardware
+   or software based, the packet and byte counts may be skewed if one
+   or more packets can be processed before the OAM processing samples
+   counters.  If OAM is implemented in software this error can be
+   quite large.
+
+5.5.3  Multipath issues
+
+   If multipath is used, in the case where the LSP is larger than the
+   component links at the ingress or egress, there may be no single
+   packet processing engine in which to inject or extract a LM packet
+   as an atomic operation to which accurate packet and byte counts can
+   be associated with the packet.
+
+   In the case where multipath is used but not at the ingress or
+   egress, the reordering of packets within the LSP can cause
+   inaccurate LM.  The inaccuracy may be bounded but will be non-zero.
+
 5.6. Client Failure Indication

    The Client Failure Indication (CSF) function, as required in section
    2.2.10 of [12], is used to help process client defects and propagate
    a client signal defect condition from the process associated with the
    local attachment circuit where the defect was detected (typically the
    source adaptation function for the local client interface) to the
    process associated with the far-end attachment circuit (typically the
@@ -1864,16 +1948,32 @@
    In addition, when the CV packet is used to check connectivity toward
    a target MIP, the number of hops to reach the target MIP should be
    configured.

    The PHB of the on-demand CV packets should be configured as well.
    This permits the verification of correct operation of QoS queuing as
    well as connectivity.

+6.1.2  Multipath considerations
+
+   Multipath techniques will distribute LSP payload over multiple
+   component links but keep a given flow on the same component link
+   except as needed for infrequent load rebalancing.
+
+   If an ability is provided to include an additional label on the
+   MPLS stack below the GAL, then the entire set of component links
+   can be exercised by CV OAM.  If an uncorrected fault exists on one
+   of the component links the result will be some loss of CV OAM
+   packets.  The use of an drop rate threshhold on CV can detect
+   failure of a component but cannot isolate the component.
+
+   In the absense of an ability to include an additional label on the
+   MPLS stack below the GAL, CV reliability cannot be assured for LSP.
+
 6.2. Packet Loss Measurement

    On-demand Packet Loss Measurement (LM) is one of the capabilities
    supported by the MPLS-TP Performance Monitoring function in order to
    facilitate diagnostic of QoS performance for a transport path, as
    required in section 2.2.11 of [12]. As proactive LM, on-demand LM is
    used to exchange counter values for the number of ingress and egress
    packets transmitted and received by the transport path monitored by a
@@ -1900,32 +2000,53 @@
    LM procedures, the transmission rate and PHB associated with the LM
    OAM packets originating from a MEP must be configured as part of the
    on-demand LM provisioning procedures. LM OAM packets should be
    transmitted with the PHB that yields the lowest packet loss
    performance among the PHB Scheduling Classes or Ordered Aggregates
    (see RFC 3260 [15]) in the monitored transport path for the relevant
    network domain(s).

+6.2.2. Sampling skew
+
+   If an implementations makes use of a hardware forwarding path which
+   operates in parallel with an OAM processing path, whether hardware
+   or software based, the packet and byte counts may be skewed if one
+   or more packets can be processed before the OAM processing samples
+   counters.  If OAM is implemented in software this error can be
+   quite large.
+
+6.2.3  Multipath issues
+
+   If multipath is used, in the case where the LSP is larger than the
+   component links at the ingress or egress, there may be no single
+   packet processing engine in which to inject or extract a LM packet
+   as an atomic operation to which accurate packet and byte counts can
+   be associated with the packet.
+
+   In the case where multipath is used but not at the ingress or
+   egress, the reordering of packets within the LSP can cause
+   inaccurate LM.  The inaccuracy may be bounded but will be non-zero.
+
 6.3. Diagnostic Tests

 6.3.1. Throughput Estimation

    Throughput estimation is an on-demand out-of-service function, as
    required in section 2.2.5 of [12], that allows verifying the
    bandwidth/throughput of an MPLS-TP transport path (LSP or PW) before
    it is put in-service.

    Throughput estimation is performed between MEPs and can be performed
    in one-way or two way modes.

    This test is performed by sending OAM test packets at increasing rate
    (up to the theoretical maximum), graphing the percentage of OAM test
    packets received and reporting the rate at which OAM test packets
-   start begin dropped. In general, this rate is dependent on the OAM
+   begin to drop. In general, this rate is dependent on the OAM
    test packet size.

    When configured to perform such tests, a MEP source inserts OAM test
    packets with test information with specified throughput, packet size
    and transmission patterns.

    For one way test, remote MEP sink receives the OAM test packets and
    calculates the packet loss. For two way test, the remote MEP
@@ -1954,16 +2075,41 @@
    generator, which is associated with the MEP. At a receiving MEP,
    provisioning is required for a test signal detector which is
    associated with the MEP.

    A MIP is transparent to the OAM test packets sent for throught
    estimation and therefore does not require any provisioning to support
    MPLS-TP throghtput estimation.

+6.3.1.1.  Limited OAM processing rate
+
+   If an implementation is able to process payload at much higher data
+   rates than OAM packets, then accurate measurement of throughput
+   using OAM packets is not acheivable.  Whether OAM packets can be
+   processed at the same rate as payload is implementation dependent.
+
+6.3.1.2.  Multipath considerations
+
+   If multipath is used, in the case where the LSP is larger than the
+   component links at the ingress or egress, there may be no single
+   packet processing engine from which all traffic is sourced or
+   counted.  A solution is to clear counters, inject traffic from
+   multiple packet processing engines, collect counts at multiple
+   processing engines, stop traffic, and then compare counters to
+   traffic sent.
+
+   If an ability is provided to include an additional label on the
+   MPLS stack below the GAL, then the entire set of component links
+   can be exercised.  If so, the throughput of the LSP is measured.
+
+   In the absense of an ability to include an additional label on the
+   MPLS stack below the GAL, at best the throughput available to a
+   flow is measured.
+
 6.3.2. Data plane Loopback

    Data plane loopback is an out-of-service function, as required in
    section 2.2.5 of [12], that permits traffic originated at the ingress
    of a transport path to be looped back to the point of origin by an
    interface at either an intermediate node or a terminating node.

    If the loopback function is to be performed at an intermediate node
@@ -2002,16 +2148,29 @@
    peer MEPs. In case a defect exist, the route trace function needs to
    be able to detect it and stop automatically returning the incomplete
    list of OAM entities that it was able to trace.

 6.4.1. Configuration considerations

    The configuration of the route trace function must at least support
    the setting of the number of trace attempts before it gives up.
+
+6.4.2  Multipath issues
+
+   If mutipath is used and MIP are assigned only to the entire node,
+   multipath has no impact on the route tracing function, however
+   individual components are not identified.  If MIP are placed at the
+   origination or termination of a multipath component link then the
+   route tracing function will return additional information.
+
+   If an ability is provided to include an additional label on the
+   MPLS stack below the GAL, then the additional information provided
+   when MIP are placed at the origination or termination of a
+   multipath component link may be useful.

 6.5. Packet Delay Measurement

    Packet Delay Measurement (DM) is one of the capabilities supported by
    the MPLS-TP PM function in order to facilitate reporting of QoS
    information for a transport path, as required in section 2.2.12 of
    [12]. Specifically, on-demand DM is used to measure packet delay and
    packet delay variation in the transport path monitored by a pair of

---

From: Greg Mirsky <[email-masked]>
Date: Wed, 31 Mar 2010 17:04:17 -0700

Dear Curtis,
> I think I disagree with characterization of Link Bundling in the first
> paragraph of your proposed section 4.6:
>
> As I understand, the RFC 4201 prevents from setting LSP that is bigger than
> Maximum LSP B/W among component links for given priority. If that is
> correct, then there would not be situation where LSP is spread over a set of
> component links (obviously, if consider a set of just single component link,
> then your statement is always true). And I'm not sure how ECMP is applicable
> to MPLS-TP (not MPLS in general, but MPLS-TP in particular) and can be
> exercised by it.
> I think that the fact that a particular technology or method are outside of
> MPLS-TP does not exclude them from MPLS deployments (as you've pointed and
> stressed). And I think that if we know that then discussion of such issues
> is outside of scope of this particular document.
>
> Regards,
> Greg
>

> snipped the rest, otherwise its to big
>