Subject: Re: [Rtg-ooam-dt] comments on draft-ooamdt-rtgwg-ooam-requirement-00

From: Gregory Mirsky <[email-masked]>
Date: Fri, 17 Jun 2016 21:12:50 +0000

Hi Alia,
thank you for your tough and thus thought provoking questions. I think we'll have great discussion before the meeting that will prepare us for the presentation in Berlin.
Please find my answers, comments and sometimes random thoughts in-line, tagged GIM>>, and highlighted in green.

                Kind regards,
                                Greg
Hi,

I am doing a read through of the work of the DT so far.
I have a few comments on draft-ooamdt-rtgwg-ooam-requirement-00.

I have cc'd a couple people who may have perspective and opinions on issues
around OAM and running an overlay network.   For instance, one of the
impediments to TRILL has been solid OAM  and associated implementations.
I'm sure that there are others who could also chime in.

First, a great deal of it reads like generic OAM requirements for any layer.
What I would like to see as concepts:

With overlay networks, it should be understood that there is NOT a static
assumption of layers.  For instance, an NSH header might be inside VXLAN-GPE
which might be inside a BIER header which is inside MPLS or IP.   Ethernet
can play as well.
GIM>> I think that the overlay OAM operates on the particular sub-layer of the overlay. In your example we have three sub-layers:

*         SFC

*         VXLAN-GPE (NVO3?)

*         BIER
with MPLS or IP transport (the latter is probably over Ethernet or something else).
And there are distinct domains on each sub-layer where corresponding OAM can operate. This OAM may help to discover, check on-demand mapping between the sub-layers and or transport (in the latter case it may be possible to do only from BIER OAM - need to discuss).
SFC domain may be over different underlays (may it?) or over combination of underlay domains and IP/MPLS transport. Discovery of all these relationships and mappings should be enabled by the Overlay OAM but it can be see only entities, Test/Maintenance Points of underlay/transport that are visible in the corresponding sub-layer. For example, SFC OAM will see only SFs, not all VXLAN-GPE nodes but only those that are underlay of the particular SF. And, in turn, from the VXLAN-GPE OAM may not see all BFRs.

One of the large challenges is understanding and seeing the topology of
overlays and dependencies along the path.  For instance, consider existing
MPLS ping, where the
set of labels on the packet and their meaning at the responding LSR, can be
returned.
GIM>> As MPLS Ping can discover topology of the given object, so and would Overlay OAM be contained within its particular domain.
Consider RFC 5837 - where a traceroute can return details about the
incoming interface, information about the Ethernet member of the LAG, and
so on.
GIM>> And MPLS Ping has DDMAP TLV. We may need to use more generic terminology in the
   REQ#13: Overlay OAM MAY support verification of the mapping between
            its data plane state and client layer services.

So - there should be a concrete way to drill down BETWEEN layers.  It might
be optional, but I should be able to do the equivalent of a traceroute and
get down to the specific layer-2 link that is being used.  Maybe I can get
a report of the different next-hops available if the TTL equiv hadn't timed
out?

I WANT you to think OUTSIDE the standard existing OAM box!  What are the
headaches of operating an overlay network?  What are the pain points that
make these things hard to deploy?
GIM>> All feedback we have - mapping between the layers.

What about policy considerations where one layer of the network is owned by
a provider?

What about MTU discovery and how that relies on things like traceroute
concepts and getting back a packet too large?  Could an OAM packet return
information about the max header added along the path?
GIM>> There's PMTU Discovery draft in BIER but only because this is multicast and we've used BIER property to optimize the discovery. In other cases existing mechanisms [RFC1191], [RFC1981], and [RFC4821] seem to suffice. Not?

Have you brainstormed with operators who are running these overlays - or
similar proprietary ones?  What are the issues?
GIM>> There are drafts that attempt to tweak existing OAM protocols to work in overlay, e.g. draft-spv-ippm-monitor-implementation-services-kpi in IPPM. draft-pang-nvo3-vxlan-path-detection, draft-browne-sfc-nsh-timestamp, and draft-nordmark-nvo3-transcending-traceroute, in my opinion, point to other use cases for Overlay OAM.

Now - to more specific details:

REQ#1:  The listed requirements MUST be supported with any type of
            transport layer over which the overlay network can be
            realized

What if one overlay has an OAM packet inside another overlay header?  Is
the outer header the only one examined?  Is it ok to do this?
GIM>>Let me turn into another question Could both overlays indicate OAM payload at the same time? I think that is too ambiguous situation and should not be allowed. Thus my answer - only inner overlay would process the OAM packet.

What restrictions might practically exist in terms of underlay to cause
this to be an issue? I.e. what issues to be solve arise from this
requirement?

What do you mean by "transport layer"?  It might be more helpful to define
terminology in terms of overlay/underlay or service/transport or...
GIM>> Thank you, should work on that.

  "REQ#2:  It MUST be possible to initialize Overlay OAM session from
            any node in the overlay network.

   REQ#3:  It SHOULD be possible to initialize an Overlay OAM session
            from a centralized controller."

First - this is assuming a session - what aspects do you have tied up with
this?
GIM>> Single act of measurement/monitoring, e.g. continuity check with BFD or performance measurement.

Second - does this handle problems like the standard traceroute & ECMP with
hashing -
where a midpoint node can do traceroute but it takes a different path
because the source
address to respond to is hashed?
GIM>> If the overlay is unidirectional then using out-band for reverse direction by OAM should not be a problem (the same as for MPLS Ping today). If the service bi-directional and co-routedness, including over multipath domain, is required, then, I assume, there are already mechanism(s) that pin flows to the same co-routed path, e.g. recent conversation-sensitive frame collection and distribution to LACP, over the multipath domain.

Is it the case that the address to respond to is in the packet?  How is it
integrity-protected?
GIM>> This is HUGE!

Frankly, most of the requirements in 4.1 and 4.2 read like a standard OAM -
within a particular layer.  Sure - we need these basics, but there's
nothing novel here.  Why not just refer to common OAM requirements from
some of the MPLS work or elsewhere?
GIM>> I couldn't find complete list of OAM Requirements when we've worked on SPRING, BIER OAM.

Focus on what is needed for OVERLAY/UNDERLAY interactions.

Regards,
Alia