Subject: [Rtg-ooam-dt] O-bit vs. OAM as Next Protocol (Re: [nvo3] Consensus call on encap proposals)

From: Greg Mirsky <[email-masked]>
Date: Sun, 7 Aug 2016 19:02:02 -0700

Dear Authors of the VxLAN-GPE, GUE, and GENEVE,
all protocols under consideration use a bit flag rather than explicit
protocol type to indicate that payload is a test packet, i.e. active OAM.
I'm trying to understand the rationale of such decision. Does use of the
bit flag rather than protocol type produce more efficient implementation,
is more HW friendly? In GUE, the the field to indicate type of the payload
even tagged Proto/ctype as its interpretation depends upon value of the C
bit. But wouldn't it be simpler if all proposals used protocol type to
identify OAM payload? And if the protocol type is OAM, then after the
protocol header have OOAM Header, e.g. as proposed in
draft-ooamdt-rtgwg-ooam-header
<https://tools.ietf.org/html/draft-ooamdt-rtgwg-ooam-header-00>. Then NVO3
protocols would be able to have common Active OAM (Fault Management and
Performance Measurement) that can be used in BIER and SFC. And the bit, the
bit I'd propose to redefine to be used for passive performance measurement
as described in draft-ietf-bier-pmmm-oam
<https://tools.ietf.org/html/draft-ietf-bier-pmmm-oam-00>. (Allocating two
bits-long field would enable more accurate measurements using the Alternate
Marking method).
And these steps will enable us to develop common Active OAM and use passive
performance measurement regardless, almost, of the data plane protocol used
in NVO layer.

Regards, Greg

On Fri, Jul 29, 2016 at 8:13 AM, Alia Atlas <[email-masked]> wrote:

> I'd like to have people focus on the key point of this thread.
>
> Are there serious technical objections (and specifically what are they) to
> moving forward with VXLAN-GPE as the standards-track protocol?
>
> Are there serious technical objections (and specifically what are they) to
> moving forward with GENEVE as the standards-track protocol?
>
> Are there serious technical objections (and specifically what are they) to
> moving forward with GUE as the standards-track protocol?
>
> We need to capture any relevant objections.  So far, there's been some
> discussion on extensibility - with Tom Herbert providing concrete concerns.
>
> I have concluded that almost all the authors would prefer to have no
> standards track solution if they can't guarantee that theirs is that
> standard.
>
> I do hear concerns about whether a decision will be too late.   I think
> that a decision can only be helpful.   It goes back to when is the best
> time to plant a tree - with the answer of 20 years ago or now.
>
> Regards,
> Alia
>
>
> On Fri, Jul 29, 2016 at 4:34 AM, Naoki Matsuhira <[email-masked]
> > wrote:
>
>>
>>
>> On 2016/07/21 23:56, Bocci, Matthew (Nokia - GB) wrote:
>>
>>> WG
>>>
>>> There was a discussion in the NVO3 WG meeting in Berlin following strong
>>> advice from our Area Director that we need to come to a consensus on
>>> converging on a common encapsulation. Two sets of questions were asked:
>>> (1) Should the WG move forward with one standards track encap?
>>> (2) For a given encap, do you have significant technical objections?
>>>
>>
>> I want to inform to this mailing list that I proposed ME6E-FP and ME6E-PR
>> at the yokohama meeting. I also have proposal M46E-FP and M46E-PR (past
>> called SA46T).
>>
>> These encapsulation technologies are based on address mapping. ME6E use
>> IPv6 address which mapping MAC address, and M46E use IPv6 address which
>> mapping IPv4 address.
>>
>> I understand too many encapsulation technologies, however these my
>> proposal are simple, and may contribute to the Internet.
>>
>> I believe address mapping approach is unique, so I want to propose again.
>>
>> sorry not the answer to the question.
>>
>> Naoki Matsuhira
>>
>>
>> _______________________________________________
>> nvo3 mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/nvo3
>>
>
>
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3
>
>

---

From: Shahram Davari <[email-masked]>
Date: Sun, 7 Aug 2016 20:27:26 -0700

Hi 

I agree with Greg. Flag bit is useful if there was no protocol type field. From HW perspective there no advantage of using flag rather than protocol field.

Regards,
Shahram

> On Aug 7, 2016, at 7:02 PM, Greg Mirsky <[email-masked]> wrote:
> 
> all protocols under consideration use a bit flag rather than explicit protocol type to indicate that payload is a test packet, i.e. active OAM. I'm trying to understand the rationale of such decision. Does use of the bit flag rather than protocol type produce more efficient implementation, is more HW friendly? In GUE, the the field to indicate type of the payload even tagged Proto/ctype as its interpretation depends upon value of the C bit. But wouldn't it be simpler if all proposals used protocol type to identify OAM payload? And if the protocol type is OAM, then after the protocol header have OOAM Header, e.g. as proposed in draft-ooamdt-rtgwg-ooam-header. Then NVO3 protocols would be able to have common Active OAM (Fault Management and Performance Measurement) that can be used in BIER and SFC. And the bit, the bit I'd propose to redefine to be used for passive performance measurement as described in draft-ietf-bier-pmmm-oam. (Allocating two bits-long field would enable more accurate measurements using the Alternate Marking method).
> And these steps will enable us to develop common Active OAM and use passive performance measurement regardless, almost, of the data plane protocol used in NVO layer.
> 
> Regards, Greg
> 
>> Are there serious technical objections (and specifically what are they) to moving forward with VXLAN-GPE as the standards-track protocol?
>> 
>> Are there serious technical objections (and specifically what are they) to moving forward with GENEVE as the standards-track protocol?
>> 
>> Are there serious technical objections (and specifically what are they) to moving forward with GUE as the standards-track protocol?
>> 
>> We need to capture any relevant objections.  So far, there's been some discussion on extensibility - with Tom Herbert providing concrete concerns.
>> 
>> I have concluded that almost all the authors would prefer to have no standards track solution if they can't guarantee that theirs is that standard.
>> 
>> I do hear concerns about whether a decision will be too late.   I think that a decision can only be helpful.   It goes back to when is the best time to plant a tree - with the answer of 20 years ago or now.
>> 
>> Regards,
>> Alia
>> 
>> 
>>> On Fri, Jul 29, 2016 at 4:34 AM, Naoki Matsuhira <[email-masked]> wrote:
>>> 
>>> 
>>>> There was a discussion in the NVO3 WG meeting in Berlin following strong advice from our Area Director that we need to come to a consensus on converging on a common encapsulation. Two sets of questions were asked:
>>> I want to inform to this mailing list that I proposed ME6E-FP and ME6E-PR at the yokohama meeting. I also have proposal M46E-FP and M46E-PR (past called SA46T).
>>> 
>>> These encapsulation technologies are based on address mapping. ME6E use IPv6 address which mapping MAC address, and M46E use IPv6 address which mapping IPv4 address.
>>> 
>>> I understand too many encapsulation technologies, however these my proposal are simple, and may contribute to the Internet.
>>>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Tue, 9 Aug 2016 00:31:05 +0000

Greg,

PID IPv6 can be a data packet or an OAM packet. Why the need for an indirection in a subsequent header to have a protocol field, when you already have one?

— Carlos.

On Aug 8, 2016, at 4:02 AM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:

Dear Authors of the VxLAN-GPE, GUE, and GENEVE,
all protocols under consideration use a bit flag rather than explicit protocol type to indicate that payload is a test packet, i.e. active OAM. I'm trying to understand the rationale of such decision. Does use of the bit flag rather than protocol type produce more efficient implementation, is more HW friendly? In GUE, the the field to indicate type of the payload even tagged Proto/ctype as its interpretation depends upon value of the C bit. But wouldn't it be simpler if all proposals used protocol type to identify OAM payload? And if the protocol type is OAM, then after the protocol header have OOAM Header, e.g. as proposed in draft-ooamdt-rtgwg-ooam-header<https://tools.ietf.org/html/draft-ooamdt-rtgwg-ooam-header-00>. Then NVO3 protocols would be able to have common Active OAM (Fault Management and Performance Measurement) that can be used in BIER and SFC. And the bit, the bit I'd propose to redefine to be used for passive performance measurement as described in draft-ietf-bier-pmmm-oam<https://tools.ietf.org/html/draft-ietf-bier-pmmm-oam-00>. (Allocating two bits-long field would enable more accurate measurements using the Alternate Marking method).
And these steps will enable us to develop common Active OAM and use passive performance measurement regardless, almost, of the data plane protocol used in NVO layer.

Regards, Greg

On Fri, Jul 29, 2016 at 8:13 AM, Alia Atlas <[email-masked]<mailto:[email-masked]>> wrote:
I'd like to have people focus on the key point of this thread.

Regards,
Alia

On Fri, Jul 29, 2016 at 4:34 AM, Naoki Matsuhira <[email-masked]<mailto:[email-masked]>> wrote:

Naoki Matsuhira

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

_______________________________________________
Rtg-ooam-dt mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtg-ooam-dt

---

From: Tom Herbert <[email-masked]>
Date: Mon, 8 Aug 2016 17:44:45 -0700

On Sun, Aug 7, 2016 at 7:02 PM, Greg Mirsky <[email-masked]> wrote:
> I'm trying to understand the rationale of such decision. Does use of the bit
> flag rather than protocol type produce more efficient implementation, is
> more HW friendly? In GUE, the the field to indicate type of the payload even
> tagged Proto/ctype as its interpretation depends upon value of the C bit.

The C-bit in GUE distinguishes data messages from control
messages.Data messages are considered to be the payload of
encapsulation, whereas control messages are about the encapsulation
itself. OAM might be one type of control message in GUE, however there
could be others. For instance if we wanted some sort of negotiation
between two endpoints to exchange capabilities or supported features
this would fit well into a control message.

> But wouldn't it be simpler if all proposals used protocol type to identify
> OAM payload? And if the protocol type is OAM, then after the protocol header
> have OOAM Header, e.g. as proposed in . Then

Each of the three protocols has a protocol next header field, however
the field is defined differently among them. The next header in GUE is
an IP protocol number, in Geneve it is an Ethertype, and VXLAN-GPE
uses a new number space. In GUE we could probably use ICMP protocol
for OAM by defining the appropriate types (that might have the
advantage of allow OAM to be generic instead of restricted to only
encapsulation). Presumably, VXLAN-GPE could define some value in the
number space for for OAM. For Geneve maybe there is an appropriate
Ethertype?

> NVO3 protocols would be able to have common Active OAM (Fault Management and
> as described in draft-ietf-bier-pmmm-oam. (Allocating two bits-long field
> would enable more accurate measurements using the Alternate Marking method).

The problem I see with trying to constrain the solution to only one or
two bits of information is that this substantially limits the
functionality. With an extensible protocol we should be able more
information to get more accurate measurement. For instance, I might
want to measure the latency of individual packets to get feedback on
path selection, correlate performance to packet loss, etc. Has the OAM
DT considered the requirements and solutions for passive performance
measurement?

Thanks,
Tom

>> that a decision can only be helpful.   It goes back to when is the best time
>> to plant a tree - with the answer of 20 years ago or now.
>>
>> Regards,
>> Alia
>>
>>

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Aug 2016 01:22:10 +0000

Hi Greg,

One of the major considerations in choosing a bit for identifying OAM in VXLAN-GPE was so the hardware only needed to look at this bit to determine that the packet needs to be kicked to the OAM processing.  The assumption is that OAM would evolve in the future and we didn’t want to bake one protocol ID into the hardware – just check the bit and kick to the software.  Also it seems possible that OAM packets could have the same protocol type as payload packets (e.g. IP), so we still need something else to differentiate data from OAM in that case.

 - Larry

From: nvo3 <[email-masked]<mailto:[email-masked]>> on behalf of Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Sunday, August 7, 2016 at 7:02 PM
To: Alia Atlas <[email-masked]<mailto:[email-masked]>>
Cc: "Bocci, Matthew (Nokia - GB)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, NVO3 <[email-masked]<mailto:[email-masked]>>
Subject: [nvo3] O-bit vs. OAM as Next Protocol (Re: Consensus call on encap proposals)

Regards, Greg

Regards,
Alia

Naoki Matsuhira

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Aug 2016 08:54:06 -0700

Can anyone (Greg in particular, as lead author) explain to me why this
document creates an IANA registry for a value that isn't indicated in
its header, and why the values in the header (e.g., Msg Type, flags)
aren't defined?

I.e., this doesn't seem like it proposes much at all...

Joe

On 8/7/2016 7:02 PM, Greg Mirsky wrote:
> But wouldn't it be simpler if all proposals used protocol type to
> protocol header have OOAM Header, e.g. as proposed
> in draft-ooamdt-rtgwg-ooam-header
> <https://tools.ietf.org/html/draft-ooamdt-rtgwg-ooam-header-00>.

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 09:07:15 -0700

Hi Tom,
many thanks for the most informative response. I've added mu notes in-line
under tag GIM>>.

Regards, Greg

On Mon, Aug 8, 2016 at 5:44 PM, Tom Herbert <[email-masked]> wrote:

> bit
GIM>> Yes, what I've proposed is clearly more than just OAM channel. In
fact, it is Associated Channel (ACh) that may be used by control,
management and OAM. And as I've used term "Associated Channel" you'll
easily recognize that I have MPLS background and draw on MPLS/MPLS-TP OAM
experience. And as Generic ACh (G-ACh) is used to advertise capabilities of
an LSR in RFC 7212, AC-h in NVO3 can support similar functionalities as
well.

> identify
GIM>> Indeed, the OAM DT had considered the requirements to enable use of
performance measurement methods as passive OAM. Should note that we use
term "passive method" somewhat differently from the definition in RFC 7799.
Such interpretation was discussed in the IPPM WG and we've agreed that if a
measurement method does not change treatment of a data packet by the
network (e.g. doesn't change its CoS, length or else), then the method
behaves as close as passive and may be characterized as such. Measurements
for a single packet are possible using the Alternate Marking method with
two bits-long marker. The draft in BIER WG has such example. I've attached
the presentation slides. Will be glad to answer any further questions.

> time

---

From: Tom Herbert <[email-masked]>
Date: Tue, 9 Aug 2016 09:28:58 -0700

On Tue, Aug 9, 2016 at 9:07 AM, Greg Mirsky <[email-masked]> wrote:
> fact, it is Associated Channel (ACh) that may be used by control, management
> and OAM. And as I've used term "Associated Channel" you'll easily recognize
> that I have MPLS background and draw on MPLS/MPLS-TP OAM experience. And as
> Generic ACh (G-ACh) is used to advertise capabilities of an LSR in RFC 7212,
> AC-h in NVO3 can support similar functionalities as well.
>>
>>
> performance measurement methods as passive OAM. Should note that we use term
> "passive method" somewhat differently from the definition in RFC 7799. Such
> interpretation was discussed in the IPPM WG and we've agreed that if a
> measurement method does not change treatment of a data packet by the network
> (e.g. doesn't change its CoS, length or else), then the method behaves as
> close as passive and may be characterized as such. Measurements for a single
> packet are possible using the Alternate Marking method with two bits-long
> marker. The draft in BIER WG has such example. I've attached the
> presentation slides. Will be glad to answer any further questions.

Yes, but number of specific packets I could measure is still limited
in some time quantum with the two bit method. Alternatively, if every
packet contained a timestamp for instance, then we can measure every
packet and filter or aggregate the measurements with arbitrary
granularity of our choosing. BIER may have been defined with as a
fixed length header so that a couple of bits are all that could
feasibly be allocated to OAM, but this is not necessarily true for
other encapsulation protocols that are purposely extensible to support
a richer set of features.

Tom

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 09:56:39 -0700

Hi Tom,
I've read a proposal to apply timestamps to data packets on a fly, I
believe submitted in the SFC WG. I cannot characterize such method as
passive as, per my understanding, it will increase the size of the packet
to accommodate the timestamps. The advantage of the Alternate Marking
method is that once marker applied the measurements of packet loss and
delay/delay variation could be performed at numerous test points along the
path or even within a node without any changes to the packet itself. To
achieve the same with direct timestamping, I think, would require
collection of N+1 64 bits-long timestamps.

Regards, Greg

On Tue, Aug 9, 2016 at 9:28 AM, Tom Herbert <[email-masked]> wrote:

> >> > more HW friendly? In GUE, the the field to indicate type of the
> payload
> >> > even
> management
> >> > NVO3 protocols would be able to have common Active OAM (Fault
> Management
> >> > and
> >> > Performance Measurement) that can be used in BIER and SFC. And the
> bit,
> >> > the
> term
> network
> >> >> Are there serious technical objections (and specifically what are
> they)
> >> >> to
> >> >> Are there serious technical objections (and specifically what are
> they)
> >> >> to
> >> >> Are there serious technical objections (and specifically what are
> they)
> >> >> to

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 10:22:36 -0700

Hi Larry,
thank you for the most detailed explanation of the choice made for
VxLAN-GPE.
As experience with MPLS Generic Association Channel shows, new control,
management and OAM functions could be added under the model of the
associated channel. And it does allow use of different encapsulations,
IPv4/IPv6 or without IP/UDP header, a.k.a. ACH. The latter may be
particularly useful in the overlay networks.

Regards, Greg

On Mon, Aug 8, 2016 at 6:22 PM, Larry Kreeger (kreeger) <[email-masked]>
wrote:

> Hi Greg,
>
> One of the major considerations in choosing a bit for identifying OAM in
> VXLAN-GPE was so the hardware only needed to look at this bit to determine
> that the packet needs to be kicked to the OAM processing.  The assumption
> is that OAM would evolve in the future and we didn’t want to bake one
> protocol ID into the hardware – just check the bit and kick to the
> software.  Also it seems possible that OAM packets could have the same
> protocol type as payload packets (e.g. IP), so we still need something else
> to differentiate data from OAM in that case.
>
>  - Larry
>
> From: nvo3 <[email-masked]> on behalf of Greg Mirsky <
> [email-masked]>
> To: Alia Atlas <[email-masked]>
> Cc: "Bocci, Matthew (Nokia - GB)" <[email-masked]>, "
> [email-masked]" <[email-masked]>, NVO3 <[email-masked]>
> <https://tools.ietf.org/html/draft-ooamdt-rtgwg-ooam-header-00>. Then
> NVO3 protocols would be able to have common Active OAM (Fault Management
> and Performance Measurement) that can be used in BIER and SFC. And the bit,
> the bit I'd propose to redefine to be used for passive performance
> measurement as described in draft-ietf-bier-pmmm-oam
> <https://tools.ietf.org/html/draft-ietf-bier-pmmm-oam-00>. (Allocating
> two bits-long field would enable more accurate measurements using the
> Alternate Marking method).
> And these steps will enable us to develop common Active OAM and use
> passive performance measurement regardless, almost, of the data plane
> protocol used in NVO layer.
>
> Regards, Greg
>
>> Are there serious technical objections (and specifically what are they)
>> to moving forward with VXLAN-GPE as the standards-track protocol?
>>
>> Are there serious technical objections (and specifically what are they)
>> to moving forward with GENEVE as the standards-track protocol?
>>
>> Are there serious technical objections (and specifically what are they)
>> to moving forward with GUE as the standards-track protocol?
>>
>>>> There was a discussion in the NVO3 WG meeting in Berlin following
>>>> strong advice from our Area Director that we need to come to a consensus on
>>> I want to inform to this mailing list that I proposed ME6E-FP and
>>> ME6E-PR at the yokohama meeting. I also have proposal M46E-FP and M46E-PR
>>> (past called SA46T).
>>>

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 10:32:33 -0700

Hi Joe,
indeed, I'll continue working on the drafts. Greatly appreciate technical
comments and welcome contributions by new authors.

Regards,
Greg

On Tue, Aug 9, 2016 at 8:54 AM, Joe Touch <[email-masked]> wrote:

> document creates an IANA registry for a value that isn't indicated in its
> header, and why the values in the header (e.g., Msg Type, flags) aren't
> defined?
>
> OAM payload? And if the protocol type is OAM, then after the protocol
> header have OOAM Header, e.g. as proposed in draft-ooamdt-rtgwg-ooam-
> header <https://tools.ietf.org/html/draft-ooamdt-rtgwg-ooam-header-00>.
>
>
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 9 Aug 2016 10:56:20 -0700

On Tue, Aug 9, 2016 at 9:56 AM, Greg Mirsky <[email-masked]> wrote:
> Hi Tom,
> I've read a proposal to apply timestamps to data packets on a fly, I believe
> submitted in the SFC WG. I cannot characterize such method as passive as,
> per my understanding, it will increase the size of the packet to accommodate
> the timestamps. The advantage of the Alternate Marking method is that once
> marker applied the measurements of packet loss and delay/delay variation
> could be performed at numerous test points along the path or even within a
> node without any changes to the packet itself. To achieve the same with
> direct timestamping, I think, would require collection of N+1 64 bits-long
> timestamps.
>
I was thinking more that only the sender would place a time stamp in
the packet. Any device on the path that sees the timestamp can
determine the one way latency from the sender as the difference of
time packet is received and the timestamp assuming the clocks are
synchronized. The timestamp allows measurements to be taken for each
packet at the cost of overhead of timestamp in packet which could be
as little four bytes. Alternatively, the timestamp could also be used
to measure RTT if it is reflected by the destination end point (like
how TCP timestamp is used)

Tom

>> >> > header
>> >> > passive
>> >> > performance measurement regardless, almost, of the data plane
>> >> > protocol
>> >> > used
>> >> > in NVO layer.
>> >>
>> >> >>>> strong
>> >> >>> ME6E-PR

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 11:09:15 -0700

Hi Tom,
agree, single timestamp may be used as a marker too. But without addition
of a sequence number accuracy of your measurement will be  affected by
packet loss, re-ordering and packet duplication. And I don't think that 32
bits-long timestamp format enables high precision of delay/delay variation
calculation. AFAIK, delay measurement protocols use 64 bits-long formats,
NTP or PTPv1/truncated PTPv2. RTT would work only for bi-directional
applications, though it simplifies operations as it doesn't require clock
synchronization in the network.

Regards, Greg

On Tue, Aug 9, 2016 at 10:56 AM, Tom Herbert <[email-masked]> wrote:

> believe
> >> >> > protocol type to indicate that payload is a test packet, i.e.
> active
> >> >> > OAM.
> >> >> > I'm trying to understand the rationale of such decision. Does use
> of
> >> >> > the
> >> >> > bit
> >> >> > flag rather than protocol type produce more efficient
> implementation,
> >> >> > is
> >> >> > tagged Proto/ctype as its interpretation depends upon value of the
> C
> >> >> > bit.
> >> >>
> >> > that I have MPLS background and draw on MPLS/MPLS-TP OAM experience.
> And
> >> > as
> >> >> >>>> advice from our Area Director that we need to come to a
> consensus
> >> >> >>>> on
> >> >> >>> These encapsulation technologies are based on address mapping.
> ME6E
> >> >> >>> use
> >> >> >>> I believe address mapping approach is unique, so I want to
> propose
> >> >> >>> again.
> >> >> >>>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Tue, 9 Aug 2016 18:56:26 +0000

Dear Greg,

On Aug 9, 2016, at 1:22 PM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:

Hi Larry,
thank you for the most detailed explanation of the choice made for VxLAN-GPE.
As experience with MPLS Generic Association Channel shows

MPLS does not have an encapsulation/tunnel header with fields to be used, a PID field, an OAM bit, etc. MPLS does have a need to prevent ECMP treatment based on its payload being misinterpreted as IP.

That’s why there’s a G-ACh. Plus the Intro in RFC5586 (coming from MPLS-TP requirements). That experience is not extrapolatable here.

, new control, management and OAM functions could be added under the model of the associated channel.

A lot “could" be added, the question is what’s needed to be done.

Thanks,

— Carlos.

And it does allow use of different encapsulations, IPv4/IPv6 or without IP/UDP header, a.k.a. ACH. The latter may be particularly useful in the overlay networks.

Regards, Greg

On Mon, Aug 8, 2016 at 6:22 PM, Larry Kreeger (kreeger) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Greg,

 - Larry

Regards, Greg

Regards,
Alia

Naoki Matsuhira

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Tue, 9 Aug 2016 19:02:45 +0000

> On Aug 9, 2016, at 12:28 PM, Tom Herbert <[email-masked]> wrote:
> 
> granularity of our choosing.

Indeed, as for example at draft-brockners-inband-oam-data-01. This is a measurement that is neither “active” nor “passive” based on the IETF definitions (where active means a probe, and passive means no change whatsoever to a packet).

> BIER may have been defined with as a

Agreed as well.

Thanks,

— Carlos.

> [email-masked]

---

From: Tom Herbert <[email-masked]>
Date: Tue, 9 Aug 2016 12:14:03 -0700

> Indeed, as for example at draft-brockners-inband-oam-data-01. This is a measurement that is neither “active” nor “passive” based on the IETF definitions (where active means a probe, and passive mea…
Carlos,

Thanks for the pointer, that looks like a good basis for an extensible
and generic OAM inband measurement mechanism. I assume for IPv6 this
would be appropriate as options in HBH extension headers, have you
defined the formats for that?

Thanks,
Tom

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Tue, 9 Aug 2016 22:07:44 +0000

Hi, Tom,

On Aug 9, 2016, at 3:14 PM, Tom Herbert <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Aug 9, 2016 at 12:02 PM, Carlos Pignataro (cpignata)
<[email-masked]<mailto:[email-masked]>> wrote:

On Aug 9, 2016, at 12:28 PM, Tom Herbert <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Aug 9, 2016 at 9:07 AM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:
Hi Tom,
many thanks for the most informative response. I've added mu notes in-line
under tag GIM>>.

Regards, Greg

On Mon, Aug 8, 2016 at 5:44 PM, Tom Herbert <[email-masked]<mailto:[email-masked]>> wrote:

On Sun, Aug 7, 2016 at 7:02 PM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:
Dear Authors of the VxLAN-GPE, GUE, and GENEVE,
all protocols under consideration use a bit flag rather than explicit
protocol type to indicate that payload is a test packet, i.e. active
OAM.
I'm trying to understand the rationale of such decision. Does use of the
bit
flag rather than protocol type produce more efficient implementation, is
more HW friendly? In GUE, the the field to indicate type of the payload
even
tagged Proto/ctype as its interpretation depends upon value of the C
bit.

NVO3 protocols would be able to have common Active OAM (Fault Management
and
Performance Measurement) that can be used in BIER and SFC. And the bit,
the
bit I'd propose to redefine to be used for passive performance
measurement
as described in draft-ietf-bier-pmmm-oam. (Allocating two bits-long
field
would enable more accurate measurements using the Alternate Marking
method).
And these steps will enable us to develop common Active OAM and use
passive
performance measurement regardless, almost, of the data plane protocol
used
in NVO layer.

Carlos,

Thanks — and you are exactly right:
https://tools.ietf.org/html/draft-brockners-inband-oam-transport-01#section-3

This was demoed as IPv6 edge to edge in Bits-n-Bites in Berlin.

Thanks,

— Carlos.

Thanks,
Tom

Agreed as well.

Thanks,

— Carlos.

Tom

Thanks,
Tom

Regards, Greg

We need to capture any relevant objections.  So far, there's been some
discussion on extensibility - with Tom Herbert providing concrete
concerns.

Regards,
Alia

On Fri, Jul 29, 2016 at 4:34 AM, Naoki Matsuhira
<[email-masked]<mailto:[email-masked]>> wrote:

WG

converging on a common encapsulation. Two sets of questions were
asked:

I want to inform to this mailing list that I proposed ME6E-FP and
ME6E-PR
at the yokohama meeting. I also have proposal M46E-FP and M46E-PR
(past
called SA46T).

These encapsulation technologies are based on address mapping. ME6E
use
IPv6 address which mapping MAC address, and M46E use IPv6 address
which
mapping IPv4 address.

I believe address mapping approach is unique, so I want to propose
again.

Naoki Matsuhira

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 15:34:13 -0700

Hi Tom,
would add two notes:

   - being in-band OAM is not exclusively property of iOAM. Active OAM can
   be in-band if test protocols are treated as data payload (something that
   all three NVO3 protocols do);
   - ability to test and/or measure for iOAM limited by presence of the
   data traffic. Active OAM, by using dedicated test packets, doesn't have
   such limitation.

My view is that combination of active, passive and in-between (e.g. iOAM)
methods provides operators with the most comprehensive OAM tool box.

Regards, Greg

On Tue, Aug 9, 2016 at 12:14 PM, Tom Herbert <[email-masked]> wrote:

> implementation, is
> And as
> > Indeed, as for example at draft-brockners-inband-oam-data-01. This is a
> measurement that is neither “active” nor “passive” based on the IETF
> definitions (where active means a probe, and passive means no change
> whatsoever to a packet).
> >
> Carlos,
>
> >>>>>> <[email-masked]> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>

---

From: "Deepak Kumar (dekumar)" <[email-masked]>
Date: Tue, 9 Aug 2016 22:55:45 +0000

Hi,

Even in case of Active OAM we need O bit as Overlay splicing or stiching can occur for scalability reason and in that case we need to look inside host payload after overlay de-cap and use it to re-encap to new tunnel and setting the OAM bit and keep the OAM channel deep inside packet.
If we use Next protocol as OAM then there won't be any host payload to handle this scenario.
If we depend on looking in customer payload to figure out it's OAM packet then hashing we will get might not be same as icmp uses l3 hashing, whereas udp/tcp uses l4 hashing and more fields.

Thanks,
Deepak

From: Rtg-ooam-dt <[email-masked]<mailto:[email-masked]>> on behalf of Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, August 9, 2016 at 3:34 PM
To: Tom Herbert <[email-masked]<mailto:[email-masked]>>
Cc: "Bocci, Matthew (Nokia - GB)" <[email-masked]<mailto:[email-masked]>>, "Carlos Pignataro (cpignata)" <[email-masked]<mailto:[email-masked]>>, NVO3 <[email-masked]<mailto:[email-masked]>>, Alia Atlas <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Rtg-ooam-dt] [nvo3] O-bit vs. OAM as Next Protocol (Re: Consensus call on encap proposals)

Hi Tom,
would add two notes:

  *   being in-band OAM is not exclusively property of iOAM. Active OAM can be in-band if test protocols are treated as data payload (something that all three NVO3 protocols do);
  *   ability to test and/or measure for iOAM limited by presence of the data traffic. Active OAM, by using dedicated test packets, doesn't have such limitation.

My view is that combination of active, passive and in-between (e.g. iOAM) methods provides operators with the most comprehensive OAM tool box.

Regards, Greg

On Tue, Aug 9, 2016 at 12:14 PM, Tom Herbert <[email-masked]<mailto:[email-masked]>> wrote:
On Tue, Aug 9, 2016 at 12:02 PM, Carlos Pignataro (cpignata)
Carlos,

Thanks,
Tom

---

From: Tom Herbert <[email-masked]>
Date: Tue, 9 Aug 2016 15:56:25 -0700

> This was demoed as IPv6 edge to edge in Bits-n-Bites in Berlin.
That's great and gives us the most flexibility. We can generically use
the OAM options in the extension headers of any IPv6 packet, or in the
case of GUE encapsulation we can put the options as the GUE payload
since the GUE header carries an IP protocol number as next protocol.

Thanks,
Tom

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 16:06:07 -0700

Hi Deepak,
I think what you're describing is not Active overlay but piggy-back OAM and
iOAM is one of examples of that. Or, perhaps, this is Service OAM
altogether.
And for ECMP, why not to have Entropy field in the overlay encapsulation
instead of "looking in customer payload". That, I believe, would be more
efficient in the fast path.

Regards, Greg

On Tue, Aug 9, 2016 at 3:55 PM, Deepak Kumar (dekumar) <[email-masked]>
wrote:

> Hi,
>
> Even in case of Active OAM we need O bit as Overlay splicing or stiching
> can occur for scalability reason and in that case we need to look inside
> host payload after overlay de-cap and use it to re-encap to new tunnel and
> setting the OAM bit and keep the OAM channel deep inside packet.
> If we depend on looking in customer payload to figure out it's OAM packet
> then hashing we will get might not be same as icmp uses l3 hashing, whereas
> udp/tcp uses l4 hashing and more fields.
>
> Thanks,
> Deepak
>
> From: Rtg-ooam-dt <[email-masked]> on behalf of Greg Mirsky
> <[email-masked]>
> To: Tom Herbert <[email-masked]>
> Cc: "Bocci, Matthew (Nokia - GB)" <[email-masked]>, "Carlos
> Pignataro (cpignata)" <[email-masked]>, NVO3 <[email-masked]>, Alia
> Atlas <[email-masked]>, "[email-masked]" <[email-masked]>
>    - being in-band OAM is not exclusively property of iOAM. Active OAM
>    can be in-band if test protocols are treated as data payload (something
>    that all three NVO3 protocols do);
>> of the
>> > Indeed, as for example at draft-brockners-inband-oam-data-01. This is
>> a measurement that is neither “active” nor “passive” based on the IETF
>> consensus on

---

From: "Deepak Kumar (dekumar)" <[email-masked]>
Date: Tue, 9 Aug 2016 23:19:35 +0000

Hi Greg,

If packet needs to be software forwarded then carry ECMP as flow entropy is fine and we use it to software forward the packet to new Overlay tunnel, but if packet needs to go in hardware and cover the complete ingress and egress pipeline as real data packet, OAM needs to be hidden and bit helps there.
So in this scenario, packet keeps getting forward as Data, and one copy is punted to software for tracing ingress/egress path or hardware can add more meta data as per their capability.

Thanks,
Deepak

From: Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, August 9, 2016 at 4:06 PM
To: dekumar <[email-masked]<mailto:[email-masked]>>
Cc: Tom Herbert <[email-masked]<mailto:[email-masked]>>, "Bocci, Matthew (Nokia - GB)" <[email-masked]<mailto:[email-masked]>>, "Carlos Pignataro (cpignata)" <[email-masked]<mailto:[email-masked]>>, NVO3 <[email-masked]<mailto:[email-masked]>>, Alia Atlas <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Rtg-ooam-dt] [nvo3] O-bit vs. OAM as Next Protocol (Re: Consensus call on encap proposals)

Hi Deepak,
I think what you're describing is not Active overlay but piggy-back OAM and iOAM is one of examples of that. Or, perhaps, this is Service OAM altogether.
And for ECMP, why not to have Entropy field in the overlay encapsulation instead of "looking in customer payload". That, I believe, would be more efficient in the fast path.

Regards, Greg

On Tue, Aug 9, 2016 at 3:55 PM, Deepak Kumar (dekumar) <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Thanks,
Deepak

Hi Tom,
would add two notes:

Regards, Greg

Carlos,

Thanks,
Tom

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 9 Aug 2016 17:22:14 -0700

Hi Deepak,
yours input, experience are very interesting. But doesn't such difference
in behavior indicates that HW is not capable of processing entropy in the
same manner one can achieve in SW-based (should I assume white-box)
solution? If that could be the state of the art, defining behaviors,
protocols that are not limited by today HW doesn't sound to me too wrong.
Re-phrasing "If we agree, they will build it." Not?

Regards, Greg

On Tue, Aug 9, 2016 at 4:19 PM, Deepak Kumar (dekumar) <[email-masked]>
wrote:

> Hi Greg,
>
> If packet needs to be software forwarded then carry ECMP as flow entropy
> is fine and we use it to software forward the packet to new Overlay tunnel,
> but if packet needs to go in hardware and cover the complete ingress and
> egress pipeline as real data packet, OAM needs to be hidden and bit helps
> there.
> So in this scenario, packet keeps getting forward as Data, and one copy is
> punted to software for tracing ingress/egress path or hardware can add more
> meta data as per their capability.
>
> Thanks,
> Deepak
>
> From: Greg Mirsky <[email-masked]>
> To: dekumar <[email-masked]>
> Cc: Tom Herbert <[email-masked]>, "Bocci, Matthew (Nokia - GB)" <
> [email-masked]>, "Carlos Pignataro (cpignata)" <
> [email-masked]>, NVO3 <[email-masked]>, Alia Atlas <[email-masked]>,
> "[email-masked]" <[email-masked]>
>
> I think what you're describing is not Active overlay but piggy-back OAM
> and iOAM is one of examples of that. Or, perhaps, this is Service OAM
> altogether.
>> From: Rtg-ooam-dt <[email-masked]> on behalf of Greg
>> Mirsky <[email-masked]>

---

From: "Deepak Kumar (dekumar)" <[email-masked]>
Date: Wed, 10 Aug 2016 01:07:31 +0000

Hi Greg,

With O bit new encapsulation will Tx them so checking for Oam in receive won't need any hardware changes and very few hardware space and new action of copy and forward to match it, and as its really 100% data it will follow the same hardware pipeline and just on Tx just same oam bit needs to be marked.

Now having flow entropy as tlv or oam header will require lot of logic in hardware path.

Also if you compare this solution it can monitor all flow like iOam with lot less hardware resources and no need to change the packet. Just customer flows needs to be generated by another mean.

All oam data can be get out of band from each device matching is done using another oam channel.

We are already asking for 2 bits for performance what's harm in extra bit for this functionality as it can provide fault management.

We just need from data path draft that they indicate overlay header reserve and flags field shouldn't be used for ecmp hash calculation.

Benefit is just matching criterion becomes easy and hardware doesn't need to program for each flow.

Thanks
Deepak

Sent from my iPhone

On Aug 9, 2016, at 5:22 PM, Greg Mirsky <[email-masked]<mailto:[email-masked]>> wrote:

Hi Deepak,
yours input, experience are very interesting. But doesn't such difference in behavior indicates that HW is not capable of processing entropy in the same manner one can achieve in SW-based (should I assume white-box) solution? If that could be the state of the art, defining behaviors, protocols that are not limited by today HW doesn't sound to me too wrong. Re-phrasing "If we agree, they will build it." Not?

Regards, Greg

On Tue, Aug 9, 2016 at 4:19 PM, Deepak Kumar (dekumar) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Greg,

Thanks,
Deepak

Hi Deepak,

Regards, Greg

Thanks,
Deepak

Hi Tom,
would add two notes:

Regards, Greg

Carlos,

Thanks,
Tom