Subject: Re: [nvo3] Consensus call on encap proposals

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 21 Jul 2016 20:04:12 +0000

Hi Matthew,

See my responses inline below.

Thanks, Larry

On 7/21/16, 7:56 AM, "nvo3 on behalf of Bocci, Matthew (Nokia - GB)"
<[email-masked] on behalf of [email-masked]> wrote:

>WG
> 
>There was a discussion in the NVO3 WG meeting in Berlin following strong
>advice from our Area Director that we need to come to a consensus on
>converging on a common encapsulation. Two sets of questions were asked:
>(1) Should the WG move forward with one standards track encap?
>(2) For a given encap, do you have significant technical objections?
>
>This email relates to the second of these questions. Please refer to the
>separate email titled ³Consensus call on moving forward with single
>encap² for discussion related to point (1).

I am sorry I missed the meeting.  Was the room polled for the option to
move forward with more than one encap?  I am interested in knowing the
response to that question since on the list, that option appeared to have
much more traction.  If the room was not polled for that option or for a
choice between the options discussed on the list, then we have
incomplete/misleading results for how to move forward.

>
>We would recommend that those not familiar with RFC 7282 "On Consensus
>and Humming in the IETF" may wish to read it for a fuller understanding
>of how the IETF handles challenging consensus decisions and why.
>
>We would like to determine the consensus on the following points on the
>list (there is a separate thread concerning point (2)):
>
>1) Does anyone have a significant technical objection to selecting Geneve
>as the single NVO3 Standards track document?  Please be as concrete and
>detailed as possible as to your technical objection.

Yes, I would strongly object to this being the only option.
There are applications where both encap/decap and deep packet inspection
of NVO3 encapsulated packets requires extremely low latency and low
jitter.  The variable nature of the Geneve header makes meeting these
requirements extremely challenging and/or costly (in terms of the number
of gates needed) to implement in hardware.

Additionally, Geneve does not have any backward compatibility mechanisms
for VXLAN support.  Arguably, VXLAN is the most widely implemented and
deployed NVO3 protocol in data centers today, so this is an important
consideration.

>
>2) Does anyone have a significant technical objection to selecting
>VXLAN-GPE as the single NVO3 Standards track document?  Please be as
>concrete and detailed as possible as to your technical objection.
>
>3)Does anyone have a significant technical objection to selecting GUE as
>the single NVO3 Standards track document?  Please be as concrete and
>detailed as possible as to your technical objection.

My major objection to GUE being the only option is the lack of backward
compatibility with VXLAN (see above).  I am also concerned about how much
actual industry implementation GUE has in both software and hardware.

>
>
>Please reply to this email thread on the NVO3 list by 29th July 2016.
>
>Please DO NOT use this thread to argue or debate the importance or
>details of any technical objections that arise.  That can be done in
>other threads. This thread should be used to state your initial
>objection. Any objections raised will be summarized in an additional
>email at the end of this consensus call so that the WG can discuss the
>results in detail.
>
>While the list of technical issues has been collected for each
>encapsulation, the chairs are discussing how to develop an acceptable
>solution.   The goal is to have an answer before IETF 97.  The chairs
>will follow up to the list shortly.
>
>Regards
>
>Matthew and Sam
>
>

---

From: Alia Atlas <[email-masked]>
Date: Thu, 21 Jul 2016 19:12:56 -0400

Hi Larry,

Very briefly in-line.

> incomplete/misleading results for how to move forward.

Yes, of course the question was asked.   There was,  as I recall, almost no
one in favor.

I would recommend watching/listening to the recording to get a feel of the
discussion.   Of course, there should be minutes out,  but not quite as
quickly.

Regards,
Alia

> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Mon, 25 Jul 2016 14:23:45 +0000

Alia,

On Jul 21, 2016, at 7:12 PM, Alia Atlas <[email-masked]<mailto:[email-masked]>> wrote:

Hi Larry,

Very briefly in-line.

On Jul 21, 2016 10:04 PM, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>> wrote:
> <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:
>
> >WG
> >

Yes, of course the question was asked.   There was,  as I recall, almost no one in favor.

Thank you for the summary of the meeting for those of us who weren't there.  Interestingly, we seem to have a different trend on the mailing list: option 1 appears to garner significant support.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 25 Jul 2016 10:38:02 -0400

Hi Paul,

On Mon, Jul 25, 2016 at 10:23 AM, Paul Quinn (paulq) <[email-masked]>
wrote:

> Alia,
>
>
> On Jul 21, 2016, at 7:12 PM, Alia Atlas <[email-masked]> wrote:
>
> Hi Larry,
>
> Very briefly in-line.
>
> Thank you for the summary of the meeting for those of us who weren't
> there.  Interestingly, we seem to have a different trend on the mailing
> list: option 1 appears to garner significant support.
>

I am well aware and curious about why.  It may be that in person, there
were more folks peripherally involved who just want the Standards process
to work.  That doesn't really explain why none of the people expressing
opinions on the mailing list - whom I know were in the room in some cases -
didn't feel comfortable raising their hands or publicly expressing their
opinion.

I am considering what to do about this discrepancy and how to proceed as we
discuss.

Regards,
Alia

---

From: Jesse Gross <[email-masked]>
Date: Mon, 25 Jul 2016 15:52:11 +0000

On Jul 25, 2016, at 7:38 AM, Alia Atlas <[email-masked]<mailto:[email-masked]>> wrote:

Hi Paul,

On Mon, Jul 25, 2016 at 10:23 AM, Paul Quinn (paulq) <[email-masked]<mailto:[email-masked]>> wrote:
Alia,

Hi Larry,

Very briefly in-line.

On Jul 21, 2016 10:04 PM, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>> wrote:
> >separate email titled ^3Consensus call on moving forward with single
> >encap^2 for discussion related to point (1).
>

I am well aware and curious about why.  It may be that in person, there were more folks peripherally involved who just want the Standards process to work.  That doesn't really explain why none of the people expressing opinions on the mailing list - whom I know were in the room in some cases - didn't feel comfortable raising their hands or publicly expressing their opinion.

I was in the room and didn't speak up on this point (though I did express my support of option #1 previously on the mailing list).

The way the question was asked was pretty abstract and therefore hard to argue against. Asking "Who is in favor of a single encap?" is somewhat the equivalent of saying "Who would like to eat ice cream?". In both cases, it's something that essentially everyone likes but probably comes with some tradeoffs. In the case of encapsulations, I think everyone would be fine if their preferred choice was selected. However, in the absence of that happening and without specifically linking what you would have to give up, the question is fairly meaningless.

When it comes to extensibility, I think the design space is pretty well explored at this point. I don't really see that it is likely that a new compromise choice emerges that will significantly change the objections that have been raised to the existing protocols. To me, extensibility is a core need-to-have component. As a result, while it would be nice to have a single encapsulation format, if that format did not include extensibility it would not be useful to me. Without it, it would not be possible to build the software that I/VMware am trying to deliver and therefore the choice to use something else would be an easy one.

I believe that others are in a similar position but opposite with regards to technical choices. The net result is that there are almost certain to be multiple formats in the wild regardless of what is decided here. Yes, that means letting the market decide rather than the IETF. I honestly don't necessarily see that as a negative since it means that it will be based on experience rather than theoretical arguments. I don't even think that it will cause more confusion or set back the industry given that timescales of ~5 years are being talked about for a new compromise encap if that were to come to be.

---

From: Lucy yong <[email-masked]>
Date: Mon, 25 Jul 2016 16:38:37 +0000

I were in NVO3 meeting room and surprised in seeing that option 2 was selected somehow (although mailing list is in favor of option 1). The question asked in the room is for supporting option 2 only and option 1 were not mentioned at all, which was like a done deal. The rest questions were to support which encap. as single NVO3 encap to work on. I don't think that is proper to make a decision after waiting for three years and many people in this community were not in the room. In fact, the responses in the room were very light because it came a shock.

Lucy

From: nvo3 [mailto:[email-masked]] On Behalf Of Jesse Gross
Sent: Monday, July 25, 2016 10:52 AM
To: Alia Atlas
Cc: Matthew Bocci; Larry Kreeger (kreeger); [email-masked]; Paul Quinn (paulq)
Subject: Re: [nvo3] Consensus call on encap proposals

Hi Larry,

Very briefly in-line.

---

From: Joe Touch <[email-masked]>
Date: Mon, 25 Jul 2016 10:06:39 -0700

Hi, all,

I sincerely hope it is never appropriate to act solely on events at an
IETF meeting, but that this is taken only as a first-step to mailing
list coordination. Many people participate online - whether because they
didn't attend the meeting, because they feel more comfortable responding
via text, or because they want more time to consider their response.

My view (having not attended) is that there are some significant issues
in developing protocols that both support efficient implementation
(hardware or software) AND are sufficiently flexible. I am not aware of
IETF consensus in any WG on what that entails, which means that coming
up with a solution to satisfy a set of undocumented requirements isn't
going to happen.

As a consequence, a small set of reasonably distinct encapsulation
designs can and should proceed as either Informational or Experimental
as per the NVO3 charter.

If and when there is consensus on one of these or a new encapsulation in
the future, they can be reviewed for promotion to standards-track by
revising the RFC through (presumably) another WG.

Until then, any standards that might need to refer to encapsulations
should either support negotiation or be encapsulation-agnostic. That
would then allow a standards-track system approach to refer to these
encapsulations non-normatively, avoiding a down-ref problem.

Joe

---

From: Fabio Maino <[email-masked]>
Date: Mon, 25 Jul 2016 10:29:58 -0700

On 7/25/16 7:38 AM, Alia Atlas wrote:
>>     On Jul 21, 2016 10:04 PM, "Larry Kreeger (kreeger)"
>>     <[email-masked] <mailto:[email-masked]>> wrote:
>>     >
>>     > Hi Matthew,
>>     >
>>     > <[email-masked] <mailto:[email-masked]> on behalf
>>     of [email-masked] <mailto:[email-masked]>> wrote:
>>     >
>>     > >WG
>>     > >
>     Thank you for the summary of the meeting for those of us who
>     weren't there.  Interestingly, we seem to have a different trend
>     on the mailing list: option 1 appears to garner significant support.
>
>
> were more folks peripherally involved who just want the Standards 
> process to work.  That doesn't really explain why none of the people 
> expressing opinions on the mailing list - whom I know were in the room 
> in some cases - didn't feel comfortable raising their hands or 
> publicly expressing their opinion.

I couldn't make the meeting (my apologies), otherwise I would have 
certainly stood up for option 1, as I did in the mailing list.

Fabio

>
> I am considering what to do about this discrepancy and how to proceed 
> as we discuss.
>
> Regards,
> Alia
>
>
>

---

From: Fabio Maino <[email-masked]>
Date: Mon, 25 Jul 2016 10:53:44 -0700

On 7/25/16 8:52 AM, Jesse Gross wrote:
> I believe that others are in a similar position but opposite with 
> regards to technical choices. The net result is that there are almost 
> certain to be multiple formats in the wild regardless of what is 
> decided here. Yes, that means letting the market decide rather than 
> the IETF. I honestly don't necessarily see that as a negative since it 
> means that it will be based on experience rather than theoretical 
> arguments. I don't even think that it will cause more confusion or set 
> back the industry given that timescales of ~5 years are being talked 
> about for a new compromise encap if that were to come to be.

Coming from a different perspective (I'm one of the authors of 
VXLAN-GPE) I totally agree with the above.

For us the design goal was to maximize reuse of VXLAN logic, adding 
multiprotocol support, and achieve extensibility via shim layer (as done 
for the SFC use case with VXLAN-GPE + NSH).

Before we set off for a new encapsulation, I would like to make sure 
that there is a well grounded use case that cannot be addressed with the 
existing ones.

Fabio

---

From: Alia Atlas <[email-masked]>
Date: Mon, 25 Jul 2016 14:27:45 -0400

Hi Jesse,

On Mon, Jul 25, 2016 at 11:52 AM, Jesse Gross <[email-masked]> wrote:

>
> On Jul 25, 2016, at 7:38 AM, Alia Atlas <[email-masked]> wrote:
>
> Hi Paul,
>
> I was in the room and didn't speak up on this point (though I did express
> my support of option #1 previously on the mailing list).
>
> The way the question was asked was pretty abstract and therefore hard to
> argue against. Asking "Who is in favor of a single encap?" is somewhat the
> equivalent of saying "Who would like to eat ice cream?". In both cases,
> it's something that essentially everyone likes but probably comes with some
> tradeoffs. In the case of encapsulations, I think everyone would be fine if
> their preferred choice was selected. However, in the absence of that
> happening and without specifically linking what you would have to give up,
> the question is fairly meaningless.
>

The first question is to determine if there's a desire to get to a single
standard encap.  Of course, it'd be easy if everyone wanted the same thing.
  I will note that I've given the working group 18 months to figure out how
to get to 2 or fewer and seen no progress.

The second is to figure out what the technical objections are to the
different encapsulations.  Once they are clear (and almost no one is
commenting with technical substance there), then a couple things can
happen.   For each, either there is a plan and the problem is addressed or
the WG decides that it has heard the technical concern but it does not need
to be resolved.

> When it comes to extensibility, I think the design space is pretty well
> explored at this point. I don't really see that it is likely that a new
> compromise choice emerges that will significantly change the objections
> that have been raised to the existing protocols. To me, extensibility is a
> core need-to-have component. As a result, while it would be nice to have a
> single encapsulation format, if that format did not include extensibility
> it would not be useful to me. Without it, it would not be possible to build
> the software that I/VMware am trying to deliver and therefore the choice to
> use something else would be an easy one.
>

I hear you completely that extensibility is really important.  I do not
know if the design space has been well explored yet; there has certainly
been little public discussion of alternatives.  I have heard ideas around
using lookup tables instead of TLVs for instance - which might have a
benefit in the argument about TLVs.

> I believe that others are in a similar position but opposite with regards
> to technical choices. The net result is that there are almost certain to be
> multiple formats in the wild regardless of what is decided here. Yes, that
> means letting the market decide rather than the IETF. I honestly don't
> necessarily see that as a negative since it means that it will be based on
> experience rather than theoretical arguments. I don't even think that it
> will cause more confusion or set back the industry given that timescales of
> ~5 years are being talked about for a new compromise encap if that were to
> come to be.
>

NVO3 has been around for about 5 years.  So far, solutions have been
brought into the IETF but not really developed.
It is rather rare that a technical solution is rubber-stamped without
changes.   At least we would be heading somewhere.
I'm also fine with taking and tweaking an existing solution so that the
technical objections are met.

Regards,
Alia

---

From: Alia Atlas <[email-masked]>
Date: Mon, 25 Jul 2016 14:28:49 -0400

On Mon, Jul 25, 2016 at 1:53 PM, Fabio Maino <[email-masked]> wrote:

> Coming from a different perspective (I'm one of the authors of VXLAN-GPE)
> I totally agree with the above.
>
> Before we set off for a new encapsulation, I would like to make sure that
> there is a well grounded use case that cannot be addressed with the
> existing ones.

That is why we are looking for the technical objections - to determine what
there is consensus to resolve and what there is consensus to not resolve.
Do please read RFC 7282, if you haven't yet.

Regards,
Alia

---

From: Alia Atlas <[email-masked]>
Date: Mon, 25 Jul 2016 14:36:24 -0400

Hi Lucy,

On Mon, Jul 25, 2016 at 12:38 PM, Lucy yong <[email-masked]> wrote:

> *I were in NVO3 meeting room and surprised in seeing that option 2 was
> selected somehow (although mailing list is in favor of option 1). The
> question asked in the room is for supporting option 2 only and option 1
> were not mentioned at all, which was like a done deal. The rest questions
> were to support which encap. as single NVO3 encap to work on. I don’t think
> that is proper to make a decision after waiting for three years and many
> people in this community were not in the room. In fact, the responses in
> the room were very light because it came a shock.*
>
>
If it came as a shock that we are having this hard conversation in person,
I am sorry.  Clearly, the discussion on the mailing list started in advance
of the meeting and I was looking forward to an animated discussion of the
pros and cons of different options.

I do view Option 1 - publish all three as Informational - as essentially
indicating that the WG can't come to consensus on a single encapsulation.
I haven't even seen any efforts to consolidate down to two.

The questions around consensus were asked in the room and have been
repeated on this list.  The IETF determines consensus on the mailing list
and all decisions are confirmed there.   I am struggling to understand the
difference in response in the room versus the mailing list, but it is quite
clear how and where consensus is determined - and that
calling it is up to the WG Chairs.

What I am hearing loud and clear on the list in this thread is a general
feeling that NVO3 should just give up and document what has been done.  If
that is not the intended message, please clarify.

Regards,
Alia

>
>
> *Lucy*
>
>
>
> *From:* nvo3 [mailto:[email-masked]] *On Behalf Of *Jesse Gross
> *Sent:* Monday, July 25, 2016 10:52 AM
> *To:* Alia Atlas
> *Cc:* Matthew Bocci; Larry Kreeger (kreeger); [email-masked]; Paul Quinn
> (paulq)
> *Subject:* Re: [nvo3] Consensus call on encap proposals
>
>
>
>
>

---

From: Alia Atlas <[email-masked]>
Date: Mon, 25 Jul 2016 14:40:30 -0400

Hi Joe,

> via text, or because they want more time to consider their response.

Absolutely!  IETF consensus decisions are made on the mailing list.
It is highly beneficial to have thoughtful and considered discussions.

> designs can and should proceed as either Informational or Experimental

I think that they would go as Informational.  There is no experiment here.
This would not allow extensions to be easily progressed either.

> revising the RFC through (presumably) another WG.

Indeed.

> encapsulations non-normatively, avoiding a down-ref problem.

Yes, precisely.

Regards,
Alia

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 25 Jul 2016 11:45:39 -0700

Hi, Alia,

One one minor point:

> I think that they would go as Informational.  There is no experiment here.

That depends on how each encaps protocol doc is worded - there could be,
or some WGs do interpret "specs that are more than de-facto but not
ready to standardize" as being themselves an "experiment" in what the
community will adopt.

I don't think this is a very important issue to anyone...

> This would not allow extensions to be easily progressed either.

I'm not sure it matters. Both experimental and informational can be
updated or replaced at will.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 25 Jul 2016 18:57:34 -0700

> I believe that others are in a similar position but opposite with regards to
> technical choices. The net result is that there are almost certain to be
I would like to meet the user who thinks having multiple interoperable
solutions that do pretty make the same thing is a good idea.

---

From: Anton Ivanov <[email-masked]>
Date: Tue, 26 Jul 2016 09:42:08 +0300

+1

On 26 July 2016 04:57:34 EEST, Tom Herbert <[email-masked]> wrote:
>regards to

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: "Michael Smith (michsmit)" <[email-masked]>
Date: Tue, 26 Jul 2016 16:26:59 +0000

Users already have to deal with multiple solutions for network overlays in
the form of VXLAN and NVGRE. Neither of which is listed as an option here.
Picking 1 more solution or 3 more solutions won¹t improve that.

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 09:31:20 -0700

The IETF should not encourage more options. Standards are suppose to unify interoperability not cause more combinations.

And every decade IETF picks 3 options. Just think if the IETF could produce fast?  ;-)

Stop the bleeding,
Dino

> On Jul 26, 2016, at 9:26 AM, Michael Smith (michsmit) <[email-masked]> wrote:
>

---

From: "Michael Smith (michsmit)" <[email-masked]>
Date: Tue, 26 Jul 2016 18:59:56 +0000

Agreed. Given that considerable time has past since the initial decision
and as long as we are re-visiting it, why not adopt VXLAN ? It has seen
considerable deployment and implementation. Its format is compatible with
LISP which serves to provide a common frame format for L2 and L3 overlays.
One issue raised in the meeting was that VXLAN is an independent track
RFC. I may be naïve, but this seems fairly easy to remedy. Worst case,
call it something else, change the UDP port number (I’m not aware of any
hardware implementations that couldn’t handle changing the port number),
or revive the pre-VXLAN L2 LISP draft.

On 7/26/16, 9:31 AM, "Dino Farinacci" <[email-masked]> wrote:

>The IETF should not encourage more options. Standards are suppose to
>unify interoperability not cause more combinations.
>
>>>> necessarily see that as a negative since it means that it will be
>>>>based
>>>> on
>>>> ~5 years are being talked about for a new compromise encap if that
>>>>were
>>>> to
>>>> come to be.
>>>>

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:04:53 -0700

> and as long as we are re-visiting it, why not adopt VXLAN ? It has seen

Well that was my vote but that was isn’t in the list of candidates.

And we know that VXLAN can support both L2 and L3 overlays. Just in the L3 overlay case, we waste 14 bytes of packet overhead. But I know merchant silicon can support this already.

So you bring up a good point Michael.

> LISP which serves to provide a common frame format for L2 and L3 overlays.

I would argue this ship has already sailed and this is indeed defacto standard.

> RFC. I may be naïve, but this seems fairly easy to remedy. Worst case,

Agree.

> hardware implementations that couldn’t handle changing the port number),

Agree.

> or revive the pre-VXLAN L2 LISP draft.

Agree.

Dino

---

From: Alia Atlas <[email-masked]>
Date: Tue, 26 Jul 2016 15:06:18 -0400

Having VXLAN as an Independent Stream RFC gives a document to be used to
innovate from.   I believe that was the intent of VXLAN-GPE - to provide
the ability
for needed extensions.

>From an IETF perspective, it is critical to have an encapsulation that can
provide
reasonable OAM and at least the potential for security when the industry is
ready.

Regards,
Alia

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:08:33 -0700

> innovate from.   I believe that was the intent of VXLAN-GPE - to provide the ability
> for needed extensions.   

The only new feature the VXLAN-GPE brings to the table is a way to identify that NSH is being encapsulated. All other flexibility features can be implemented in the options Michael outlined.

Dino

---

From: Fabio Maino <[email-masked]>
Date: Tue, 26 Jul 2016 12:11:38 -0700

On 7/26/16 12:08 PM, Dino Farinacci wrote:
> The only new feature the VXLAN-GPE brings to the table is a way to identify that NSH is being encapsulated.

or any other shim header, NSH just happens to be one.

Fabio

> All other flexibility features can be implemented in the options Michael outlined.
>
> Dino
>

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:12:48 -0700

> or any other shim header, NSH just happens to be one.

Which will require a hardware change. Since “another” isn’t specific yet, we don’t know if it is a bug or a feature.  ;-)

Dino

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Jul 2016 12:13:17 -0700

> hardware implementations that couldn’t handle changing the port number),

All recent assignments are *required* to support in-band versioning, so
at worst a version number bump would be sufficient.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 26 Jul 2016 12:21:47 -0700

> or revive the pre-VXLAN L2 LISP draft.
Please see the technical objections that have been raised for
VXLAN-GPE on this thread. Most of these will be applicable to VXLAN as
well. If you don't think these objections are valid then we can have
that discussion, but that should be on a separate thread as Matthew
requested.

Tom

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:22:03 -0700

Now, let’s think about this. Why waste 4-bits or a byte for every single packet when the UDP port number can be your version number. That UDP port number has to be in every single packet anyways.

Why keep the port number the same and change the version number when the same cost of product change will occur. To save UDP port numbers? 

What if people wanted to filter v1 versus v2, doing it with a UDP port number is a simpler and already deployed way to differentiate services. Now those middle boxes have to look even deeper into the header?

Lets be practical,
Dino

> On Jul 26, 2016, at 12:13 PM, Joe Touch <[email-masked]> wrote:
> 
> 
>

---

From: Nadeau Thomas <[email-masked]>
Date: Tue, 26 Jul 2016 15:24:33 -0400

Exactly Dino. Its puzzling why this is even being debated.

	—Tom

> On Jul 26, 2016:3:08 PM, at 3:08 PM, Dino Farinacci <[email-masked]> wrote:
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Jul 2016 12:32:56 -0700

Read BCP 165.

The cost to your protocol is as little as 1 bit and it protects a global resource. 

Joe

> On Jul 26, 2016, at 12:22 PM, Dino Farinacci <[email-masked]> wrote:
>

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:41:28 -0700

Sorry, not convinced. What you point me to is general services that run at a session layer. We are talking about using (err, misusing?) UDP for a network layer function.

Dino

> On Jul 26, 2016, at 12:32 PM, Joe Touch <[email-masked]> wrote:
> 
> Read BCP 165.
>

---

From: Alia Atlas <[email-masked]>
Date: Tue, 26 Jul 2016 15:42:36 -0400

I know it's tempting to dive into technology details and micro-optimization,
but can we please stay focused.

This is already a rather long thread - and I am strongly hoping to hear
from those
who aren't championing their own technology.

The question is:

I've heard a lot of concern that we just can't do it, that it's too late,
etc.
I personally feel that neither of these is true.  Working Groups do move
past these
challenging consensus discussions and go on to be successful and have
impact.

We can always flip a coin - if the solutions are technically equivalent and
so is the
support.

I personally strongly believe that it will be useful for the IETF to have a
single standards track
solution.

Regards,
Alia

On Tue, Jul 26, 2016 at 3:32 PM, Joe Touch <[email-masked]> wrote:

> > Now, let’s think about this. Why waste 4-bits or a byte for every single
> packet when the UDP port number can be your version number. That UDP port
> number has to be in every single packet anyways.
> >
> > Why keep the port number the same and change the version number when the
> same cost of product change will occur. To save UDP port numbers?
> >
> > What if people wanted to filter v1 versus v2, doing it with a UDP port
> number is a simpler and already deployed way to differentiate services. Now
> those middle boxes have to look even deeper into the header?
> >
> > Lets be practical,
> > Dino
> >

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Jul 2016 12:46:07 -0700

Some other points:

> What if people wanted to filter v1 versus v2, doing it with a UDP port number is a simpler and already deployed way to differentiate services. Now those middle boxes have to look even deeper into t…

Alternately, changing the port number then disables your protocol on all
default-off configurations until it's opened up, which means V1 works
but V2 doesn't. That can be very difficult to debug and track.

In a nutshell, one of the reasons we're on a good track with the port
number space is our conservative preservation. That includes requiring
new assignments to support in-band versioning.

The issue is "tragedy of the commons" - the cost of doing this "just for
you" is low, but the cost to the community if everyone has permission
*has already been deemed* too high.

Finally, while I appreciate some of these considerations, they appear to
be your opinion of the key aspects of IETF protocol design. They are not
consistent with IETF BCPs, nor are they agreed as the best way to
develop protocols that are useful *not only now* but in the future as well.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Jul 2016 12:49:22 -0700

> Sorry, not convinced. What you point me to is general services that run at a session layer. We are talking about using (err, misusing?) UDP for a network layer function.

That BCP applies to *all* services assigned transport protocol port
numbers and names, including a fairly large number that already have
assigned values.

It's your choice to use that layer for network layer functions.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Jul 2016 12:50:53 -0700

On 7/26/2016 12:42 PM, Alia Atlas wrote:
> ...
> I've heard a lot of concern that we just can't do it, that it's too
> late, etc.

I don't think it's ever "too late", but I also don't see consensus
forming either.
...
> We can always flip a coin - if the solutions are technically
> equivalent and so is the
> support.

That defines "random", not "consensus".

Joe

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 12:57:07 -0700

When the current architecture is broken, you have to use what works. Just one example is NAT-traversal. UDP is the only choice. And default-off is a feature.

Dino

P.S. Ending thread per Alia’s request to stay on topic of a decision.

> On Jul 26, 2016, at 12:46 PM, Joe Touch <[email-masked]> wrote:
> 
> Some other points:
>

---

From: Alia Atlas <[email-masked]>
Date: Tue, 26 Jul 2016 16:04:56 -0400

On Tue, Jul 26, 2016 at 3:50 PM, Joe Touch <[email-masked]> wrote:

> I don't think it's ever "too late", but I also don't see consensus forming
> either.
>

It may be that many folks are recovering from IETF and not responding yet.

> ...
>
> We can always flip a coin - if the solutions are technically equivalent
> and so is the
> support.
>
>

It's not a preferred way - but it's a way of breaking a deadlock.
Of course, the WG would have to have consensus on that being the decision
process.
Take a read through RFC 7282.  I keep praising it because it gives really
good perspectives.

---

From: Lucy yong <[email-masked]>
Date: Tue, 26 Jul 2016 21:25:56 +0000

I am in favor to publish three as informational because of existing implementations.

If the WG get consensus to work on one standards track encap., We should work out one IETF quality protocol that can last for a while and do not intend to match an existing implementation. Thus, I prefer use of GUE as the reference base for WG to work on. GUE has the best semantics among three: simple, extensible, complimentary for hardware and software implementation, and already specifies security, fragmentation, checksum-offload features.

Objection to VXLAN-GPE: VXLAN-GPE format is wired and use of VXLAN-GPE+NSH to provide extensibility adds quite complex and burden. For example, NSH is designed for SFC, it has its own ver, OAM bit indication and SFC header. It is not clear that when VXLAN is used to carry SFC traffic, will Mandatory context header be used to carry SFC related meta data or for VXLAN related features. I don’t think it is good idea to bundle NVO3 encap. and SFC encaps. Together although SFC may exist in a NVO3 instance.

Objection to Geneve: Geneve design is in favor of software implementation. Without specify TLVs, not sure what hardware can implement. Of course, we may have many pure software applications without interworking with a hardware component, which can benefit from the TLV design. GUE has a private field (optional)  that can be used for this purpose in future. Thus we can avoid to specify two protocols in this WG.

Regards,
Lucy

From: nvo3 [mailto:[email-masked]] On Behalf Of Alia Atlas
Sent: Tuesday, July 26, 2016 3:05 PM
To: Joe Touch
Cc: Tom Herbert; Michael Smith (michsmit); [email-masked]; Matthew Bocci; Paul Quinn; Dino Farinacci; Jesse Gross; Larry Kreeger
Subject: Re: [nvo3] Consensus call on encap proposals

On Tue, Jul 26, 2016 at 3:50 PM, Joe Touch <[email-masked]<mailto:[email-masked]>> wrote:

The question is:

I've heard a lot of concern that we just can't do it, that it's too late, etc.

I don't think it's ever "too late", but I also don't see consensus forming either.

...
We can always flip a coin - if the solutions are technically equivalent and so is the
support.

It's not a preferred way - but it's a way of breaking a deadlock.
Of course, the WG would have to have consensus on that being the decision process.
Take a read through RFC 7282.  I keep praising it because it gives really good perspectives.

Joe

---

From: "Reith, Lothar" <[email-masked]>
Date: Tue, 26 Jul 2016 21:45:06 +0000

Hi Alia,

I fully support your approach. I am strongly in favor  of moving forward with one standards track encap.

May be it would be an approach to try and agree on a handful of criteria, such as extensibility, backwards compatibility with VXLAN and/or NVGRE, number of running code, ease of hardware implementation (evidenced by number of running hardware code), interoperability with VXLAN/NVGRE, Security, Cost in terms of protocol overhead, Cost in terms of scarce numbering resources, interoperability with legacy deployments in the field, support by deployed control planes, and yes: Simplicity (the opposite of complexity).

Did I miss any important category?

One could agree on the categories, agree on a process to rank the 3 candidate protocols – e.g. identify the best and the worst by the number and severity of objections in that category, the third is in the middle.

The final rank would be average of the ranks per category. That one should be promoted to standards track.

One could consider flipping a coin to decide the rank in a category – but only in a contentious situation between 2 of the 3 candidates.

Lothar

Von: nvo3 [mailto:[email-masked]] Im Auftrag von Alia Atlas
Gesendet: Dienstag, 26. Juli 2016 22:05
An: Joe Touch <[email-masked]>
Cc: Tom Herbert <[email-masked]>; Michael Smith (michsmit) <[email-masked]>; [email-masked]; Matthew Bocci <[email-masked]>; Paul Quinn <[email-masked]>; Dino Farinacci <[email-masked]>; Jesse Gross <[email-masked]>; Larry Kreeger <[email-masked]>
Betreff: Re: [nvo3] Consensus call on encap proposals

The question is:

...

Joe

---

From: Alia Atlas <[email-masked]>
Date: Tue, 26 Jul 2016 17:50:56 -0400

Hi Lothar,

On Tue, Jul 26, 2016 at 5:45 PM, Reith, Lothar <[email-masked]>
wrote:

> May be it would be an approach to try and agree on a handful of criteria,
> such as extensibility, backwards compatibility with VXLAN and/or NVGRE,
> number of running code, ease of hardware implementation (evidenced by
> number of running hardware code), interoperability with VXLAN/NVGRE,
> Security, Cost in terms of protocol overhead, Cost in terms of scarce
> numbering resources, interoperability with legacy deployments in the field,
> support by deployed control planes, and yes: Simplicity (the opposite of
> complexity).
>

Thanks for your feedback and joining in the conversation.

I've been trying to ask from the perspective of technical objects for a
reason that a particular approach CAN NOT solve the problem.
This ties back to how and why the IETF does consensus (see RFC 7282).

> One could agree on the categories, agree on a process to rank the 3
> candidate protocols – e.g. identify the best and the worst by the number
> and severity of objections in that category, the third is in the middle.
>

I suspect that a design team approach will be needed to work rapidly and
try for a fair evaluation of the problems with each one.

> be promoted to standards track.
I'm a bit twitchy about using averages - (average of 6 foot tall and 2 foot
tall is 4 foot tall - but not helpful).

I'm happy to see discussion about how this decision can be made.

> One could consider flipping a coin to decide the rank in a category – but
> only in a contentious situation between 2 of the 3 candidates.
>

I would strongly prefer to avoid flipping a coin :-)

Regards,
Alia

> Lothar
>
>
>
>
>
>
>
> *Von:* nvo3 [mailto:[email-masked]] *Im Auftrag von *Alia Atlas
> *Gesendet:* Dienstag, 26. Juli 2016 22:05
> *An:* Joe Touch <[email-masked]>
> *Cc:* Tom Herbert <[email-masked]>; Michael Smith (michsmit) <
> [email-masked]>; [email-masked]; Matthew Bocci <[email-masked]>;
> Paul Quinn <[email-masked]>; Dino Farinacci <[email-masked]>; Jesse
> Gross <[email-masked]>; Larry Kreeger <[email-masked]>
> *Betreff:* Re: [nvo3] Consensus call on encap proposals
>
>
>

---

From: Lucy yong <[email-masked]>
Date: Tue, 26 Jul 2016 21:52:53 +0000

All three can’t interwork with VXLAN nor NVGRE. To interwork with them, an end point has to support multiple encaps and sync with peer what to use by configuration or control plane.

Lucy

From: nvo3 [mailto:[email-masked]] On Behalf Of Reith, Lothar
Sent: Tuesday, July 26, 2016 4:45 PM
To: Alia Atlas; Joe Touch
Cc: Tom Herbert; Michael Smith (michsmit); [email-masked]; Matthew Bocci; Paul Quinn; Dino Farinacci; Jesse Gross; Larry Kreeger
Subject: Re: [nvo3] Consensus call on encap proposals

Hi Alia,

Lothar

Von: nvo3 [mailto:[email-masked]] Im Auftrag von Alia Atlas
Gesendet: Dienstag, 26. Juli 2016 22:05
An: Joe Touch <[email-masked]<mailto:[email-masked]>>
Cc: Tom Herbert <[email-masked]<mailto:[email-masked]>>; Michael Smith (michsmit) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Matthew Bocci <[email-masked]<mailto:[email-masked]>>; Paul Quinn <[email-masked]<mailto:[email-masked]>>; Dino Farinacci <[email-masked]<mailto:[email-masked]>>; Jesse Gross <[email-masked]<mailto:[email-masked]>>; Larry Kreeger <[email-masked]<mailto:[email-masked]>>
Betreff: Re: [nvo3] Consensus call on encap proposals

The question is:

...

Joe

---

From: Anton Ivanov <[email-masked]>
Date: Tue, 26 Jul 2016 23:05:02 +0100

On 26/07/16 22:52, Lucy yong wrote:
>
> /All three can’t interwork with VXLAN nor NVGRE. To interwork with 
> them, an end point has to support multiple encaps and sync with peer 
> what to use by configuration or control plane./
>

I would not call that interworking.

For me (and I have written more than one paravirtual vNIC driver) means 
that a driver for encaps A can parse and produce frames in encaps B 
solely by means of configuration.

The interworking between these three from a technical point of view is 
EXACTLY ZERO.

A.

> //
>
> //
>
> /Lucy/
>
> //
>
> *From:*nvo3 [mailto:[email-masked]] *On Behalf Of *Reith, Lothar
> *Sent:* Tuesday, July 26, 2016 4:45 PM
> *To:* Alia Atlas; Joe Touch
> *Cc:* Tom Herbert; Michael Smith (michsmit); [email-masked]; Matthew 
> Bocci; Paul Quinn; Dino Farinacci; Jesse Gross; Larry Kreeger
> May be it would be an approach to try and agree on a handful of 
> criteria, such as extensibility, backwards compatibility with VXLAN 
> and/or NVGRE, number of running code, ease of hardware implementation 
> (evidenced by number of running hardware code), interoperability with 
> VXLAN/NVGRE, Security, Cost in terms of protocol overhead, Cost in 
> terms of scarce numbering resources, interoperability with legacy 
> deployments in the field, support by deployed control planes, and yes: 
> Simplicity (the opposite of complexity).
>
> candidate protocols – e.g. identify the best and the worst by the 
> number and severity of objections in that category, the third is in 
> the middle.
>
> One could consider flipping a coin to decide the rank in a category – 
> but only in a contentious situation between 2 of the 3 candidates.
>
> Lothar
>
> *Von:*nvo3 [mailto:[email-masked]] *Im Auftrag von *Alia Atlas
> *An:* Joe Touch <[email-masked] <mailto:[email-masked]>>
> *Cc:* Tom Herbert <[email-masked] <mailto:[email-masked]>>; 
> Michael Smith (michsmit) <[email-masked] 
> <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; 
> Matthew Bocci <[email-masked] 
> <mailto:[email-masked]>>; Paul Quinn <[email-masked] 
> <mailto:[email-masked]>>; Dino Farinacci <[email-masked] 
> <mailto:[email-masked]>>; Jesse Gross <[email-masked] 
> <mailto:[email-masked]>>; Larry Kreeger <[email-masked] 
> <mailto:[email-masked]>>

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 26 Jul 2016 15:09:26 -0700

And you kill multicast applications as well. 

The point of multicast is to save replication bandwidth at the source and bandwidth in the network by sending a single encapsulated packet using an outer multicast destination address. But now you have to undo all of that because some NVEs do VXLAN, another NVGRE, etc.

Dino

> On Jul 26, 2016, at 3:05 PM, Anton Ivanov <[email-masked]> wrote:
> 
> For me (and I have written more than one paravirtual vNIC driver) means that a driver for encaps A can parse and produce frames in encaps B solely by means of configuration.
> 
> The interworking between these three from a technical point of view is EXACTLY ZERO. 
> 
> A.
> 
>>  
>> Lucy
>>

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 27 Jul 2016 03:33:38 +0000

I fully agree with Dino's points. Before picking one of the three WG docs as a candidate to publish, we'd better evaluate whether those already published, implemented and deployed NVo3 technologies (see below) are good enough from the TECHNICAL point of view:

1. VXLAN (RFC7348 Informational)
2. NVGRE (RFC7637 Informational)
3. MPLS-in-GRE (RFC4023 Standards Track)
4. MPLS-in-UDP (RFC7510 Standards Track)

My observation of the above four options is as follows: 

VXLAN: it was originally designed for Layer2 overlay only. That's the reason why it's called Virtual Extended LAN. However, due to the real demand for distributed gateways, Layer3 overlay is becoming more and more important. Although VXLAN can be used for Layer3 overlay by inserting fake MAC addresses, it seems a bit complex. That's the reason why VXLAN-GPE was proposed.

NVGRE: it can be used for both Layer2 and Layer3 overlays. However, its ECMP capability is not good enough due to the following two reasons:1) the GRE key field based load-balancing is not widely supported; 2) the length of the entropy field (one-byte) is not enough. That may be one of the major technical reasons why VXLAN is more popular than NVGRE as an MPLS-free NVo3 encapsulation.

MPLS-in-GRE: it can be used for both Layer2 and Layer3 overlays. More importantly, it can be used for interconnecting DC VPN and WAN VPN (i.e., MPLS-based VPN) in a much scalable and seamless way (i.e., by using inter-AS option B). This is a significant advantage of those MPLS-based NVo3 approaches (e.g., MPLS-in-GRE and MPLS-in-UDP) over those MPLS-free NVo3 approaches (e.g., VXLAN and NVGRE) in the hybrid cloud scenario. That's the reason why many DC network vendors recommend using MPLS-in-GRE rather than VXLAN/NVGRE for the traffic moving across DC gateways in their DC solution whitepapers. However, its ECMP capability is almost the same as NVGRE (i.e., the GRE-key based LB is not widely supported). It seems a bit pity that the NVo3 WG didn't pay enough attention to the hybrid cloud scenario where it's much desirable to interconnect the DC VPN and WAN VPN in a much scalable and seamless way.

MPLS-in-UDP: Similar to MPLS-in-GRE, it can be used for both Layer2 and Layer3 overlays, and can be used for interconnecting DC VPN and WAN VPN in a scalable and seamless way (i.e., by using inter-AS option B). In addition, the use of the UDP tunnel could leverage the UDP five-tuple based load-balancing capability very well. 

Best regards,
Xiaohu

> -----Original Message-----
> From: nvo3 [mailto:[email-masked]] On Behalf Of Dino Farinacci
> Sent: Wednesday, July 27, 2016 12:31 AM
> To: Michael Smith (michsmit)
> Cc: Tom Herbert; [email-masked]; Alia Atlas; Matthew Bocci; Paul Quinn; Jesse
> Gross; Larry Kreeger
> The IETF should not encourage more options. Standards are suppose to unify
> interoperability not cause more combinations.
> 
> > Users already have to deal with multiple solutions for network
> > overlays in the form of VXLAN and NVGRE. Neither of which is listed as an
> option here.
> >>> regards to technical choices. The net result is that there are
> >>> almost certain to be multiple formats in the wild regardless of what
> >>> is decided here. Yes, that means letting the market decide rather
> >>> than the IETF. I honestly don't necessarily see that as a negative
> >>> since it means that it will be based on experience rather than
> >>> theoretical arguments. I don't even think that it will cause more
> >>> confusion or set back the industry given that timescales of
> >>> were to come to be.
> >> I would like to meet the user who thinks having multiple
> >> interoperable solutions that do pretty make the same thing is a good idea.
> >>

---

From: "Vina Ermagan (vermagan)" <[email-masked]>
Date: Wed, 27 Jul 2016 10:48:42 +0000

I am in favor of publishing the three encaps as informational.

As Tom explained, questions asked in the room were perhaps misinterpreted at the time, which is clear from the list now with many people who were present in the room clearly favoring publishing the three options.

Nevertheless, if one option has to be picked, I strongly hope coins don't play a role in it. :)

Best,
Vina

From: nvo3 <[email-masked]<mailto:[email-masked]>> on behalf of Alia Atlas <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, July 26, 2016 at 12:42 PM
To: Joe Touch <[email-masked]<mailto:[email-masked]>>
Cc: Tom Herbert <[email-masked]<mailto:[email-masked]>>, "Michael Smith (michsmit)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Matthew Bocci <[email-masked]<mailto:[email-masked]>>, "Paul Quinn (paulq)" <[email-masked]<mailto:[email-masked]>>, Dino Farinacci <[email-masked]<mailto:[email-masked]>>, Jesse Gross <[email-masked]<mailto:[email-masked]>>, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] Consensus call on encap proposals

This is already a rather long thread - and I am strongly hoping to hear from those
who aren't championing their own technology.

The question is:

I've heard a lot of concern that we just can't do it, that it's too late, etc.
I personally feel that neither of these is true.  Working Groups do move past these
challenging consensus discussions and go on to be successful and have impact.

I personally strongly believe that it will be useful for the IETF to have a single standards track
solution.

Regards,
Alia

On Tue, Jul 26, 2016 at 3:32 PM, Joe Touch <[email-masked]<mailto:[email-masked]>> wrote:
Read BCP 165.

Joe

> On Jul 26, 2016, at 12:22 PM, Dino Farinacci <[email-masked]<mailto:[email-masked]>> wrote:
>
> Now, let's think about this. Why waste 4-bits or a byte for every single packet when the UDP port number can be your version number. That UDP port number has to be in every single packet anyways.
>
>> On Jul 26, 2016, at 12:13 PM, Joe Touch <[email-masked]<mailto:[email-masked]>> wrote:
>>
>>
>>
>>> call it something else, change the UDP port number (I'm not aware of any
>>> hardware implementations that couldn't handle changing the port number),
>>

---

From: Tom Herbert <[email-masked]>
Date: Wed, 27 Jul 2016 08:25:20 -0700

> I fully agree with Dino's points. Before picking one of the three WG docs as a candidate to publish, we'd better evaluate whether those already published, implemented and deployed NVo3 technologies…
Please read Matthew's initial posting on this thread. The question
raised is "what are the specific TECHNICAL objections to GUE,
VXLAN-GPE, and Geneve". Raising the technical objections as the method
to evaluate these proposal is what the AD has asked for and is
described in RFC7282. Suggesting VXLAN or eight other protocols on
this thread really is not helpful. So do you have any specific
technical objections regarding the three proposed protocols in the WG?

Tom

---

From: Sam Aldrin <[email-masked]>
Date: Wed, 27 Jul 2016 09:28:55 -0700

<WG chair hat on>
As asked in the original email, please do provide technical objections to
the solutions within the NVo3 WG.

This will enable to form consensus in order to resolve the issues and come
up with a solution(s) adopted by WG @IETF.
As mentioned by Alia, consensus and decision process will be based on
RFC7282. If you haven't read it, please do.
</WG chair hat on>

<WG chair hat off>
Just having 3 in NVo3 doesn't mean we need to have 3 encap types adopted by
WG.
Informational shouldn't preclude that the drafts will be adopted as is,
without addressing issues/requirements/technical objections.
As as user, I'd like to see a protocol which is a standard adopted by the
community, not because it won the beauty contest.
If WG cannot determine on one, as I said at the mic, would rather prefer
existing protocols which are standard/defacto standard protocols.
</WG chair hat off>

-sam

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Wed, 27 Jul 2016 17:31:46 +0000

> On Jul 26, 2016, at 3:11 PM, Fabio Maino (fmaino) <[email-masked]> wrote:
> 

Precisely.  GPE addresses the limitation associated with VXLAN, namely the lack of multi-protocol support.  As we've seen on the list, other protocols have come up: MPLS for example.  

There are some other technical benefits as well: version and OAM.

GPE also aims for be both hardware "friendly" and provide a modicum of backward compatibility with VXLAN implementations.

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 27 Jul 2016 10:44:26 -0700

> Precisely.  GPE addresses the limitation associated with VXLAN, namely the lack of multi-protocol support.  As we've seen on the list, other protocols have come up: MPLS for example.  

This is technically not true. As long as you are willing to spend 14 bytes on a MAC header, then anything with an ethertype can be encapsulated with VXLAN.

And I have working code that encapsulates both IPv4 and IPv6 in VXLAN with ethertypes 0x0800 and 0x86dd, respectively.

> There are some other technical benefits as well: version and OAM.

As does LISP.

Agree.

Dino

---

From: Sam Aldrin <[email-masked]>
Date: Wed, 27 Jul 2016 10:46:54 -0700

Resending with some edits (post coffee).

<WG chair hat on>

<WG chair hat off>
Just having 3 in NVo3 doesn't mean we need to have 3 encap types adopted by
WG.
Informational shouldn't preclude that the drafts will be adopted as is,
without addressing issues/requirements/technical objections.
As as user, I'd like to see a protocol which is a standard and adopted by
the community.
This is not because it has won the beauty contest, but meets the
requirements of NVo3.
If WG cannot zero in on one specific protocol, as I said at the mic, would
rather prefer to live with existing protocols like VXLAN, GRE, LISP etc.
which are standard/defacto standard protocols.
</WG chair hat off>

---

From: Tom Herbert <[email-masked]>
Date: Wed, 27 Jul 2016 10:53:18 -0700

>> There are some other technical benefits as well: version and OAM.
As do Geneve and GUE. But this thread is not about touting the
features of these protocols, it is about the technical objections to
them. My major objections (from the list I posted) to VXLAN-GPE is
that it has no extensibility and offers no security of the VNI. These
are showstoppers in my deployment.

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 27 Jul 2016 10:58:49 -0700

> are showstoppers in my deployment.

Tom, there are enable bits in GPE just like GUE. Doesn’t that allow for extensibility. I agree with you that the security stuff may be hard to extend into GPE (because GUE has more available bits then GPE does at the moment).

Dino

---

From: Fabio Maino <[email-masked]>
Date: Wed, 27 Jul 2016 11:00:13 -0700

> are showstoppers in my deployment.

Tom,

extensibility is achieved via shim header:

- if you want to carry end to end metadata you can define the 
appropriate shim header

- If you want to protect the integrity of the VNI, you can include an 
HMAC of the VNI in the shim header

IMO NSH is proof that the functionality of VXLAN-GPE can be extended 
pretty far.

Fabio

---

From: Tom Herbert <[email-masked]>
Date: Wed, 27 Jul 2016 12:27:19 -0700

On Wed, Jul 27, 2016 at 11:00 AM, Fabio Maino <[email-masked]> wrote:
>>>> Precisely.  GPE addresses the limitation associated with VXLAN, namely
>>>> the lack of multi-protocol support.  As we've seen on the list, other
>>>> protocols have come up: MPLS for example.
>>>
>>> This is technically not true. As long as you are willing to spend 14
>>> bytes on a MAC header, then anything with an ethertype can be encapsulated
>>> with VXLAN.
>>>
>>> And I have working code that encapsulates both IPv4 and IPv6 in VXLAN
>>> with ethertypes 0x0800 and 0x86dd, respectively.
>>>
> - if you want to carry end to end metadata you can define the appropriate
> shim header
>
> - If you want to protect the integrity of the VNI, you can include an HMAC
> of the VNI in the shim header
>
Please point me to the specification where this shim header with the
HMAC is described and add a reference to it in the security
considerations section of the VXLAN-GPE draft.

Thanks,
Tom

---

From: Fabio Maino <[email-masked]>
Date: Wed, 27 Jul 2016 13:15:25 -0700

> considerations section of the VXLAN-GPE draft.

Hi Tom,
there are a lot of use cases that can be addressed with shim headers, 
and each solution may look fairly different depending on the requirements.

Can you point me to an nvo3 specification of what are the requirements 
that need to be addressed to provide VNI integrity protection? If the 
requirements are the same you used for GUE, the shim header could use an 
encoding similar to the GUE security option.

Lacking the requirement document, I'm sure we can add a paragraph in the 
security section that describes how it can be done.

I'll be happy to work on whatever can move the WG out of this impasse.

Fabio