Subject: [nvo3] Second Working Group Last call for draft-ietf-nvo3-encap

From: "Bocci, Matthew (Nokia - GB)" <[email-masked]>
Date: Thu, 1 Jul 2021 15:09:19 +0000

This email begins a two-week working group last call for draft-ietf-nvo3-encap-06.

Please review the draft and post any comments to the NVO3 working group list. If you have read the latest version of the draft but have no comments and believe it is ready for publication as an informational RFC, please also indicate so to the WG email list.

We are also polling for knowledge of any undisclosed IPR that applies to this document, to ensure that IPR has been disclosed in compliance with IETF IPR rules (see RFCs 3979, 4879, 3669 and 5378 for more details).
If you are listed as an Author or a Contributor of this document, please respond to this email and indicate whether or not you are aware of any relevant undisclosed IPR. The Document won't progress without answers from all the Authors and Contributors.

Currently there are no IPR disclosures against this document.

If you are not listed as an Author or a Contributor, then please explicitly respond only if you are aware of any IPR that has not yet been disclosed in conformance with IETF rules.

As a reminder, we are pursuing publication of this document in order to permanently document the experience of one working group in choosing between multiple proposed standards track encapsulation drafts. The idea was that this would provide helpful guidance to others in the community going forward.

This poll will run until Thursday 15th July 2021.

Regards

Matthew and Sam

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 1 Jul 2021 19:26:30 -0400

I support publication of this draft as an editor.

I am unaware of any IPR in this draft not disclosed to the IETF.

Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 2386 Panoramic Circle, Apopka, FL 32703 USA
 [email-masked]

---

From: <[email-masked]>
Date: Sat, 3 Jul 2021 11:51:31 +0800 (CST)

Hi Matthew and Sam,

I've read the latest version of this draft.
I think this document is valuable and ready for publication.

Best Regards,
Xiao Min

------------------原始邮件------------------
发件人：Bocci,Matthew(Nokia-GB)
收件人：NVO3;
抄送人：[email-masked];
日 期 ：2021年07月01日 23:09
主 题 ：[nvo3] Second Working Group Last call for draft-ietf-nvo3-encap
_______________________________________________
nvo3 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 3 Jul 2021 11:25:32 -0400

Dear Authors

I have read the latest draft and support publication.

This is a very important document for operators to help provide guidance on
NVO3 directional encapsulation and what will be the eventual end all be all
encapsulation which from the draft it sounds like GENEVE is out in front.

Few comments.

The draft mentions GUE but does not expand what GUE stands for.

As RFC 7348 VXLAN is the most widely of all deployments worldwide I am
wondering why it is not listed in the NVO3 encapsulation types.

Also RFC 7637 NVGRE as well has implementations by vendors and deployments
by operators I wonder why that was excluded in the list of NVO3
encapsulations examined.

The goal of NVO3 charter is to have a single encapsulation type and GENEVE
based on section 7 is the starting point for a proposed standard for NVO3
encapsulation.

So let’s say NVO3 ends up being the chosen end all be all standard
encapsulation type, what would vendors as well as operators do with
existing VXLAN and NVGRE deployments and would those deployments eventually
require software or I am guessing hardware upgrades to support GENEVE.

It seems that out of the three encapsulation types in the draft, GENEVE was
chosen as it builds on existing VXLAN framework with some added
extensibility so not too difficult for operators migration in theory from
VXLAN to GENEVE.

Kind Regards

Gyan

On Fri, Jul 2, 2021 at 11:51 PM <[email-masked]> wrote:

> Please review the draft and post any comments to the NVO3 working group
> list. If you have read the latest version of the draft but have no comments
> and believe it is ready for publication  as an informational RFC, please
> also indicate so to the WG email list.
> We are also polling for knowledge of any undisclosed IPR that applies to
> this document, to ensure that IPR has been disclosed in compliance with
> IETF IPR rules (see RFCs 3979, 4879, 3669  and 5378 for more details).
> If you are listed as an Author or a Contributor of this document, please
> respond to this email and indicate whether or not you are aware of any
> relevant undisclosed IPR. The Document  won't progress without answers from
> all the Authors and Contributors.
> If you are not listed as an Author or a Contributor, then please
> explicitly respond only if you are aware of any IPR that has not yet been
> disclosed in conformance with IETF rules.
> As a reminder, we are pursuing publication of this document in order to
> permanently document the experience of one working group in choosing
> between multiple proposed standards track  encapsulation drafts. The idea
> was that this would provide helpful guidance to others in the community
> going forward.
> Matthew and Sam_______________________________________________
> nvo3 mailing list
> [email-masked]
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: Donald Eastlake <[email-masked]>
Date: Sat, 3 Jul 2021 14:43:53 -0400

Hi Gyan,

On Sat, Jul 3, 2021 at 11:25 AM Gyan Mishra <[email-masked]> wrote:
> This is a very important document for operators to help provide guidance on NVO3 directional encapsulation and what will be the eventual end all be all encapsulation which from the draft it sounds like GENEVE is out in front.
>
> Few comments.
>

Sure, we can add the expansion of GUE.

The questions below seem like they would be better answered by a
member of the Design Team of which I am not.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 2386 Panoramic Circle, Apopka, FL 32703 USA
 [email-masked]

> As RFC 7348 VXLAN is the most widely of all deployments worldwide I am wondering why it is not listed in the NVO3 encapsulation types.
>
> Also RFC 7637 NVGRE as well has implementations by vendors and deployments by operators I wonder why that was excluded in the list of NVO3 encapsulations examined.
>
> The goal of NVO3 charter is to have a single encapsulation type and GENEVE based on section 7 is the starting point for a proposed standard for NVO3 encapsulation.
>
> So let’s say NVO3 ends up being the chosen end all be all standard encapsulation type, what would vendors as well as operators do with existing VXLAN and NVGRE deployments and would those deployments eventually require software or I am guessing hardware upgrades to support GENEVE.
>
> It seems that out of the three encapsulation types in the draft, GENEVE was chosen as it builds on existing VXLAN framework with some added extensibility so not too difficult for operators migration in theory from VXLAN to GENEVE.
>
> Kind Regards
>
> Gyan
>
>
> Gyan Mishra
> Network Solutions Architect
>
> Email [email-masked]
>
> M 301 502-1347
>
>

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 3 Jul 2021 15:44:46 -0400

Thank you!

Agreed, questions for DT.

Gyan

On Sat, Jul 3, 2021 at 2:44 PM Donald Eastlake <[email-masked]> wrote:

> > This is a very important document for operators to help provide guidance
> on NVO3 directional encapsulation and what will be the eventual end all be
> all encapsulation which from the draft it sounds like GENEVE is out in
> front.
> >
> > Few comments.
> >
> > Also RFC 7637 NVGRE as well has implementations by vendors and
> deployments by operators I wonder why that was excluded in the list of NVO3
> encapsulations examined.
> >
> > The goal of NVO3 charter is to have a single encapsulation type and
> GENEVE based on section 7 is the starting point for a proposed standard for
> NVO3 encapsulation.
> >
> > It seems that out of the three encapsulation types in the draft, GENEVE
> was chosen as it builds on existing VXLAN framework with some added
> >> We are also polling for knowledge of any undisclosed IPR that applies
> to this document, to ensure that IPR has been disclosed in compliance with
> >> If you are listed as an Author or a Contributor of this document,
> please respond to this email and indicate whether or not you are aware of
> any relevant undisclosed IPR. The Document  won't progress without answers
> from all the Authors and Contributors.
-- 

*Gyan Mishra*

*M 301 502-1347*

---

From: Liyizhou <[email-masked]>
Date: Wed, 7 Jul 2021 01:29:29 +0000

I support its publication.

This document was supposed to be published before the standard track protocol to provide the guidance. I think it is still valuable to publish it now to give the readers the reasoning behind the protocol being chosen.

I found (native) VxLAN was first mentioned in section 6.2.2 without the reference. Authors may either add a reference or change it to “NVO3 encapsulated”.

Rgds,
Yizhou

From: nvo3 [mailto:[email-masked]] On Behalf Of Bocci, Matthew (Nokia - GB)
Sent: Thursday, July 1, 2021 11:09 PM
To: NVO3 <[email-masked]>
Cc: [email-masked]
Subject: [nvo3] Second Working Group Last call for draft-ietf-nvo3-encap

Regards

Matthew and Sam

---

From: Donald Eastlake <[email-masked]>
Date: Tue, 6 Jul 2021 22:56:00 -0400

Hi,

> I support its publication.

Thanks.

> I found (native) VxLAN was first mentioned in section 6.2.2 without the reference. Authors may either add a reference or change it to “NVO3 encapsulated”.

Adding the reference for VxLAN, which is used several times, seems
like a good idea. Checking the draft, I notice there is also one use
of NVGRE so it seems like that reference should be added also

Thanks,
Donald

Thanks,
Donald

---

From: "Ganga, Ilango S" <[email-masked]>
Date: Wed, 14 Jul 2021 01:08:06 +0000

I support the publication of this document as an informational RFC.

As a design team member and contributor, I am not aware of any undisclosed IPR relevant to this document.

Thanks,
Ilango

From: nvo3 <[email-masked]> On Behalf Of Bocci, Matthew (Nokia - GB)
Sent: Thursday, July 1, 2021 8:09 AM
To: NVO3 <[email-masked]>
Cc: [email-masked]
Subject: [nvo3] Second Working Group Last call for draft-ietf-nvo3-encap

Regards

Matthew and Sam

---

From: Rajeev Manur <[email-masked]>
Date: Tue, 13 Jul 2021 18:38:33 -0700

I support this as a contributor and am not aware of any IPR.

Thanks!
--Rajeev

On Tue, Jul 13, 2021 at 6:08 PM Ganga, Ilango S <[email-masked]>
wrote:

> *From:* nvo3 <[email-masked]> *On Behalf Of *Bocci, Matthew (Nokia
> - GB)
> *Sent:* Thursday, July 1, 2021 8:09 AM
> *To:* NVO3 <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* [nvo3] Second Working Group Last call for draft-ietf-nvo3-encap
>
>
>

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 27 Jul 2021 22:16:08 +0200

On 7/1/21 17:09, Bocci, Matthew (Nokia - GB) wrote:
> list. If you have read the latest version of the draft but have no 
> comments and believe it is ready for publication as an informational 
> RFC, please also indicate so to the WG email list.
> 
> relevant undisclosed IPR. The Document won't progress without answers 

Matthew and Sam,

I'm only aware of this request (as of a few minutes ago) since Donald 
forwarded it to me. If you really wanted answers from all the 
contributors it might have been useful to put their email addresses on 
the to or cc lines.

In any case, I am not personally aware of any IPR in this document.

FWIW I also support publishing this to document the experience.

Regards,
    Erik

> explicitly respond only if you are aware of any IPR that has not yet 
> been disclosed in conformance with IETF rules.
>

---

From: Sam Aldrin <[email-masked]>
Date: Tue, 27 Jul 2021 13:41:31 -0700

Thanks!
-sam

---

From: Jeff Tantsura <[email-masked]>
Date: Tue, 27 Jul 2021 14:11:15 -0700

Yes/support

> On Jul 1, 2021, at 8:09 AM, Bocci, Matthew (Nokia - GB) <[email-masked]> wrote:
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/nvo3 <https://www.ietf.org/mailman/listinfo/nvo3>

---

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 27 Jul 2021 17:06:50 -0700

I support publication of the document.  I have the following comments --
mostly editorial and around use of consistent terminology.

==

Section 2

- Expand first use of DT.

Section 6.2.1

- "IP-address, or MAC-address" ->

  IP addresses, and MAC addresses.

- It would be good to include a reference to INT and/or IOAM, so

  it's clear what is being discussed.

Section 6.2.2

- "in some NVO3 extension" ->

   in an NVO3 extension

- "This is nice" ->

  This is desirable.

- "we don't need a separate UDP" ->

  we don't need a separate UDP port

Section 6.2.3

- (section header)

  Group Base Policy ->

  Group Based Policy

Section 6.3

- "NICs doing TCP offload" ->

  NICs implementing various TCP offload mechanisms

Section 6.4

- "unnecessarily constrained" ->

  unnecessarily constrain

- design team -> DT

- "total extension header length selected" ->

  total extension header length specified

- "Single Extension size" ->

  The size of an extension header"

- "The maximum length of a single option" ->

  The maximum length of a single extension header

Section 6.5

(Several of the subsections use extension, extension header,

option, and TLV interchangeably.  I have tried to use extension

header in this section, but other sections also have similar

issues.  Would recommend editor search doc for "extension" and

"option" and "TLV" and make sure usage is correct/consistent.

In some cases it makes sense to use TLV, but there is almost no

case where "option" or "extension" makes sense over "extension

header".)

- (section header)

  Extension Ordering ->

  Ordering of Extension Headers

- "one or a few extensions TLVs"

  a subset of the extension headers

- "specific TLV" ->

  specific extension header

- "order of TLVs" ->

  order of extension headers

- "Transit devices doesn't" ->

  Transit devices don't

- "process the options" ->

  process the extension headers

- "they need to process only a small subset of options" ->

  they may need to process only subset of extension headers

Section 6.6

- "bit-field approach" ->

  bit fields approach

- "and support in the control plane such that" ->

  and support via the control plane a method such that

- "they need more effort" ->

  some other method must be used.

- "In a Bit fields" ->

  In a bit fields

- "does a firewall" ->

  implements a firewall

- "that allows different software" ->

  that allow different software modules

- "to handle different options" ->

  to process different options

Section 6.7

(Same issue with extension v extension headers.

Also look for receiver v target v target NVE.

Should we be using "egress NVE" per the framework?)

- "EVPN and others"

  provide reference.

- "they only care" ->

  it only cares

- "only care about particular extensions"

  only support certain extensions

- "requested extensions" ->

  supported extensions

- "cares about a few extensions" ->

  "supports only certain extensions"

- "with minimal hardware requirements" ->

  with simplified hardware requirements

  for the target NVE.

- "Note that in this approach" ->

  Note that with this approach

- "enumerate the supported NVO3 extensions and their order" ->

  indicated the supported NVO3 extensions and their order, for

  each of encapsulations supported.

Section 6.8

- Add reference to RFC 8394.

- "Ether type" -> EtherType

Section 7

- Add references to VXLAN and NVGRE docs.

(Also doc uses VxLAN and VXLAN and VxLAN-GPE and VXLAN-GPE.

Would recommend stick to VXLAN and VXLAN-GPE.  Check throughout.)

- "lack of header length" ->

  lack of a header length field

- Another full doc check should be done for

  bit-fields vs bit fields vs Bit-field vs Bit field vs Bit Field.

  Recommend using "bit fields" everywhere.

- "By using Geneve option" ->

  By using Geneve options

- Security extension TLVs ->

  security extension TLVs

- "There are implemented Geneve options today in production" ->

  There are already implementations of Geneve options deployed in

  production networks as of this writing.

- Bullet 8 -- add reference to INT spec.

- "We recommend the following enhancements to Geneve to make it more

   suitable to hardware and yet provide the flexibility for software:"

  Indent and add quotations around following paragraph(s).

- "The Geneve draft could specify" ->

  The Geneve draft should specify

Appendix

- Check terminology usage "tunnel endpoint" vs NVE/egress NVE.

---

From: "Boutros, Sami" <[email-masked]>
Date: Wed, 28 Jul 2021 21:40:22 +0000

I support this as an author and am not aware of any IPR.

Thanks,

Sami
On Tue, Jul 27, 2021 at 1:17 PM Erik Nordmark <[email-masked]<mailto:[email-masked]>> wrote:

Matthew and Sam,

Regards,
    Erik

> [email-masked]<mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/nvo3 [ietf.org]<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/nvo3__;!!OSsGDw!ftdhAlpPQsVy93YHF_ikVcXhz0SXFsUjmQtcXlcxoPYG_O_xNu2u3VQ0ol8K0A$>
>

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3 [ietf.org]<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/nvo3__;!!OSsGDw!ftdhAlpPQsVy93YHF_ikVcXhz0SXFsUjmQtcXlcxoPYG_O_xNu2u3VQ0ol8K0A$>

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 29 Jul 2021 00:16:04 -0400

Hi Anoop,

Thank you for your very detailed review. :-\

See my responses below where I didn't exactly agree with your
suggestion. If I don't respond to a point it means I think your
comment is a reasonable change.

Thanks,
Donald

On Tue, Jul 27, 2021 at 8:07 PM Anoop Ghanwani <[email-masked]>
wrote:
>
> I support publication of the document.  I have the following
> comments -- mostly editorial and around use of consistent
> terminology.
>
> ==
>
> Section 2

Instead replace "design team", which occurs earlier in Section 2, with
"design team (DT)".

> case where "option" or "extension" makes sense over "extension

I did some searching and replacing in the document but I probably
didn't change as many instances as you would have.

> Should we be using "egress NVE" per the framework?)

I changed occurrences of "target" to "target NVE".

>   provide reference.

Added reference to RFC 7432.

>   only support certain extensions

The above two changes overlap resulting in
"it only supports certain extensions"

> - Bullet 8 -- add reference to INT spec.

I'm not sure how to find such a reference.

>   Indent and add quotations around following paragraph(s).

Indenting OK but I think quotes are overkill for a relatively large
amount of text like this.

> On Thu, Jul 1, 2021 at 8:09 AM Bocci, Matthew (Nokia - GB) <[email-masked]> wrote:
>>

---

From: Anoop Ghanwani <[email-masked]>
Date: Thu, 29 Jul 2021 00:49:09 -0700

Hi Donald,

For an EVPN reference, RFC 8365 which deals with EVPN VXLAN may be better
than RFC 7432.

The current version of the INT spec is available at p4.org
https://p4.org/p4-spec/docs/INT_v2_1.pdf
Section 5.7 talks about how to carry INT headers of VXLAN GPE and Geneve.

The other responses look good to me.

Thanks,
Anoop

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 29 Jul 2021 10:40:48 -0400

Hi Anoop,

Thanks for the additional suggestions. I'll update the draft accordingly.

Donald

On Thu, Jul 29, 2021 at 3:49 AM Anoop Ghanwani <[email-masked]>
wrote:

>> >> Please review the draft and post any comments to the NVO3 working
>> group list. If you have read the latest version of the draft but have no
>> comments and believe it is ready for publication as an informational RFC,
>> please also indicate so to the WG email list.
>> >>
>> >>
>> >>
>> >> As a reminder, we are pursuing publication of this document in order
>> to permanently document the experience of one working group in choosing