Subject: [nvo3-dt-encap] Discussion summary - 1/10/2017

From: Pankaj Garg <[email-masked]>
Date: Wed, 11 Jan 2017 07:25:53 -0800

Hi All,

Thanks for attending the call. My curated notes are below.

Participants:
    Erik Nordmark (Arista)
    Michael Smith (Cisco)
    Ignas Bagdonas (Equinix)
    Ilango Ganga (Intel)
    Sami Boutros (VMware)
    Tal Mizrahi (Marvell)
    David Mozes (Mellanox)
    Pankaj Garg (Microsoft)

1. We discussed the text for requiring at minimum 64-bytes to be
   supported in Geneve. There is consensus that we should propose a text
   to Geneve authors on minimum option length requirement. This would
   ensure there is common minimum support from all compliant Geneve
   implementations.

2. We discussed the idea of TLV0 further and upon discussion it was
   realized that it adds more confusion without any significant gain.
   Hence design team agreed to drop this from the proposed changes.

3. We agreed that we should put some text either in Geneve draft or
   design team report around supporting option ordering in Geneve
   implemnentations. The idea is that if a hardware can process options
   in TCAM but it can only process the first TLV, then control plane in
   a deployment can use this capability by ensuring a specific TLV is
   always the first one in the packet. We need someone to write this
   text.

4. We discussed split-NVE case and agreed that we need to ask WG whether
   in split-NVE case, options needs to be carried in other packet
   formats such as 802.1q and if they do, then would carrying Geneve
   frame in 802.1q be needed? Based on WG recommendation, an ETYPE
   and/or IP protocol number can be allocated for Geneve as well. Erik
   has agreed to write text around this question. Thanks Erik.

5. Sami has volunteered to write the initial draft for design team
   report. This report would contain our rationale for proposing a
   specific encapsulation and few specific extension use cases and how
   propose encapsulation meet those requirements. Thanks Sami.

6. Pankaj would send one extension use case around diagnostics. We would
   check with Tom on extension around security. Those will probably be
   two use cases we would cover in our design report.

Please let me know if I missed anything.

Thanks,
Pankaj (on behalf of NVO3 Encap Design Team)

---

From: Rajeev Manur <[email-masked]>
Date: Wed, 11 Jan 2017 10:59:56 -0800

Hi Pankaj,

Thanks for the notes. I participated in yesterday's discussion as well. Looks like you missed my name on the participants list. 

Thanks!
--Rajeev

Sent from my iPhone

> On Jan 11, 2017, at 7:25 AM, Pankaj Garg <[email-masked]> wrote:
> 
> Hi All,
> 
> _______________________________________________
> nvo3-dt-encap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3-dt-encap

---

From: Pankaj Garg <[email-masked]>
Date: Thu, 12 Jan 2017 00:12:59 -0800

Yes I did, my mistake, sorry about it.

On Jan 11, 2017 11:00 AM, "Rajeev Manur" <[email-masked]> wrote:

> Hi Pankaj,
>
> Thanks for the notes. I participated in yesterday's discussion as well.
> Looks like you missed my name on the participants list.
>
> Thanks!
> --Rajeev
>
> Sent from my iPhone
>

---

From: Sami Boutros <[email-masked]>
Date: Mon, 16 Jan 2017 05:59:40 +0000

Please find attached the initial draft.

Thanks,

Sami
From: nvo3-dt-encap <[email-masked]<mailto:[email-masked]>> on behalf of Pankaj Garg <[email-masked]<mailto:[email-masked]>>
Date: Thursday, January 12, 2017 at 12:12 AM
To: Rajeev Manur <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3-dt-encap] Discussion summary - 1/10/2017

On Jan 11, 2017 11:00 AM, "Rajeev Manur" <[email-masked]<mailto:[email-masked]>> wrote:
Hi Pankaj,

Thanks!
--Rajeev

Sent from my iPhone

> On Jan 11, 2017, at 7:25 AM, Pankaj Garg <[email-masked]<mailto:[email-masked]>> wrote:
>
> Hi All,
>
> [email-masked]<mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/nvo3-dt-encap<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_nvo3-2Ddt-2Dencap&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=IVzcTRLQdpta08L0b_y2zDkqvwJhRKMCAbX-2K-LV98&m=nm_Ctkl_dvuZF0yR7wFkzKY9084l5Bdok5sR2g1LFX4&s=t-U-Xb4i7VQD8CuCVfPVBuNiMKRYKD3RkKDuxtkXWxs&e=>

---

From: David Mozes <[email-masked]>
Date: Tue, 17 Jan 2017 14:30:55 +0000

Hi Sami  ,
Thanks  for the draft

1)      Can you elaborate on the trace _id option  ?

2)      Extension TLV Ordering in Geneve implementations

“We would propose a text such as, TLV processing may not be easy

   processed in TCAM. There may be some critical information such as

   secure hash that must get processed at lowest latency. A control

  plane can guarantee certain TLV ordering, to allow hardware capable

   of processing one or few options TLV in TCAM, to have the those TLVs

   first. “

I believe we didn’t agree on that . The only think  we agree is text regarding TLV 0 we already have

Thx

David

From: nvo3-dt-encap [mailto:[email-masked]] On Behalf Of Sami Boutros
Sent: Monday, January 16, 2017 8:00 AM
To: Pankaj Garg <[email-masked]>; Rajeev Manur <[email-masked]>
Cc: [email-masked]
Subject: Re: [nvo3-dt-encap] Discussion summary - 1/10/2017

Thanks,

Sami

Thanks!
--Rajeev

Sent from my iPhone

---

From: Tal Mizrahi <[email-masked]>
Date: Mon, 23 Jan 2017 12:34:22 +0000

Hi Sami,

Thanks for putting together this draft.

I would like to propose one of two options:

(1) Replace subsections 6.1-6.6 with the encapsulation comparison section we reviewed a couple of weeks ago.
Or
(2) Replace subsections 6.1-6.6 with the following text, and append the comparison section as an appendix. This would be the text at the beginning of Section 6:
------------------------------------------------------------------

6.1 Encapsulation Comparison

Appendix A includes a detailed comparison between the three proposed encapsulations. The comparison indicates several common properties, but also three major differences among the encapsulations:

-	Extensibility: Geneve and GUE were defined with built-in extensibility, while VXLAN-GPE is not inherently extensible. Note that any of the three encapsulations can be extended using the Network Service Header (NSH).
-	Extension method: Geneve is extensible using Type/Length/Value (TLV) fields, while GUE uses a small set of possible extensions, and a set of flags that indicate which extension is present.
-	Length field: Geneve and GUE include a Length field, indicating the length of the encapsulation header, while VXLAN-GPE does not include such a field.

Further details can be found in Appendix A.

------------------------------------------------------------------
And then we add Appendix A:
------------------------------------------------------------------

3.  Appendix A: NVO3 Encapsulation Comparison

3.1.  Overview

   This section presents a comparison of the three NVO3 encapsulation
   proposals, Geneve, GUE, and VXLAN-GPE.  The three encapsulations use
   an outer UDP/IP transport.  Geneve and VXLAN-GPE use an 8-octet
   header, while GUE uses a 4-octet header.  In addition to the base
   header, optional extensions may be included in the encapsulation, as
   discussed in Section 3.2 below.

3.2.  Extensibility

3.2.1.  Native Extensibility Support

   The Geneve and GUE encapsulations both enable optional headers to be
   incorporated at the end of the base encapsulation header.

   VXLAN-GPE does not provide native support for header extensions.
   However, as discussed in [I-D.ietf-nvo3-vxlan-gpe], extensibility can
   be attained to some extent if the Network Service Header (NSH)
   [I-D.ietf-sfc-nsh] is used immediately following the VXLAN-GPE
   header.  NSH supports either a fixed-size extension (MD Type 1), or a
   variable-size TLV-based extension (MD Type 2).  It should be noted
   that NSH-over-VXLAN-GPE implies an additional overhead of the
   8-octets NSH header, in addition to the VXLAN-GPE header.

3.2.2.  Extension Parsing

   The Geneve Variable Length Options are defined as Type/Length/Value
   (TLV) extensions.  Similarly, VXLAN-GPE, when using NSH, can include
   NSH TLV-based extensions.  In contrast, GUE defines a small set of
   possible extension fields (proposed in [I-D.herbert-gue-extensions]),
   and a set of flags in the GUE header that indicate for each extension
   type whether it is present or not.

   TLV-based extensions, as defined in Geneve, provide the flexibility
   for a large number of possible extension types.  Similar behavior can
   be supported in NSH-over-VXLAN-GPE when using MD Type 2.  The flag-
   based approach taken in GUE strives to simplify implementations by
   defining a small number of possible extensions, used in a fixed
   order.

   The Geneve and GUE headers both include a length field, defining the
   total length of the encapsulation, including the optional extensions.
   The length field simplifies the parsing of transit devices that skip
   the encapsulation header without parsing its extensions.

3.2.3.  Critical Extensions

   The Geneve encapsulation header includes the 'C' field, which
   indicates whether the current Geneve header includes critical
   options, which must be parsed by the tunnel endpoint.  If the
   endpoint is not able to process the critical option, the packet is
   discarded.

3.2.4.  Maximal Header Length

   The maximal header length in Geneve, including options, is 260
   octets.  GUE defines the maximal header to be 128 octets.  VXLAN-GPE
   uses a fixed-length header of 8 octets, unless NSH-over-VXLAN-GPE is
   used, yielding an encapsulation header of up to 264 octets.

3.3.  Encapsulation Header

3.3.1.  Virtual Network Identifier (VNI)

   The Geneve and VXLAN-GPE headers both include a 24-bit VNI field.
   GUE, on the other hand, enables the use of a 32-bit field called
   VNID; this field is not included in the GUE header, but was defined
   as an optional extension in [I-D.herbert-gue-extensions].

   The VXLAN-GPE header includes the 'I' bit, indicating that the VNI
   field is valid in the current header.  A similar indicator is defined
   as a flag in the GUE header [I-D.herbert-gue-extensions].

3.3.2.  Next Protocol

   The three encapsulation headers include a field that specifies the
   type of the next protocol header, which resides after the NVO3
   encapsulation header.  The Geneve header includes a 16-bit field that
   uses the IEEE Ethertype convention.  GUE uses an 8-bit field, which
   uses the IANA Internet protocol numbering.  The VXLAN-GPE header
   incorporates an 8-bit Next Protocol field, using a VXLAN-GPE-specific
   registry, defined in [I-D.ietf-nvo3-vxlan-gpe].

   The VXLAN-GPE header also includes the 'P' bit, which explicitly
   indicates whether the Next Protocol field is present in the current
   header.

3.3.3.  Other Header Fields

   The OAM bit, which is defined in Geneve and in VXLAN-GPE, indicates
   whether the current packet is an OAM packet.  The GUE header includes
   a similar field, but uses different terminology; the GUE 'C-bit'
   specifies whether the current packet is a control packet.  Note that
   the GUE control bit can potentially be used in a large set of
   protocols that are not OAM protocols.  However, the control packet
   examples discussed in [I-D.ietf-nvo3-gue] are OAM-related.

   Each of the three NVO3 encapsulation headers includes a 2-bit Version
   field, which is currently defined to be zero.

   The Geneve and VXLAN-GPE headers include reserved fields; 14 bits in
   the Genever header, and 27 bits in the VXLAN-GPE header are reserved.

3.4.  Comparison Summary

   The following table summarizes the comparison between the three NVO3
   encapsulations.

   +----------------+----------------+----------------+----------------+
   |                |     Geneve     |      GUE       |   VXLAN-GPE    |
   +----------------+----------------+----------------+----------------+
   | Outer transport|     UDP/IP     |     UDP/IP     |     UDP/IP     |
   +----------------+----------------+----------------+----------------+
   | Base header    |    8 octets    |    4 octets    |    8 octets    |
   | length         |                |                |  (16 octets    |
   |                |                |                |   using NSH)   |
   +----------------+----------------+----------------+----------------+
   | Extensibility  |Variable length |Extension fields| No native ext- |
   |                |    options     |                | ensibility.    |
   |                |                |                | Extensible     |
   |                |                |                | using NSH.     |
   +----------------+----------------+----------------+----------------+
   | Extension      |   TLV-based    |   Flag-based   |   TLV-based    |
   | parsing method |                |                |(using NSH with |
   |                |                |                |   MD Type 2)   |
   +----------------+----------------+----------------+----------------+
   | Extension      |    Variable    |     Fixed      |    Variable    |
   | order          |                |                |  (using NSH)   |
   +----------------+----------------+----------------+----------------+
   | Length field   |       +        |       +        |       -        |
   +----------------+----------------+----------------+----------------+
   | Max Header     |   260 octets   |   128 octets   |    8 octets    |
   | Length         |                |                |(264 using NSH) |
   +----------------+----------------+----------------+----------------+
   | Critical exte- |       +        |       -        |       -        |
   | nsion bit      |                |                |                |
   +----------------+----------------+----------------+----------------+
   | VNI field size |    24 bits     |    32 bits     |    24 bits     |
   |                |                |  (extension)   |                |
   +----------------+----------------+----------------+----------------+
   | Next protocol  |    16 bits     |     8 bits     |     8 bits     |
   | field          |   Ethertype    | Internet prot- |  New registry  |
   |                |   registry     | ocol registry  |                |
   +----------------+----------------+----------------+----------------+
   | Next protocol  |       -        |       -        |       +        |
   | indicator      |                |                |                |
   +----------------+----------------+----------------+----------------+
   | OAM / control  |    OAM bit     |  Control bit   |    OAM bit     |
   | field          |                |                |                |
   +----------------+----------------+----------------+----------------+
   | Version field  |    2 bits      |    2 bits      |    2 bits      |
   +----------------+----------------+----------------+----------------+
   | Reserved bits  |    14 bits     |       -        |    27 bits     |
   +----------------+----------------+----------------+----------------+

                  Figure 1: NVO3 Encapsulation Comparison

>-----Original Message-----
>To: Pankaj Garg; Rajeev Manur
>Subject: [EXT] Re: [nvo3-dt-encap] Discussion summary - 1/10/2017
>
>External Email
>________________________________________
>From: nvo3-dt-encap <[email-masked]> on behalf of Pankaj
>Garg <[email-masked]>
>To: Rajeev Manur <[email-masked]>
>Cc: "[email-masked]" <[email-masked]>
>Thanks for the notes. I participated in yesterday's discussion as well. Looks like
>you missed my name on the participants list.
>
>Thanks!
>--Rajeev
>
>Sent from my iPhone
>

---

From: "Michael Smith (michsmit)" <[email-masked]>
Date: Tue, 24 Jan 2017 15:40:10 +0000

Here is some text for a few sections with my name:

Section 6.2 >>>
Another use case would be to carry the Group Based Policy (GBP) source group information within a NVO3 header extension in a similar manner as has been implemented for VXLAN [reference]. This allows various forms of policy such as access control and QoS to be applied between abstract groups rather than coupled to specific endpoint addresses.

Section 6.4 >>>
Extension header length has a significant impact to hardware and software implementations. A total header length that is too small will unnecessarily constraint software flexibility. A total header length that is too large will place a nontrivial cost on hardware implementations. Thus, the design team recommends that there be a minimum and maximum total extension header length selected. The maximum total header length is determined by the bits allocated for the total extension header length field. The risk with this approach is that it may be difficult to extend the total header size in the future. The minimum total header length is determined by a requirement in the specifications that all implementations must meet. The risk with this approach is that all implementations will only implement the minimum total header length which would then become the defacto maximum total header length. The recommended minimum total header length is TBD. 64 ?

Thanks,
Michael

From: nvo3-dt-encap <[email-masked]<mailto:[email-masked]>> on behalf of Sami Boutros <[email-masked]<mailto:[email-masked]>>
Date: Sunday, January 15, 2017 at 9:59 PM
To: Pankaj Garg <[email-masked]<mailto:[email-masked]>>, Rajeev Manur <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3-dt-encap] Discussion summary - 1/10/2017

Thanks,

Sami

Thanks!
--Rajeev

Sent from my iPhone