Subject: [nvo3-dt-encap] NVO3 Design Team - 12/12/2016 Discussion Summary

From: Pankaj Garg <[email-masked]>
Date: Thu, 15 Dec 2016 15:58:13 -0800

Hi All,

Thank you for attending the meeting yesterday (12/12). It was very productive
discussion. Thanks to Ignas for setting it up. Please find my curated summary
of the discussion below.

1. Agenda Setting
   We spent some time discussing agenda (below):

    - Meeting Schedule/Logistics
      We agreed to setup a weekly 2 hour meeting. Pankaj would be sending
      doodle.com requests for people to vote on their preferred time and then
      decide upon a time for regular meeting. This would be online only
      meeting and would be converted to f2f meeting on need basis.

    - Draft write-up volunteer
      Ignas and Tal have stepped up to begin writing a draft document. Tal would
      focus on capturing comparison between various existing encapsulation
      protocols and Ignas would be writing a draft on the design team
      recommended encapsuation.

    - Discussion notes
      Pankaj would capture meeting notes as well as setup online share to store
      these notes.

2. Technical Discussion
   Team spent significant time discussing various design options, concerns
   raised on nvo3 mailing list with existing protocols and how we can make a
   proposal that would be good enough to meet various requirements without
   making it prohibitevly expensive for hardware to implement.

   We categorized various important aspects in the proposed solution, on which a
   decision needs to be made, as described below:

   - Extension Size - What should be maximum size of extensions? This is
     important because a small size may be too restrictive and a large size
     would make it expensive for hardware to support.

   - Single Extension Size - What should be minimum and maximum size of
     individual extension? This may be important to align packets.

   - Extension Ordering - For TLV style extensions, should we require certain
     ordering to make parsing and processing easier (less intensive)?

   - Fixed or Variable Size - Should each extension be fixed size or variable
     sized?

   - Decoupled Metadata - Should we decouple extension data such that it can be
     carried in other protocols? This has advantages for split-NVE type
     scenarios where metadata may be needed to go over VLAN.

   - TLV vs Bit Fields - A decision needs to be made on this. There are pros and
     cons of each. TLVs give more flexibility but thus can be more expensive for
     hardware.

   - 32-bit or larger VNI - Discussed whether we should make VNI 32-bit or
     larger. It was decided that we would keep 24-bit VNI and if a larger VNI is
     needed, an extension can be used to support that. This would avoid
     unncessary changes with existing proposals and implementations that are
     almost all, if not all, are using 24-bit VNI.

   - ETYPE vs New Protocol Type Registry - Discussed and decided that we should
     stick to ETYPE instead of creating new registry like NSH did.

Note: I am using the term metadata to represent data that is carried using
extensions. We can formalize the terminology in our next discussion.

Besides these, there was discussion on metadata itself and why VNI itself is not
a metadata and thus not part of extensions itself. We agreed that we should
define network virtualiztion encapsulation (referred as from here on nvencap),
in such a way there is a base nvencap header that is lean and carries only
critical information required for network virtualization and has an extension
header part that can be used for additional metadata. We also agreed that we
should define the extension header in such a way that allows using it in other
protocols.

As an example, we can do following:

--------------------------------
Ver|HLEN|O|Rsvd|     ETYPE     |
--------------------------------
          VNI         |  Rsvd  |
--------------------------------
          Metadata
             ...
--------------------------------

And define a separate 4-byte header, if metadata represented by extension needs
to be carried over other protocols such as 802.1q etc. For example:

--------------------------------
Ver|HLEN| Rsvd |     ETYPE     |
--------------------------------
          Metadata
             ...
--------------------------------

We concluded the discussion at this point. Others please feel free to add, if I
missed anything. I am also attaching the pictures from the whiteboard.

I would put these notes on some online share, so we can keep track of the
discussions, and send a link out.

Thanks
Pankaj

---

From: Pankaj Garg <[email-masked]>
Date: Thu, 15 Dec 2016 15:58:53 -0800

Whiteboard pictures from 12/12.