Subject: [nvo3] Comments to draft-lasserre-nvo3-framework-02

From: Linda Dunbar <[email-masked]>
Date: Fri, 29 Jun 2012 17:59:47 +0000

Marc, Florin, Thomas, Nabil, and Yakov,

Applications in data center frequently communicate with peers in different subnets. However, the "nvo3 framework" document hasn't addressed this important aspect of data center.

For example the Figure 3 in the draft should at least show Gateway Routers.
If you say that Gateway routers would be one of those "Tenant End System" in the Figure3, then all the VIDs associated with different Virtual networks under all other NV Edge will appear on the links from VNEdge to "Gateway router's Tenant End System".

The generic reference model shown in Figure 4 only addressed communications of TES within one subnet (VLAN), but hasn't addressed inter subnet communications.

Section 3.1.5.1 states that "NVEs must be able to select appropriate VNI for each TES". It should say NVEs must be able to map the VNI of a TES to an appropriate NVID. VNI (virtual network instance) is associated with TES. VNID is assigned by the network (NVE or management system).

The second paragraph of Section 3.1.5.1 states "A mechanism for communicating this information between TES and Local NVE is required".  It is not realistic to require all the Guest OSs (applications) being deployed to support a new protocol to communicate with NVE.  A more appropriate statement would be:
NVEs would need assistance from external entities or information (e.g. messages) from TES to map TES's VNI to its corresponding VNID.

Because it is not realistic to require all Guest OSs (applications) which might be instantiated to VMs of hypervisors, you need to add this sentence to the end of the third paragraph of 3.1.5.1:
The auto-discovery protocol between Tenant End System and their local NVE should include options of not requiring any new features to be added to applications OSs.

As described in http://www.ietf.org/id/draft-dunbar-nvo3-overlay-mobility-issues-00.txt, mobility introduces new challenges and problems to overlay network. Therefore, more subsections  are needed in Auto-provisioning/Service discovery of Section 3.1.5 (Control Plane Comp).

If NVEs can get NVID mapping and local VID mapping from external entities before tenant TESs are attached to NVEs, then "Auto-provisioning/service discovery" becomes straight forward. VAP creation can be triggered by the external entity. Even those straight forward steps could require several potential actions by  NVEs:

-        Mapping the VID encoded in data frame to appropriate VAP (Virtual Access Point) with its VNID.

-        Removing the VID encoded in data frames from TES before encapsulation.

-        Change the VID encoded in the data frames from TES to a different value before the encapsulation. or

-        Map untagged data frame to appropriate VAPs

When NVEs don't have any interface to an external entity who is aware of TES being added to or deleted from the NVE, the scheme becomes much trickier. Following scenarios have to be considered:

-        A new VID is observed by NVE

-        A new address (e.g. MAC) is observed by NVE

-        Age out local VIDs under NVE.

I think that 3.1.5 should add a subsection on Local VID management.

3.1.5.x Local VID management

It is necessary to manage local VID when data frames arriving at NVEs are VID encoded (e.g. by VMs themselves), or the remote NVEs (i.e. egress NVE) need  local VID to segregate traffic among all the attached nodes.

-        Local VIDs Managed by External Controller
When NVE can be informed of TESs being added/deleted and their associated tenant virtual networks via external controller, NVE should be able to get the specific VNID from its controller for data frames arriving at its Virtual Access Points [VNo3-framework 3.1.1].

If the data frame is tagged before reaching the NVE's Virtual Access Point (e.g. tagged data frames from VMs) and NVE is more than one hop away from TES, the first (virtual) port facing the TES has be informed by the external controller of the new local VID to replace the VID encoded in the data frames. For reverse direction, i.e. data frames coming from core towards TESs, the first switching port facing TESs have to convert the VIDs encoded in the data frames to the VIDs used by TESs.

Since local VIDs under each NVE are really locally significant, it might be less confusing to egress NVE if ingress NVE remove the local VID attached to the data frame. So that egress NVE always has to assign its own local VID to data frame before sending the decapsulated data frame to attached VMs.
If, for whatever reason, it is necessary to have local VID in the data frames before encapsulating outer header of EgressNVE-DA/ IngressNVE-SA /VNID, NVE should get the specific local VID from the external Controller for those untagged data frames coming to each Virtual Access Point.

-        Local VIDs Managed by NVE
If NVEs don't have interface to any controllers which can be informed of TESs being added to or deleted from NVEs, then NVEs have to learn new TESs/VLANs being attached, figure out to which virtual network instance  those TESs/VLANs belong, and/or age out TESs/VLANs after a specified timer expires. Network management system has to assist NVEs in making the decision, even if the network management system doesn't have interface to VM/server managers.
When NVE receives a data frame with a new TES address (e.g. MAC) in a tagged data frame from its Virtual Access Point, the new TES could be from an existing local virtual network, from a different virtual network (being brought in as the VM being added in), or from an illegal TES.
Upon NVE learns a new TES being added, either by learning a new MAC address or a new VID, it needs its management system to confirm the validity of the new VID and/or new address. If the new address or VID is from invalid or illegal source, the data frame has to be dropped.

Regards, Linda Dunbar