Subject: [nvo3] draft-ietf-nvo3-framework-03.txt

From: Thomas Narten <[email-masked]>
Date: Tue, 29 Oct 2013 11:01:54 -0400

Hi.

What is the status of this document? It doesn't seem to have been
revised since July, despite being "almost done" in Berlin. Can the
authors please comment on what is holding this document back?

I went back and reviewed the -03 version and found some things that
should be cleaned up before sending to the IESG. But all the changes
are editorial and should be straightforward to make (see below).

Thomas

2013-10-29: Review of -03

Now that document is going to IESG, abstract should be redone to
reflect what the document actually contains.

>     8. Acknowledgments 

This section needs updating to properly give credit.

>     1.1. Conventions used in this document  
> 
>        The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 
>        "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 
>        document are to be interpreted as described in RFC-2119 [RFC2119].  
> 
>        In this document, these words will appear with that interpretation   
>        only when in ALL CAPS. Lower case uses of these words are not to be    
>        interpreted as carrying RFC-2119 significance. 

These definitions don't actually make sense in this document. The
framework is not a specification, so things like

  1. MUST   This word, or the terms "REQUIRED" or "SHALL", mean that the
     definition is an absolute requirement of the specification.

don't really make sense. That said, if you search for upper case
words, the document doesn't really use them (I find one
SHOULD). Why not just lower case everything and remove Section 1.1?

>        Network Virtualization Edge (NVE). An NVE is the network entity that 
>        sits at the edge of an underlay network and implements L2 and/or L3 
>        network virtualization functions. The network-facing side of the NVE 
>        uses the underlying L3 network to tunnel frames to and from other 

/frames/Tenant System frames/ ?

>        protocol (and address family) from that of the overlay. In the case 
>        of NVO3, the underlay network is typically IP. 

s/is typically IP/will be IP/

(NVO3 is only doing overlays across an IP underlay)

>        Virtual Access Points (VAPs): Tenant Systems are connected to VNIs 
>        through VAPs. VAPs can be physical ports or virtual ports identified 
>        through logical interface identifiers (e.g., VLAN ID, internal 
>        vSwitch Interface ID connected to a VM). 

This could be clearer that the VAP is on the NVE side (as opposed to
the TSI).

Better:

       Virtual Access Points (VAPs): A logical connection point on the
       NVE for connecting a Tenant System to a virtual network. Tenant
       Systems connect to VNIs at an NVE through VAPs. VAPs can be
       physical ports or virtual ports identified through logical
       interface identifiers (e.g., VLAN ID, internal vSwitch
       Interface ID connected to a VM).

>        Network Virtualization Authority (NVA): Entity that provides 
>        reachability and forwarding information to NVEs. An NVA is also 
>        known as a controller.  

Drop the last sentence? Term "controller" means different things to
different people and I don't think it's approriate to equate NVA with
controller.

>        The NVE implements network virtualization functions that allow for 
>        L2 and/or L3 tenant separation.

It does more than "allow for" it provides that service...

Better:

        The NVE implements the network virtualization functions
        corresponding to the L2 or L3 service being provided.

Or, just drop the sentence entirely - its not clear why this sentence
is included here (we've said the above before).

>        Underlay nodes utilize L3 technologies to interconnect NVE nodes. 
>        These nodes perform forwarding based on outer L3 header information, 
>        and generally do not maintain per tenant-service state albeit some 
>        applications (e.g., multicast) may require control plane or 
>        forwarding plane information that pertain to a tenant, group of 
>        tenants, tenant service or a set of services that belong to one or 
>        more tenants. When such tenant or tenant-service related information 
>        is maintained in the underlay, mechanisms to control that 
>        information should be provided.

I think its inaccurate to imply that when the underlay uses IP
multicast, it has "tenant state".

Better:

        Underlay nodes utilize L3 technologies to interconnect NVE
        nodes.  These nodes perform forwarding based on outer L3
        header information, and generally do not need to know anything
        about the tenant traffic they are carrying. The case of tenant
        multicast/broadcast may be a slight exception. One way to
        implement tenant multicast/broadcast is to map the tenant
        broadcast/multicast group into an IP multicast group on the
        underlay. In such cases, tenant multicast would be sent on the
        underlay using the underlay's native IP multicast
        capability. In some sense, per-tenant information is being
        used in the underlay. But because the underlay is using
        normal IP functionality, the underlay does not need to know
        the details about the multi-tenancy it is supporting.

Or, just drop the talk about the underlay having per tenant state.

>        Virtual Network Instance (VNI): A specific instance of a VN. 

This definition is not consistent with how VNI is used within this
document. Later, VNI refers to a way to reference a VN on *one*
*single* NVE.

Better:

        Virtual Network Instance (VNI): A specific instance of a VN
        from the perspective of an NVE.

>     2.3. NVE Service Types 
> 
>        NVE components may be used to provide different types of virtualized 
>        network services. This section defines the service types and 
>        associated attributes. Note that an NVE may be capable of providing 
>        both L2 and L3 services. 

Replace first sentence with:

An NVE provides virtualized network services to Tenant Systems.

>        Once the VN context identifier is added to the frame, a L3 Tunnel 
>        encapsulation is used to transport the frame to the destination NVE. 
>        The underlay devices do not usually keep any per service state, 
>        simply forwarding the frames based on the outer tunnel header. 

why "usually?"

Better something like: "does not need to keep ..."

>        may be configured or dynamically established. Solutions SHOULD 

s/SHOULD/should/

>        stateless. In this document, however, tunneling and tunneling 
>        encapsulation are used interchangeably to simply mean the 
>        encapsulation of a tenant packet with a tunneling header necessary 
>        to deliver the packet between an ingress NVE and an egress NVE 

s/to deliver/to carry/

>           o Decoupling of the overlay addresses (MAC and IP) used by VMs 
>             from the underlay network for tenant separation and separation 
>             of the tenant address spaces and the underlay address space. 

s/and the underlay/from the underlay/

>        Overlay networks also create several challenges: 
> 
>           o Overlay networks have no controls of underlay networks and lack 
>             critical underlay network information 
> 

Not sure what that last point is trying to say. Either clarify (give
an example?) or delete. It is not right to say "lack critical
information" without saying what this really means. This is just hand
waving.

>                o Overlay networks and/or their associated management 
>                  entities typically probe the network to measure link or 
>                  path properties, such as available bandwidth or packet 
>                  loss rate. It is difficult to accurately evaluate network 
>                  properties. It might be preferable for the underlay 
>                  network to expose usage and performance information. 

Indention seems off here. Also, where does the "typically" come from?
This implies it is standard practice today. Is it?

>        In this section, we will only consider the case of an IP overlay. 

Remove? NVO3 only addresses IP overlays.

>           . Avoiding packets from reaching the wrong NVI, especially during 
>             VM moves. 
s/Avoiding/Preventing/

Nits:

>        NVO3 Network: An overlay network that provides an Layer2 (L2) or 
>        Layer3 (L3) service to Tenant Systems over an L3 underlay network, 

s/,//

>        End Device: A physical device that connects directly to the DC 
>        Underlay Network. This is in contrast to a tenant system, which 
>        connects to a corresponding tenant VN. An End Device is administered 

s/tenant system/Tenant System/

>             swicthes are usually multi-homed to aggregation switches in the 

spell

>        L2 NVE implements Ethernet LAN emulation, an Ethernet based 

s/L2/An L2/

>        L3 NVE provides Virtualized IP forwarding service, similar from a 

s/L3/An L3/

s/similar from/similar to/

>        A VNI is a specific VN instance on a NVE. Each VNI defines a 

s/a NVE/an NVE/

>        Once the VN context identifier is added to the frame, a L3 Tunnel 

s/a L3/an/ L3/

>        across the underlay (e.g., IP tunneling over an IP underlay. 

s/underlay/underlay)/

>           o Classical ICMP-based MTU Path Discovery [RFC1191] [RFC1981] 
> 
>                o 
>                 Tenant Systems rely on ICMP messages to discover the MTU of 
>                  the end-to-end path to its destination. This method is not 
>                  always possible, such as when traversing middle boxes 
>                  (e.g. firewalls) which disable ICMP for security reasons 
> 

formatting issue...

>        Nvo3 solutions must at least consider and address the following: 

s/Nvo3/NVO3/ Indeed, go through document and use consistent spelling
(I find nvo3 as well)

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 29 Oct 2013 15:36:55 +0000

Thomas,

The -04 version which includes editorial comments received so far is ready.
The draft did not get posted in time (i.e. before the deadline) because of some late editorial suggestions.

Hence, the draft will get posted next week when the filing window gets open again.
I will address your comments at that time.

Thanks,
Marc

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Thomas Narten
> Sent: Tuesday, October 29, 2013 4:02 PM
> To: [email-masked]
> Subject: [nvo3] draft-ietf-nvo3-framework-03.txt
> 
> Hi.
> 
> What is the status of this document? It doesn't seem to have 
> been revised since July, despite being "almost done" in 
> Berlin. Can the authors please comment on what is holding 
> this document back?
> 
> I went back and reviewed the -03 version and found some 
> things that should be cleaned up before sending to the IESG. 
> But all the changes are editorial and should be 
> straightforward to make (see below).
> 
> Thomas
> 
> Now that document is going to IESG, abstract should be redone 
> to reflect what the document actually contains.
> 
> >     8. Acknowledgments
> 
> These definitions don't actually make sense in this document. 
> The framework is not a specification, so things like
> 
> don't really make sense. That said, if you search for upper 
> case words, the document doesn't really use them (I find one 
> This could be clearer that the VAP is on the NVE side (as 
> opposed to the TSI).
> 
> Better:
> 
> Drop the last sentence? Term "controller" means different 
> things to different people and I don't think it's approriate 
> to equate NVA with controller.
> 
> Or, just drop the sentence entirely - its not clear why this 
> sentence is included here (we've said the above before).
> 
> I think its inaccurate to imply that when the underlay uses 
> IP multicast, it has "tenant state".
> 
> Better:
> 
> This definition is not consistent with how VNI is used within 
> this document. Later, VNI refers to a way to reference a VN on *one*
> *single* NVE.
> 
> Better:
> 
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3
>

---

From: Lucy yong <[email-masked]>
Date: Tue, 29 Oct 2013 18:23:11 +0000

Hi Marc,

Don't know if you follow the nvo3 mail discussion regarding to the distributed gateways. Now more people support defining a new NVE service type, i.e. L2/L3 NVE service type, where multiple L2 VNs can be interconnected via a distributed L3 router/gateway. Do you plan to add it into the framework document?  

Regards,
Lucy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of LASSERRE, MARC (MARC)
Sent: Tuesday, October 29, 2013 10:37 AM
To: Thomas Narten; [email-masked]
Subject: Re: [nvo3] draft-ietf-nvo3-framework-03.txt

Thomas,

Thanks,
Marc

> From: [email-masked] [mailto:[email-masked]] On Behalf 
> Of Thomas Narten
> should be cleaned up before sending to the IESG.
> But all the changes are editorial and should be straightforward to 
> make (see below).
> 
> Thomas
> 
> words, the document doesn't really use them (I find one SHOULD). Why 
> not just lower case everything and remove Section 1.1?
>

---

From: Thomas Narten <[email-masked]>
Date: Tue, 29 Oct 2013 15:58:20 -0400

Marc,

Getting a revision out next week sounds good. It would be good to get
this one out of the WG.

Thomas

> The -04 version which includes editorial comments received so far is
> ready.  The draft did not get posted in time (i.e. before the
> deadline) because of some late editorial suggestions.

> Hence, the draft will get posted next week when the filing window
> gets open again.  I will address your comments at that time.

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Wed, 30 Oct 2013 12:50:18 +0000

Lucy,

This discussion is within the context of the architecture draft and hence there is no plan at this stage (where the framework document is about to go through IESG review) to make significant changes. 

Thanks,
Marc

> From: Lucy yong [mailto:[email-masked]] 
> Sent: Tuesday, October 29, 2013 7:23 PM
> To: LASSERRE, MARC (MARC); Thomas Narten; [email-masked]
> Subject: RE: [nvo3] draft-ietf-nvo3-framework-03.txt
> 
> Hi Marc,
> 
> Don't know if you follow the nvo3 mail discussion regarding 
> to the distributed gateways. Now more people support defining 
> a new NVE service type, i.e. L2/L3 NVE service type, where 
> multiple L2 VNs can be interconnected via a distributed L3 
> router/gateway. Do you plan to add it into the framework document?  
> 
> Regards,
> Lucy
> 
> Behalf Of LASSERRE, MARC (MARC)
> The draft did not get posted in time (i.e. before the 
> SHOULD). Why 
> > >        uses the underlying L3 network to tunnel frames to 
> and from 
> > > other
> > 
> opposed to

---

From: Lucy yong <[email-masked]>
Date: Wed, 30 Oct 2013 15:19:30 +0000

Marc and the Chairs,

The architecture draft has added distributed gateway, which means that NVE may implement inter VN gateway function. During the discussions, many people agree that, in order for NVE to implement inter VN gateway function, a new NVE service type is necessary. This belongs to the framework, right? If the WG decides no change on the current scope of the framework draft, it means that we will have a framework addition draft to describe the distributed gateway. Furthermore the distributed gateway function also adds additional requirements to data plane and control plane. Larry will add related control plane requirements in the cp draft. Should we add additional data plane requirements into the dp draft or capture them in a separated draft?

We submitted the framework and data plane requirement addition draft while age, and be glad to use it if the WG decides documenting distributed gateway framework and data plane requirements in a separated document. 

IMO: we can do in either way, but the WG should make a decision on this soon.

Regards,
Lucy

-----Original Message-----
From: LASSERRE, MARC (MARC) [mailto:[email-masked]] 
Sent: Wednesday, October 30, 2013 7:50 AM
To: Lucy yong; Thomas Narten; [email-masked]
Subject: RE: [nvo3] draft-ietf-nvo3-framework-03.txt

Lucy,

Thanks,
Marc

> Don't know if you follow the nvo3 mail discussion regarding to the 
> distributed gateways. Now more people support defining a new NVE 
> service type, i.e. L2/L3 NVE service type, where multiple L2 VNs can 
> be interconnected via a distributed L3 router/gateway. Do you plan to 
> add it into the framework document?
> 
> Regards,
> Lucy
> 
> Of LASSERRE, MARC (MARC)
> ready.