Subject: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 00:20:45 +0000

Hi Thomas,

How about the following text:

          A tenant network may require both L2 and L3
          services, i.e. the packets from a tenant may get L2 and/or L3 treatment.
          An NVE receiving packets from a TS determines the type of service to be applied to
          the packet on a per-packet basis and as indicated by the
          destination MAC address on the packet.  If
          the MAC address corresponds to that of an L3 router (as
          determined by the NVE), traffic is given L3
          semantics. Otherwise, the packet is given L2
          semantics. Thus the packet from a tenant must contain IP header (as well
         as Ethernet header) .  A combined L2/L3 service presents no special
          considerations for NVO3, other than packets received from a
          tenant must be classified as to what type of service they
          are to be given before they can be processed.

Comment: should we use a tenant system (TS) or a tenant in the text to make consistency?

Regards,
Lucy

---------- Forwarded message ----------
From: Thomas Narten <[email-masked]<mailto:[email-masked]>>
Date: Mon, Nov 18, 2013 at 3:25 PM
Subject: [nvo3] Arch: proposed text for Combined L2/L3 Service
To: [email-masked]<mailto:[email-masked]>

There has been some discussion on the list about whether or not to
have a combined L2/L3 service. Here is proposed text for the
architecture document:

        <t>
          A virtual network can also provide a combined L2 and L3
          service to tenants. In such cases, a tenant sends and
          receives both L2 and L3 packets. An NVE recieving packets
          from a TS determines the type of service to be applied to
          the packet on a per-packet basis as indicated by the
          packet's destination MAC address as provided by the TS.  If
          the MAC address corresponds to that of an L3 router (as
          determined by the NVE), traffic is given L3
          semantics. Otherwise, the packet is given L2 service
          semantics. A combined L2/L3 service presents no special
          considerations for NVO3, other than packets received from a
          tenant must be classified as to what type of service they
          are to be given before they can be processed.
        </t>

This text would go at the end of Section 3.1 "VN Service (L2 and L3)".

Make sense?

Additional thinking behind this (taken from mail within the DT):

> FWIW, I think that there are separate issues here. One is simply
> describing what we mean by a combined L2/L3 service. That is what my
> text was trying to do. The thinking is if the service definition is
> clear and simple, supporting it in NVO3 is not a problem. I.e., it's
> not really much different to provide a combined service than if you
> offer an L2 or L3 service. And the service semantics for combined
> L2/L3 are easy to understand and explain.  That is goodness. :-)
>
> Whether and how it makes sense to have distributed gateways in a
> combined service is a separate matter. What I'd like to be able to do
> here (if we need to say anything at all) is be able to say that if a
> distributed GW makes sense for L2 service, then having an L2
> distributed GW for the L2 traffic would make sense in the combined
> service case too. Ditto for L3 service.
>
> A key point is that having a combined service is pretty much the same
> as taking the two separate L2 and L3 services and combining them into
> one implementation. There is nothing really "special" about this  that
> would complicate the overall architecture. Where we can easily get
> into trouble is if we start defining a combined service as has special
> cases that have to be dealt with that don't appear when you have only
> L2 or only L3 service semantics.

Thomas

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 19 Nov 2013 16:55:52 -0800

Lucy,

What would the rules be for multicast packets?
Such packets are not addressed to a router's MAC address but instead the 
router receives all of them and processes them based on the L3 multicast 
FIB.

(For products which implement both L2 and L3 the processing of multicast 
depends on whether the input port is L2 or L3, and in some cases that 
will result in both bridging the packet out other L2 ports, and 
forwarding the packet out other L3 (virtual) ports.)

Thus AFAICT the multicast rules can't depend on the destination MAC 
address to determine whether L2 or L3 service should be applied.

    Erik

On 11/19/13 4:20 PM, Lucy yong wrote:
> From: *Thomas Narten* <[email-masked] <mailto:[email-masked]>>
> To: [email-masked] <mailto:[email-masked]>
>
>
> [email-masked] <mailto:[email-masked]>
> [email-masked]

---

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 03:41:33 +0000

Erik,

Good point. L2 service provides a broadcast domain. Whether the broadcast/multicast packet should be sent to another VN is depend on inter VN policy. It is possible. When the broadcast/multicast packet deliver to the L3 router too.  If the policy allows the broadcast/multicast to broadcast to another VN, L3 router will forward the packet to another VN with the same DMAC. If the policy does not allow, L3 router will drop the packet.

You are right, if dMAC is broadcast/multicast address, it should send to L3 router for the further process. I change the text to following.

          A tenant network may require both L2 and L3
          services, i.e. the packets from a tenant may get L2 and/or L3 treatment. 
          An NVE receiving packets from a TS determines the type of service to be applied to
          the packet on a per-packet basis and as indicated by the
          destination MAC address on the packet.  If
          the MAC address corresponds to that of an L3 router (as
          determined by the NVE) traffic is given L3
          semantics with inter-VN policy. When an NVE receives a broadcast/multicast packet, 
          it should be processed by the L3 router. Depending on the inter-VN policy, 
          the broadcast/multicast may be allowed or prohibited. Otherwise, the packet is given L2
          semantics. Thus the packet from a tenant must contain IP header (as well
         as Ethernet header) .  A combined L2/L3 service presents no special
          considerations for NVO3, other than packets received from a
          tenant must be classified as to what type of service they
          are to be given before they can be processed.

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]] 
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

What would the rules be for multicast packets?
Such packets are not addressed to a router's MAC address but instead the router receives all of them and processes them based on the L3 multicast FIB.

(For products which implement both L2 and L3 the processing of multicast depends on whether the input port is L2 or L3, and in some cases that will result in both bridging the packet out other L2 ports, and forwarding the packet out other L3 (virtual) ports.)

Thus AFAICT the multicast rules can't depend on the destination MAC address to determine whether L2 or L3 service should be applied.

    Erik

On 11/19/13 4:20 PM, Lucy yong wrote:
>  > FWIW, I think that there are separate issues here. One is simply  > 
> > offer an L2 or L3 service. And the service semantics for combined  > 
> L2/L3 are easy to understand and explain.  That is goodness. :-)  >  > 
> Whether and how it makes sense to have distributed gateways in a  > 
> > distributed GW makes sense for L2 service, then having an L2  > 
> distributed GW for the L2 traffic would make sense in the combined  > 
>  > A key point is that having a combined service is pretty much the 
> same  > as taking the two separate L2 and L3 services and combining 
> them into  > one implementation. There is nothing really "special" 
> about this  that  > would complicate the overall architecture. Where 
> we can easily get  > into trouble is if we start defining a combined 
> service as has special  > cases that have to be dealt with that don't 
> appear when you have only  > L2 or only L3 service semantics.
>
> Thomas
>

---

From: "Vivek Kumar" <[email-masked]>
Date: Wed, 20 Nov 2013 05:07:29 +0000

Hi Lucy ,
 One comment regarding your statement . It should also be specified how to detect if Tenant Frame has to be L2 serviced or L3 serviced when decapsulating NVO3 overlay header .

Regards,
Vivek

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 9:12 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]] 
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 20 Nov 2013 05:15:22 +0000

Hi Vivek,

Can you explain what you mean?  We are discussing the service provided to
the TS.  How would it look different to the TS for "L2 serviced or L3
serviced when decapsulating"?

Thanks, Larry

On 11/19/13 9:07 PM, "Vivek Kumar" <[email-masked]> wrote:

>Hi Lucy ,
> One comment regarding your statement . It should also be specified how
>to detect if Tenant Frame has to be L2 serviced or L3 serviced when
>decapsulating NVO3 overlay header .
>
>Regards,
>Vivek
>
>
>Good point. L2 service provides a broadcast domain. Whether the
>broadcast/multicast packet should be sent to another VN is depend on
>inter VN policy. It is possible. When the broadcast/multicast packet
>deliver to the L3 router too.  If the policy allows the
>broadcast/multicast to broadcast to another VN, L3 router will forward
>the packet to another VN with the same DMAC. If the policy does not
>allow, L3 router will drop the packet.
>
>You are right, if dMAC is broadcast/multicast address, it should send to
>L3 router for the further process. I change the text to following.
>

---

From: "Vivek Kumar" <[email-masked]>
Date: Wed, 20 Nov 2013 06:00:11 +0000

Hi Larry,
 Consider following diagram . 
TS1->NVE1->L3 network ->NVE2->TS2 .

Now , I can have L2/L3 services between TS attached to NVE1 and NVE2 . Lucy comment clearly explains how NVE1 will detect whether to perform L2 or L3 services for Tenant frame . My question is when NVE2 decapsulates the packet , how it will know which service to provide . 

Let me know if I am still not clear.

Regards,
Vivek

-----Original Message-----
From: Larry Kreeger (kreeger) [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 10:45 AM
To: Vivek Kumar; Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Vivek,

Thanks, Larry

---

From: David Allan I <[email-masked]>
Date: Wed, 20 Nov 2013 07:40:22 +0000

I believe you are over complicating this.

There are L2 multicast frames that have no L3 significance and are confined to the L2 broadcast domain. And there are L3 multicast frames that are also multicast or broadcast in the L2 domain (depending on if L2 filtering prunes the L2 topology or not).

Any IP multicast frame has L2 terminated and is processed at L3.  From an L2 POV the NVE is a bud node. It may simply go no further or mapped to one or more other subnets....(which may require the equivalent of additional edge replication if that is how mcast Is implemented)....

I see this as a well understood behavior. Pulling policy into it is conflating the concept significantly....There is significant NVE/NVA interactions required to offer L3 multicast long before considering policy that should be in or out of scope.

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 4:42 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]]
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: Pankaj Garg <[email-masked]>
Date: Wed, 20 Nov 2013 08:07:20 +0000

Wouldn't the decision to do L2 or L3 service be based on the inner frame fields i.e. destination MAC/IP in the inner frame? Similar to how switches/routers process packets i.e. based on frame's destination MAC and destination IP address (if present)?

IMHO, Thomas's original text (pasted below) describes this quite well and concisely.

>>            are to be given before they can be processed.

Pankaj

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Vivek Kumar
Sent: Wednesday, November 20, 2013 11:30 AM
To: Larry Kreeger (kreeger); Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Larry,

Regards,
Vivek

-----Original Message-----
From: Larry Kreeger (kreeger) [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 10:45 AM
To: Vivek Kumar; Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Vivek,

Thanks, Larry

---

From: Balaji P <[email-masked]>
Date: Wed, 20 Nov 2013 10:01:07 +0000

Lucy,

As you know L2 and L3 Services for Tenant networks might be based on the Tenant Network Topology. Where TS can be configured for multiple L2 Services and L3 Services. IMHO treating L2 and L3 as separate services might be better options w.r.t NVE. So that based on NVE configuration for L2 and L3 Services will be applied to the traffic.

Regards,
Balaji.P

---

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 11:48:42 +0000

Hi Balaji,

Please see on line.

-----Original Message-----
From: Balaji P [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 3:56 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

As you know L2 and L3 Services for Tenant networks might be based on the Tenant Network Topology. 
[Lucy] agreed.

Where TS can be configured for multiple L2 Services and L3 Services.
[Lucy] Do you mean the TS as network compliances w/ multiple VAPs and individual VAPs are associated to a L2 VNI or L3 VNI? If yes, agree too.

IMHO treating L2 and L3 as separate services might be better options w.r.t NVE. So that based on NVE configuration for L2 and L3 Services will be applied to the traffic.
[Lucy] The framework has L2 and L3 NVE service type, which apply to many cases. You can use them to construct various tenant network topologies. But we also have the case that an NVE may serve the distributed gateway function. In this case, the VAP may be associated to an L2 VNI on the NVE, the packet from a TS via the VAP may get L2 or L3 treatment on the NVE. IMO: it is necessary to differentiate this NVE case from other the two NVE services.  

Thanks,
Lucy

Regards,
Balaji.P

> Such packets are not addressed to a router's MAC address but instead 
> the router receives all of them and processes them based on the L3 
> multicast FIB.
> 
> (For products which implement both L2 and L3 the processing of 
> multicast depends on whether the input port is L2 or L3, and in some 
> cases that will result in both bridging the packet out other L2 ports, 
> and forwarding the packet out other L3 (virtual) ports.)
> 
> >            services, i.e. the packets from a tenant may get L2 
> > and/or
> > L3 treatment.
> >
> > > text was trying to do. The thinking is if the service definition 
> > > is clear and simple, supporting it in NVO3 is not a problem. I.e., 
> > > it's not really much different to provide a combined service than 
> > > if you offer an L2 or L3 service. And the service semantics for 
> > > combined  >
> > L2/L3 are easy to understand and explain.  That is goodness. :-)  >  
> > > here (if we need to say anything at all) is be able to say that if 
> > > a distributed GW makes sense for L2 service, then having an L2  >
> > service as has special  > cases that have to be dealt with that 
> > don't appear when you have only  > L2 or only L3 service semantics.
> >
> > Thomas
> >

---

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 12:18:20 +0000

See inline.

-----Original Message-----
From: David Allan I [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 1:40 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

I see this as a well understood behavior. Pulling policy into it is conflating the concept significantly....There is significant NVE/NVA interactions required to offer L3 multicast long before considering policy that should be in or out of scope.
[Lucy] Yes, handling multicast traffic across VNs is more complex network function regardless using distributed gateway function or a gateway. IMO, NVO3 has to able to handle the networking policy. The policy can be simple or very complex. Excluding the policy completely in NVO3, IMO, is it a virtualized network? 

Lucy

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 4:42 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]]
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: David Allan I <[email-masked]>
Date: Wed, 20 Nov 2013 12:25:57 +0000

HI Lucy:

If I am offering a tenant a combined L2/L3 service that includes multicast then I am not sure what multicast specific policies  I would be enforcing, other than perhaps saving the tenant from themselves.

If the VN connects to the internet, not sure I'd offer multicast as part of the service, nor for tenant extranets ....

My initial reaction was that there was a lot of other things that needed to work to implement L3 multicast before we started considering policy....and use cases for policy did not leap out at me....

I hope this helps
Dave

-----Original Message-----
From: Lucy yong [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 1:18 PM
To: David Allan I; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

See inline.

-----Original Message-----
From: David Allan I [mailto:[email-masked]]
Sent: Wednesday, November 20, 2013 1:40 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 4:42 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]]
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 12:26:01 +0000

BTW: draft-yong-nvo3-nve describes use of distributed gateway function and a gateway. A tenant network can use one or both based on its application. NVO3 architecture should allow these networking functions, and let the application to choose. Just like that you can choose a L2 service or L3 service.

Lucy 

-----Original Message-----
From: David Allan I [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 1:40 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 4:42 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]]
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: Lucy yong <[email-masked]>
Date: Wed, 20 Nov 2013 12:33:43 +0000

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of David Allan I
Sent: Wednesday, November 20, 2013 6:26 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

HI Lucy:

My initial reaction was that there was a lot of other things that needed to work to implement L3 multicast before we started considering policy....and use cases for policy did not leap out at me....
[Lucy] I agree and do not like to go that far now .....  

Lucy

I hope this helps
Dave

-----Original Message-----
From: Lucy yong [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 1:18 PM
To: David Allan I; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

See inline.

-----Original Message-----
From: David Allan I [mailto:[email-masked]]
Sent: Wednesday, November 20, 2013 1:40 AM
To: Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, November 20, 2013 4:42 AM
To: Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Erik,

Thanks,
Lucy

-----Original Message-----
From: Erik Nordmark [mailto:[email-masked]]
Sent: Tuesday, November 19, 2013 6:56 PM
To: Lucy yong; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

    Erik

---

From: Lucy yong <[email-masked]>
Date: Thu, 21 Nov 2013 00:06:46 +0000

Sorry to chip in. See inline below.

-----Original Message-----
From: Vivek Kumar [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 12:00 AM
To: Larry Kreeger (kreeger); Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Larry,

[Lucy] The given example is L3 VN. Both NVE1 and NVE2 are the member of the L3VN. NVE2 just decapsulates it and forward to TS2. No service needs to provide at egress. In the case, the VAP interface between TS2-NVE2 is VLAN interface, NVE2 learns the TS2 MAC address ahead and inserts the TS2 MAC as the dst MAC and its MAC as its src MAC prior sending to TS2. If the VAP is the IP interface, NVE2 forward the packet directly to TS2.

Lucy   

Regards,
Vivek

-----Original Message-----
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Wednesday, November 20, 2013 10:45 AM
To: Vivek Kumar; Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Vivek,

Can you explain what you mean?  We are discussing the service provided to the TS.  How would it look different to the TS for "L2 serviced or L3 serviced when decapsulating"?

Thanks, Larry

On 11/19/13 9:07 PM, "Vivek Kumar" <[email-masked]> wrote:

>> > clear and simple, supporting it in NVO3 is not a problem. I.e.,

---

From: "Vivek Kumar" <[email-masked]>
Date: Thu, 21 Nov 2013 04:32:46 +0000

Hi Lucy ,
See inline below.

-----Original Message-----
From: Vivek Kumar [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 12:00 AM
To: Larry Kreeger (kreeger); Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Larry,

[Lucy] The given example is L3 VN. Both NVE1 and NVE2 are the member of the L3VN. NVE2 just decapsulates it and forward to TS2. No service needs to provide at egress. In the case, the VAP interface between TS2-NVE2 is VLAN interface, NVE2 learns the TS2 MAC address ahead and inserts the TS2 MAC as the dst MAC and its MAC as its src MAC prior sending to TS2. If the VAP is the IP interface, NVE2 forward the packet directly to TS2.
[Vivek] Thanks for explaining L3 services. How the lookup will happen at NVE2 for combined L2/L3 services ?

Lucy   

Regards,
Vivek

-----Original Message-----
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Wednesday, November 20, 2013 10:45 AM
To: Vivek Kumar; Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Vivek,

Thanks, Larry

---

From: Balaji P <[email-masked]>
Date: Thu, 21 Nov 2013 12:26:22 +0000

Lucy,

> IMHO treating L2 and L3 as separate services might be better options
> w.r.t NVE. So that based on NVE configuration for L2 and L3 Services will
> be applied to the traffic.
> [Lucy] The framework has L2 and L3 NVE service type, which apply to many
> cases. You can use them to construct various tenant network topologies.
> But we also have the case that an NVE may serve the distributed gateway
> function. In this case, the VAP may be associated to an L2 VNI on the
> NVE, the packet from a TS via the VAP may get L2 or L3 treatment on the
> NVE. IMO: it is necessary to differentiate this NVE case from other the
> two NVE services.

[Balaji] Supporting distributed gateway function definitely makes sense along with L2 and L3 services.

Copied from below text:

" A combined L2/L3 service presents no special considerations for NVO3, other than packets received from a tenant must be classified as to what type of service they are to be given before they can be processed."

[Balaji]when VAP is associated with L2/L3 distributed gateway. Based on the above text how do we classify this. As we can associate VAP to an L2 VNI on the NVE or L3 VNI  on NVE. Are we going to have any other VNI type for this?

Regards,
Balaji.P

> Sent: Wednesday, November 20, 2013 5:19 PM
> To: P Balaji-B37839; Erik Nordmark; Thomas Narten
> [Lucy] Do you mean the TS as network compliances w/ multiple VAPs and
> individual VAPs are associated to a L2 VNI or L3 VNI? If yes, agree too.
> 
> IMHO treating L2 and L3 as separate services might be better options
> w.r.t NVE. So that based on NVE configuration for L2 and L3 Services will
> be applied to the traffic.
> [Lucy] The framework has L2 and L3 NVE service type, which apply to many
> cases. You can use them to construct various tenant network topologies.
> But we also have the case that an NVE may serve the distributed gateway
> function. In this case, the VAP may be associated to an L2 VNI on the
> NVE, the packet from a TS via the VAP may get L2 or L3 treatment on the
> NVE. IMO: it is necessary to differentiate this NVE case from other the
> two NVE services.
> 
> Thanks,
> Lucy
> 
> Regards,
> Balaji.P
>

---

From: Lucy yong <[email-masked]>
Date: Thu, 21 Nov 2013 15:44:19 +0000

Please see [lucy1]

[Lucy] The given example is L3 VN. Both NVE1 and NVE2 are the member of the L3VN. NVE2 just decapsulates it and forward to TS2. No service needs to provide at egress. In the case, the VAP interface between TS2-NVE2 is VLAN interface, NVE2 learns the TS2 MAC address ahead and inserts the TS2 MAC as the dst MAC and its MAC as its src MAC prior sending to TS2. If the VAP is the IP interface, NVE2 forward the packet directly to TS2.
[Vivek] Thanks for explaining L3 services. How the lookup will happen at NVE2 for combined L2/L3 services ?
[Lucy1] No, as I said, your case is L3VN case. Only do IP lookup. In a case of combined L2/L3 service, it is always the ingress NVE performing the distributed gateway functions, i.e. the packed may be treated as L2 or L3. When sending the packet to the remote NVE, if using L2VN overlay, the remove NVE will do Ethernet lookup; if using L3VN overlay, the remove will do IP lookup. This is described in the draft-yong-nv03-nve draft. See figure 4 in the draft. 

Lucy   

Regards,
Vivek

-----Original Message-----
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Wednesday, November 20, 2013 10:45 AM
To: Vivek Kumar; Lucy yong; Erik Nordmark; Thomas Narten
Cc: [email-masked]; Linda Dunbar
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi Vivek,

Thanks, Larry

---

From: Lucy yong <[email-masked]>
Date: Thu, 21 Nov 2013 16:05:34 +0000

Copied from below text:

[Balaji]when VAP is associated with L2/L3 distributed gateway. Based on the above text how do we classify this. As we can associate VAP to an L2 VNI on the NVE or L3 VNI  on NVE. Are we going to have any other VNI type for this?
[Lucy] In combined L2/L3 NVE case, at least one VAP associates to an L2VNI on the NVE. In other words, the NVE is at least the member of an L2VNI.
Please note, the distributed gateway function is an option to be used on NVE. If a tenant does not allow inter-VN communication or want to use a gateway for inter-VN communication only, no need to use the combined L2/L3 NVE at all. 

Lucy

Regards,
Balaji.P

> w.r.t NVE. So that based on NVE configuration for L2 and L3 Services 
> will be applied to the traffic.
> [Lucy] The framework has L2 and L3 NVE service type, which apply to 
> many cases. You can use them to construct various tenant network topologies.
> But we also have the case that an NVE may serve the distributed 
> gateway function. In this case, the VAP may be associated to an L2 VNI 
> on the NVE, the packet from a TS via the VAP may get L2 or L3 
> treatment on the NVE. IMO: it is necessary to differentiate this NVE 
> case from other the two NVE services.
> 
> Thanks,
> Lucy
> 
> Regards,
> Balaji.P
> 
> > broadcast/multicast to broadcast to another VN, L3 router will 
> > forward the packet to another VN with the same DMAC. If the policy 
> > does not allow, L3 router will drop the packet.
> >
> > cases that will result in both bridging the packet out other L2 
> > ports, and forwarding the packet out other L3 (virtual) ports.)
> >
> > >            considerations for NVO3, other than packets received 
> > > from
> a
> > >            considerations for NVO3, other than packets received 
> > > from
> a
> > > > describing what we mean by a combined L2/L3 service. That is 
> > > > what
> > > my
> > > > is clear and simple, supporting it in NVO3 is not a problem. 
> > > > I.e., it's not really much different to provide a combined 
> > > > service than if you offer an L2 or L3 service. And the service 
> > > > semantics for combined  >
> > > > combined service is a separate matter. What I'd like to be able 
> > > > to
> > > do
> > > > here (if we need to say anything at all) is be able to say that 
> > > > if a distributed GW makes sense for L2 service, then having an 
> > > > L2  >
> > >  > A key point is that having a combined service is pretty much 
> > > the same  > as taking the two separate L2 and L3 services and 
> > > combining them into  > one implementation. There is nothing really "special"
> > > about this  that  > would complicate the overall architecture. 
> > > Where we can easily get  > into trouble is if we start defining a 
> > > combined service as has special  > cases that have to be dealt 
> > > with that don't appear when you have only  > L2 or only L3 service semantics.
> > >
> > > Thomas
> > >

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 22 Nov 2013 11:12:29 -0800

>>>             are to be given before they can be processed.

What is missing for me is a higher-level statement whether or not we see 
an NVE providing combined L2 and L3 service as being architecturally 
different that the non-overlay case of a bridge+router that provides 
combined service L2 and L3 today.

If we think it is just the same architecturally, then it would make 
sense to state that. If we think it is different, then I think we need 
more details that Thomas' text above.

FWIW the existing bridge+routers handle multicast conceptually as 
bridge-route-bridge. A received multicast packet might need to be 
bridged out other L2 ports in the same bridge domain. Then one copy of 
packet is passed to the L3 function, which does L3 multicast routing 
(check iIF, decrement ttl, determine oIFs). Finally, a given L3 oIF 
might correspond to a bridge domain i.e., multiple packets might need to 
be sent out different L2 ports for each oIF.

While that is a bit complex, it is a lot better if the NVO3 architecture 
is the same as existing combined bridge+router boxes.

And note that an existing combined bridge+router is architecturally 
consistent with separate bridges and a router where the bridges only do 
L2 and the router only does L3.

    Erik

---

From: "Black, David" <[email-masked]>
Date: Fri, 22 Nov 2013 19:40:12 -0500

> more details that Thomas' text above.

IMHO, it should be architecturally the same, and we should say so.  The
quoted text was intended to head in that direction, so an explicit statement
seems like a fine idea.   I think the touchstone for how L3 service is provided
in an L2/L3 service combination should be: "what would happen if there was no
network virtualization?"

Thanks,
--David

> From: nvo3 [mailto:[email-masked]] On Behalf Of Erik Nordmark
> Sent: Friday, November 22, 2013 2:12 PM
> To: Pankaj Garg; Vivek Kumar; Larry Kreeger (kreeger); Lucy yong; Thomas
> Narten
> > Wouldn't the decision to do L2 or L3 service be based on the inner frame
> fields i.e. destination MAC/IP in the inner frame? Similar to how
> switches/routers process packets i.e. based on frame's destination MAC and
> destination IP address (if present)?
> >

---

From: Lucy yong <[email-masked]>
Date: Mon, 25 Nov 2013 00:49:59 +0000

Here is my 2cent whether combined L2/L3 service is architecturally same or different from the non-overlay IRB case used today.

They are the same in term of the function feature, i.e. a router can provide bridging capability for intra VLAN traffic, and bridging capability for inter VLANs traffic via routing. 

But they have differences too. First, an NVE supports this feature in term of multi-tenancy environment, while a route does it in a single tenancy case (in one address space only). Second, an NVE has both VAP interfaces and overlay tunnel interfaces plus internal gw interface.  A router only has ports or L2 interfaces, which are equivalent to the VAPs on NVE case. (you point out this)

In a multi-tenancy environment, an NVE may be the member for multiple VNs. Some VN may have independent address spaces, some may share the same address space. An NVE needs to have a VN interconnection policy table which indicates which VNs can communicate and which VNs can't. Furthermore the policy may have finer granularity to a level. For example, between L2VNx and L2VNy, the policy is: for unicast traffic, L2VNx->L2VNy permit; L2VNx<-L2VNy via a firewall; for multicast traffic, L2VNx<->L2VNy not permit.

IMO: the inter-VN policy should be controlled at the VN level, not to the particular address (ACL level) or particular application (TCP, HTTP level) in the VNs. The latter belongs to the firewall function.    

Lucy

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Black, David
Sent: Friday, November 22, 2013 6:40 PM
To: Erik Nordmark
Cc: [email-masked]
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Writing as an individual, not co-author of draft-narten-nvo3-arch:

> What is missing for me is a higher-level statement whether or not we 
> see an NVE providing combined L2 and L3 service as being 
> architecturally different that the non-overlay case of a bridge+router 
> that provides combined service L2 and L3 today.
> 

IMHO, it should be architecturally the same, and we should say so.  The quoted text was intended to head in that direction, so an explicit statement
seems like a fine idea.   I think the touchstone for how L3 service is provided
in an L2/L3 service combination should be: "what would happen if there was no network virtualization?"

Thanks,
--David

> To: Pankaj Garg; Vivek Kumar; Larry Kreeger (kreeger); Lucy yong; 
> Thomas Narten
> switches/routers process packets i.e. based on frame's destination MAC 
> and destination IP address (if present)?
> >
> > IMHO, Thomas's original text (pasted below) describes this quite 
> > well and
> concisely.
> >
> >>>           <t>
> What is missing for me is a higher-level statement whether or not we 
> see an NVE providing combined L2 and L3 service as being 
> architecturally different that the non-overlay case of a bridge+router 
> that provides combined service L2 and L3 today.
> 
> might correspond to a bridge domain i.e., multiple packets might need 
> to be sent out different L2 ports for each oIF.
> 
> While that is a bit complex, it is a lot better if the NVO3 
> architecture is the same as existing combined bridge+router boxes.
>

---

From: Pankaj Garg <[email-masked]>
Date: Mon, 25 Nov 2013 04:13:56 +0000

+1 for this: ", it should be architecturally the same, and we should say so". 

IMHO, NVE behavior should  be same as existing "bridge+router" behavior, whether collocated or not.

> Sent: Saturday, November 23, 2013 6:10 AM
> To: Erik Nordmark
> Cc: [email-masked]

---

From: Jon Hudson <[email-masked]>
Date: Mon, 25 Nov 2013 00:30:37 -0800

+1 agreed

>seems like a fine idea.....

On Fri, Nov 22, 2013 at 4:40 PM, Black, David <[email-masked]> wrote:

> > > IMHO, Thomas's original text (pasted below) describes this quite well
> and
> > concisely.
> > >
> > >>>           <t>
> from a

-- 
"Do not lie. And do not do what you hate."

---

From: Balaji P <[email-masked]>
Date: Mon, 25 Nov 2013 09:54:17 +0000

+1 for David comments.

Also if we can add some text w.r.t use cases of handling of L2/L3 service combination will be good.

Regards,
Balaji.P

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Mon, 25 Nov 2013 12:25:01 +0000

I agree with David.

Marc 

> Sent: Saturday, November 23, 2013 1:40 AM
> To: Erik Nordmark
> Cc: [email-masked]
> IMHO, it should be architecturally the same, and we should 
> say so.  The quoted text was intended to head in that 
> direction, so an explicit statement
> > > Wouldn't the decision to do L2 or L3 service be based on 
> the inner 
> > > frame
> > consistent with separate bridges and a router where the 
> bridges only 
> > do

---

From: "Black, David" <[email-masked]>
Date: Mon, 25 Nov 2013 09:36:02 -0500

Lucy,

> Here is my 2cent whether combined L2/L3 service is architecturally same or
> different from the non-overlay IRB case used today.
> 
> They are the same in term of the function feature, i.e. a router can provide
> bridging capability for intra VLAN traffic, and bridging capability for inter
> VLANs traffic via routing.
> 
> But they have differences too. First, an NVE supports this feature in term of
> multi-tenancy environment, while a route does it in a single tenancy case (in
> one address space only).

I firmly disagree.  VLANs can be used for multi-tenancy, with the result that
there can be multiple instances of the router function in the same physical
network node in multiple IP address spaces.

> Second, an NVE has both VAP interfaces and overlay
> tunnel interfaces plus internal gw interface.  A router only has ports or L2
> interfaces, which are equivalent to the VAPs on NVE case. (you point out this)

I think this is over-simplified.  A router has interfaces that correspond to both
overlay tunnel interfaces and its internal gateway (not sure whether this was
intended to be the gateway facing end systems, or the interface that "default"
is pointed at, if applicable, but the analogy holds for both).  For a non-overlay
environment, these interfaces are not encapsulated, but in general, a router
has L2/link connectivity to other routers, not just end systems.

> In a multi-tenancy environment, an NVE may be the member for multiple VNs.

And a physical routing implementation may provide logically distinct routing for
multiple VLANs.

> Some VN may have independent address spaces, some may share the same address
> space. An NVE needs to have a VN interconnection policy table which indicates
> which VNs can communicate and which VNs can't

Likewise for routing and VLANs.

> Furthermore the policy may have
> finer granularity to a level. For example, between L2VNx and L2VNy, the policy
> is: for unicast traffic, L2VNx->L2VNy permit; L2VNx<-L2VNy via a firewall; for
> multicast traffic, L2VNx<->L2VNy not permit.
>
> IMO: the inter-VN policy should be controlled at the VN level, not to the
> particular address (ACL level) or particular application (TCP, HTTP level) in
> the VNs. The latter belongs to the firewall function.

And also likewise for routing and VLANs.

Thanks,
--David

> Sent: Sunday, November 24, 2013 7:50 PM
> To: Black, David; Erik Nordmark
> Cc: [email-masked]
> Here is my 2cent whether combined L2/L3 service is architecturally same or
> different from the non-overlay IRB case used today.
> 
> They are the same in term of the function feature, i.e. a router can provide
> bridging capability for intra VLAN traffic, and bridging capability for inter
> VLANs traffic via routing.
> 
> But they have differences too. First, an NVE supports this feature in term of
> multi-tenancy environment, while a route does it in a single tenancy case (in
> one address space only). Second, an NVE has both VAP interfaces and overlay
> tunnel interfaces plus internal gw interface.  A router only has ports or L2
> interfaces, which are equivalent to the VAPs on NVE case. (you point out this)
> 
> In a multi-tenancy environment, an NVE may be the member for multiple VNs.
> Some VN may have independent address spaces, some may share the same address
> space. An NVE needs to have a VN interconnection policy table which indicates
> which VNs can communicate and which VNs can't. Furthermore the policy may have
> finer granularity to a level. For example, between L2VNx and L2VNy, the policy
> is: for unicast traffic, L2VNx->L2VNy permit; L2VNx<-L2VNy via a firewall; for
> multicast traffic, L2VNx<->L2VNy not permit.
> 
> IMO: the inter-VN policy should be controlled at the VN level, not to the
> particular address (ACL level) or particular application (TCP, HTTP level) in
> the VNs. The latter belongs to the firewall function.
> 
> Lucy
> 
> 
> 
> 
> 
> IMHO, it should be architecturally the same, and we should say so.  The quoted
> text was intended to head in that direction, so an explicit statement

---

From: Lucy yong <[email-masked]>
Date: Mon, 25 Nov 2013 16:11:43 +0000

Hi David,

Please see inline.

-----Original Message-----
From: Black, David [mailto:[email-masked]] 
Sent: Monday, November 25, 2013 8:36 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

> Here is my 2cent whether combined L2/L3 service is architecturally 
> same or different from the non-overlay IRB case used today.
> 
> They are the same in term of the function feature, i.e. a router can 
> provide bridging capability for intra VLAN traffic, and bridging 
> capability for inter VLANs traffic via routing.
> 
> But they have differences too. First, an NVE supports this feature in 
> term of multi-tenancy environment, while a route does it in a single 
> tenancy case (in one address space only).

I firmly disagree.  VLANs can be used for multi-tenancy, with the result that there can be multiple instances of the router function in the same physical network node in multiple IP address spaces.
[Lucy] Yes, VLANs can be used for multi-tenancy. But, in this context, we talk about a router device with bridge capability (i.e. IRB), Isn't such device operated under one L3 address space, including physical network address space and source/destination address space? In other words, endpoint and physical network are on the same address space. In traditional DC environment, even VLAN can provide virtual networks, MAC address is uniquely assigned to physical device, so it is effectively only one MAC address space regardless whether VLAN is used or not. VLAN provides a virtual private network, it was not originally intended to provide independent address spaces.  

> Second, an NVE has both VAP interfaces and overlay tunnel interfaces 
> plus internal gw interface.  A router only has ports or L2 interfaces, 
> which are equivalent to the VAPs on NVE case. (you point out this)

I think this is over-simplified.  A router has interfaces that correspond to both overlay tunnel interfaces and its internal gateway (not sure whether this was intended to be the gateway facing end systems, or the interface that "default"
is pointed at, if applicable, but the analogy holds for both).  For a non-overlay environment, these interfaces are not encapsulated, but in general, a router has L2/link connectivity to other routers, not just end systems.
[Lucy] I may over-simplify the router to focus on this case only. My point is that the router device in this case typically handles one L3 address space and one MAC address space in the reality. 

Lucy

> In a multi-tenancy environment, an NVE may be the member for multiple VNs.

And a physical routing implementation may provide logically distinct routing for multiple VLANs.

> Some VN may have independent address spaces, some may share the same 
> address space. An NVE needs to have a VN interconnection policy table 
> which indicates which VNs can communicate and which VNs can't

> is: for unicast traffic, L2VNx->L2VNy permit; L2VNx<-L2VNy via a 
> firewall; for multicast traffic, L2VNx<->L2VNy not permit.
>
> IMO: the inter-VN policy should be controlled at the VN level, not to 
> the particular address (ACL level) or particular application (TCP, 
> HTTP level) in the VNs. The latter belongs to the firewall function.

Thanks,
--David

> Here is my 2cent whether combined L2/L3 service is architecturally 
> same or different from the non-overlay IRB case used today.
> 
> They are the same in term of the function feature, i.e. a router can 
> provide bridging capability for intra VLAN traffic, and bridging 
> capability for inter VLANs traffic via routing.
> 
> But they have differences too. First, an NVE supports this feature in 
> term of multi-tenancy environment, while a route does it in a single 
> tenancy case (in one address space only). Second, an NVE has both VAP 
> interfaces and overlay tunnel interfaces plus internal gw interface.  
> A router only has ports or L2 interfaces, which are equivalent to the 
> VAPs on NVE case. (you point out this)
> 
> Some VN may have independent address spaces, some may share the same 
> address space. An NVE needs to have a VN interconnection policy table 
> which indicates which VNs can communicate and which VNs can't. 
> Furthermore the policy may have finer granularity to a level. For 
> example, between L2VNx and L2VNy, the policy
> is: for unicast traffic, L2VNx->L2VNy permit; L2VNx<-L2VNy via a 
> firewall; for multicast traffic, L2VNx<->L2VNy not permit.
> 
> IMO: the inter-VN policy should be controlled at the VN level, not to 
> the particular address (ACL level) or particular application (TCP, 
> HTTP level) in the VNs. The latter belongs to the firewall function.
> 
> Lucy
> 
> 
> 
> 
> 
> > architecturally different that the non-overlay case of a 
> > bridge+router that provides combined service L2 and L3 today.
> >
> > sense to state that. If we think it is different, then I think we 
> > need more details that Thomas' text above.
> 
> IMHO, it should be architecturally the same, and we should say so.  
> The quoted text was intended to head in that direction, so an explicit statement
> > switches/routers process packets i.e. based on frame's destination 
> > MAC and destination IP address (if present)?
> > >
> > architecturally different that the non-overlay case of a 
> > bridge+router that provides combined service L2 and L3 today.
> >
> > sense to state that. If we think it is different, then I think we 
> > need more details that Thomas' text above.
> >
> > bridged out other L2 ports in the same bridge domain. Then one copy 
> > of packet is passed to the L3 function, which does L3 multicast 
> > routing (check iIF, decrement ttl, determine oIFs). Finally, a given 
> > L3 oIF might correspond to a bridge domain i.e., multiple packets 
> > might need to be sent out different L2 ports for each oIF.
> >

---

From: "Black, David" <[email-masked]>
Date: Mon, 25 Nov 2013 12:55:45 -0500

Lucy,

--
--
> [Lucy] Yes, VLANs can be used for multi-tenancy. But, in this context, we talk
> about a router device with bridge capability (i.e. IRB), Isn't such device
> operated under one L3 address space, including physical network address space
> and source/destination address space?

No, we aren't talking about a single "router device" - when I wrote "multiple
instances of the router function", I meant a device that can operate multiple
instances of the router function that can be in separate source/destination
address spaces (e.g., I expect to see multiple independent instances of the
10.0.0.0/8 address block in a multi-tenant data center).  In addition, an
NVE has to deal with the underlay address space.

I view that underlay address space as secondary - the routing function makes
an IP routing decision about where to forward a packet.  That decision is about
what "logical port" to emit the packet on ("logical interface" is a better
term in this context) - the encapsulation and the underlay address are functional
consequences of that forwarding decision.

> [Lucy] I may over-simplify the router to focus on this case only. My point is
> that the router device in this case typically handles one L3 address space and
> one MAC address space in the reality.

See above on "multiple instances" of the "router".

Thanks,
--David

> Sent: Monday, November 25, 2013 11:12 AM
> To: Black, David
> Cc: [email-masked]
> [Lucy] Yes, VLANs can be used for multi-tenancy. But, in this context, we talk
> about a router device with bridge capability (i.e. IRB), Isn't such device
> operated under one L3 address space, including physical network address space
> and source/destination address space? In other words, endpoint and physical
> network are on the same address space. In traditional DC environment, even
> VLAN can provide virtual networks, MAC address is uniquely assigned to
> physical device, so it is effectively only one MAC address space regardless
> whether VLAN is used or not. VLAN provides a virtual private network, it was
> not originally intended to provide independent address spaces.
> 
> I think this is over-simplified.  A router has interfaces that correspond to
> both overlay tunnel interfaces and its internal gateway (not sure whether this
> was intended to be the gateway facing end systems, or the interface that
> "default"
> is pointed at, if applicable, but the analogy holds for both).  For a non-
> overlay environment, these interfaces are not encapsulated, but in general, a
> router has L2/link connectivity to other routers, not just end systems.
> [Lucy] I may over-simplify the router to focus on this case only. My point is
> that the router device in this case typically handles one L3 address space and
> one MAC address space in the reality.
> 
> Lucy
>

---

From: Lucy yong <[email-masked]>
Date: Mon, 25 Nov 2013 18:16:16 +0000

David,

I know that NVO3 architecture targets multi-tenant environment from day one. 

Eric initially question is if combined L3/L2 service in NVO3 is the architecturally same as the today's router with bridging capability, i.e. IRB used in a physical network today. This is the router I referenced in my mail.

I think that my view is the same as yours.

Thanks,
Lucy

-----Original Message-----
From: Black, David [mailto:[email-masked]] 
Sent: Monday, November 25, 2013 11:56 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

> > But they have differences too. First, an NVE supports this feature 
> > in term of multi-tenancy environment, while a route does it in a 
> > single tenancy case (in one address space only).
> 
> I firmly disagree.  VLANs can be used for multi-tenancy, with the 
> result that there can be multiple instances of the router function in 
> the same physical network node in multiple IP address spaces.
--
> I firmly disagree.  VLANs can be used for multi-tenancy, with the 
> result that there can be multiple instances of the router function in 
> the same physical network node in multiple IP address spaces.
--
> [Lucy] Yes, VLANs can be used for multi-tenancy. But, in this context, 
> we talk about a router device with bridge capability (i.e. IRB), Isn't 
> such device operated under one L3 address space, including physical 
> network address space and source/destination address space?

No, we aren't talking about a single "router device" - when I wrote "multiple instances of the router function", I meant a device that can operate multiple instances of the router function that can be in separate source/destination address spaces (e.g., I expect to see multiple independent instances of the
10.0.0.0/8 address block in a multi-tenant data center).  In addition, an NVE has to deal with the underlay address space.

I view that underlay address space as secondary - the routing function makes an IP routing decision about where to forward a packet.  That decision is about what "logical port" to emit the packet on ("logical interface" is a better term in this context) - the encapsulation and the underlay address are functional consequences of that forwarding decision.

> [Lucy] I may over-simplify the router to focus on this case only. My 
> point is that the router device in this case typically handles one L3 
> address space and one MAC address space in the reality.

Thanks,
--David

> > But they have differences too. First, an NVE supports this feature 
> > in term of multi-tenancy environment, while a route does it in a 
> > single tenancy case (in one address space only).
> 
> I firmly disagree.  VLANs can be used for multi-tenancy, with the 
> result that there can be multiple instances of the router function in 
> the same physical network node in multiple IP address spaces.
> [Lucy] Yes, VLANs can be used for multi-tenancy. But, in this context, 
> we talk about a router device with bridge capability (i.e. IRB), Isn't 
> such device operated under one L3 address space, including physical 
> network address space and source/destination address space? In other 
> words, endpoint and physical network are on the same address space. In 
> traditional DC environment, even VLAN can provide virtual networks, 
> MAC address is uniquely assigned to physical device, so it is 
> effectively only one MAC address space regardless whether VLAN is used 
> or not. VLAN provides a virtual private network, it was not originally intended to provide independent address spaces.
> 
> > plus internal gw interface.  A router only has ports or L2 
> I think this is over-simplified.  A router has interfaces that 
> correspond to both overlay tunnel interfaces and its internal gateway 
> (not sure whether this was intended to be the gateway facing end 
> systems, or the interface that "default"
> is pointed at, if applicable, but the analogy holds for both).  For a 
> non- overlay environment, these interfaces are not encapsulated, but 
> in general, a router has L2/link connectivity to other routers, not just end systems.
> [Lucy] I may over-simplify the router to focus on this case only. My 
> point is that the router device in this case typically handles one L3 
> address space and one MAC address space in the reality.
> 
> Lucy
> 
> > address space. An NVE needs to have a VN interconnection policy 
> > table which indicates which VNs can communicate and which VNs can't
> 
> > IMO: the inter-VN policy should be controlled at the VN level, not 
> > to the particular address (ACL level) or particular application 
> > (TCP, HTTP level) in the VNs. The latter belongs to the firewall function.
> 
> > But they have differences too. First, an NVE supports this feature 
> > in term of multi-tenancy environment, while a route does it in a 
> > single tenancy case (in one address space only). Second, an NVE has 
> > both VAP interfaces and overlay tunnel interfaces plus internal gw interface.
> > A router only has ports or L2 interfaces, which are equivalent to 
> > the VAPs on NVE case. (you point out this)
> >
> > address space. An NVE needs to have a VN interconnection policy 
> > table which indicates which VNs can communicate and which VNs can't.
> > IMO: the inter-VN policy should be controlled at the VN level, not 
> > to the particular address (ACL level) or particular application 
> > (TCP, HTTP level) in the VNs. The latter belongs to the firewall function.
> >
> > Lucy
> >
> >
> >
> >
> >
> > > What is missing for me is a higher-level statement whether or not 
> > > we see an NVE providing combined L2 and L3 service as being 
> > > If we think it is just the same architecturally, then it would 
> > > make sense to state that. If we think it is different, then I 
> > > think we need more details that Thomas' text above.
> >
> > The quoted text was intended to head in that direction, so an 
> > explicit
> statement
> > > >>>             considerations for NVO3, other than packets 
> > > >>> received from
> a
> > > What is missing for me is a higher-level statement whether or not 
> > > we see an NVE providing combined L2 and L3 service as being 
> > > If we think it is just the same architecturally, then it would 
> > > make sense to state that. If we think it is different, then I 
> > > think we need more details that Thomas' text above.
> > >
> > > bridged out other L2 ports in the same bridge domain. Then one 
> > > copy of packet is passed to the L3 function, which does L3 
> > > multicast routing (check iIF, decrement ttl, determine oIFs). 
> > > Finally, a given
> > > And note that an existing combined bridge+router is 
> > > architecturally consistent with separate bridges and a router 
> > > where the bridges only do

---

From: David Allan I <[email-masked]>
Date: Tue, 26 Nov 2013 00:29:46 +0000

Lucy,

Any non-interconnect of tenant networks in a common routing implementation will require lots of ACLs which can be gotten wrong, and would not permit reuse of private addressing by multiple tenants sharing a router FDB....and could be messed up by duplicate MAC addresses.

IMO it is a bad idea, it places a lot of constraints on virtualization (e.g. it can only work if..and if... and if...etc.) so it becomes somewhat half baked. 

If a tenant requires multiple interconnected subnets they need their own VRF/VR implementation, own ARP cache etc to obviate the potential problems. Why would we WANT to allow otherwise?

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Monday, November 25, 2013 8:12 AM
To: Black, David
Cc: [email-masked]
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi David,

Please see inline.

-----Original Message-----
From: Black, David [mailto:[email-masked]]
Sent: Monday, November 25, 2013 8:36 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

Lucy

Thanks,
--David

---

From: Lucy yong <[email-masked]>
Date: Tue, 26 Nov 2013 14:51:11 +0000

David,

Please see inline.

-----Original Message-----
From: David Allan I [mailto:[email-masked]] 
Sent: Monday, November 25, 2013 6:30 PM
To: Lucy yong; Black, David
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

Any non-interconnect of tenant networks in a common routing implementation will require lots of ACLs which can be gotten wrong, and would not permit reuse of private addressing by multiple tenants sharing a router FDB....and could be messed up by duplicate MAC addresses.
[Lucy] agreed. SHOULD not use ACL doing that. 

If a tenant requires multiple interconnected subnets they need their own VRF/VR implementation, own ARP cache etc to obviate the potential problems. Why would we WANT to allow otherwise?
[Lucy] Agreed. But NVO3 goal is the solution in multi-tenancy environment. Network virtualization needs to figure out an efficient and secure way to implement it. Software can do a lot. :) 

Thanks,
Lucy

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Monday, November 25, 2013 8:12 AM
To: Black, David
Cc: [email-masked]
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi David,

Please see inline.

-----Original Message-----
From: Black, David [mailto:[email-masked]]
Sent: Monday, November 25, 2013 8:36 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

Lucy

Thanks,
--David

---

From: David Allan I <[email-masked]>
Date: Tue, 26 Nov 2013 21:31:19 +0000

HI Lucy:

Your point completely escapes me. Could you clarify?

Cheers
Dave

-----Original Message-----
From: Lucy yong [mailto:[email-masked]] 
Sent: Tuesday, November 26, 2013 6:51 AM
To: David Allan I; Black, David
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

David,

Please see inline.

-----Original Message-----
From: David Allan I [mailto:[email-masked]]
Sent: Monday, November 25, 2013 6:30 PM
To: Lucy yong; Black, David
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

Thanks,
Lucy

My 2 cents
Dave

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Monday, November 25, 2013 8:12 AM
To: Black, David
Cc: [email-masked]
Subject: Re: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Hi David,

Please see inline.

-----Original Message-----
From: Black, David [mailto:[email-masked]]
Sent: Monday, November 25, 2013 8:36 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Fwd: Arch: proposed text for Combined L2/L3 Service

Lucy,

Lucy

Thanks,
--David