Subject: [nvo3] Applicability of RFC 5512 to NVO3?

From: Shane Amante <[email-masked]>
Date: Fri, 17 Feb 2012 23:27:40 -0700

I am curious to know if the following RFC has been looked at to see if it's applicable to the NVO3 solution space. Furthermore, if it was looked at, I'd be curious to know why it was insufficient?

http://tools.ietf.org/html/rfc5512

Thanks,

-shane

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 18 Feb 2012 11:35:41 +0100

Hi Shane,

5512 would require each host to support BGP.

That for amount of information scaling would automatically mean that 
each host in the data center would need to also support RTC (rfc4684).

Session scaling is another challenge if we follow Igor's numbers.

And I am not sure that the p2mp information distribution nature of BGP 
fits well to the set of requirements here. It seems that host to VM 
controller/orchestration/ect is much more p2p in it's nature. From 
looking at our data centers I see the common user's VMs residing on very 
little number of hosts.

Even if one would go that path (and maybe we would restart talks about 
bgp-lite .. what may be regardless useful for PE-CE bgp scaling in other 
areas) which BGP implementation you would recommend to use on the hosts ?

Cheers,
R.

> I am curious to know if the following RFC has been looked at to see
> if it's applicable to the NVO3 solution space. Furthermore, if it was
> looked at, I'd be curious to know why it was insufficient?
>
> -shane _______________________________________________ nvo3 mailing
> list [email-masked] https://www.ietf.org/mailman/listinfo/nvo3
>
>

---

From: Shane Amante <[email-masked]>
Date: Sat, 18 Feb 2012 09:35:28 -0700

Hi Robert,

1) I was viewing RFC 5512 as something that may be able to be run only down to and on ToR switches, not (necessarily/always) down to individual Hypervisors, specifically for the scale reasons you cite.
2) With respect to the latter point, I can potentially see something like draft-marques-l3vpn-end-system-03 as being complementary to RFC 5512. Specifically, draft-marques-l3vpn-end-system-03 would run on/in a Hypervisor and talk to a ToR (running RFC 5512), as appropriate.

Regardless, at a high level, my question was really meant to ask the more broad question of whether RFC 5512 had been reviewed by the authors/proponents of NVO3 to see if it was applicable and, if not, why not. When I reviewed the Section 6 of the NVO3 draft, (and attended the NVO3 discussion in Taipei), I had not seen it mentioned (even though there are several other protocols/mechanisms discussed), which is what prompted my question.
https://tools.ietf.org/html/draft-narten-nvo3-overlay-problem-statement-01#section-6
... or, said differently, could the NVO3 effort go *faster* (and, hopefully avoid inventing a new, almost identical signaling protocol) if it was able to leverage RFC 5512 for its signaling control plane[1] and, perhaps, focus mostly on (for example) adopting/refining/etc. (something like) draft-marques-l3vpn-end-system, as I mention in #2 above (and, others have mentioned on the list previously).

In summary, I was attempting to help narrow the scope of NVO3, by re-using /if possible/ what we've already developed and, secondarily, focus on the new things that don't exist.

-shane

[1] As you mention, BGP may be challenged from a scaling perspective, wrt to NVO3. Then again, the same arguments were made about BGP back during the early days of the L2VPN & L3VPN, when those WG's were just getting off the ground and look where the industry is now.

On Feb 18, 2012, at 3:35 AM, Robert Raszuk wrote:
> That for amount of information scaling would automatically mean that each host in the data center would need to also support RTC (rfc4684).
> 
> And I am not sure that the p2mp information distribution nature of BGP fits well to the set of requirements here. It seems that host to VM controller/orchestration/ect is much more p2p in it's nature. From looking at our data centers I see the common user's VMs residing on very little number of hosts.
> 
> Even if one would go that path (and maybe we would restart talks about bgp-lite .. what may be regardless useful for PE-CE bgp scaling in other areas) which BGP implementation you would recommend to use on the hosts ?
> 
> Cheers,
> R.
>

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 18 Feb 2012 18:24:02 +0100

Hi Shane,

> [1] As you mention, BGP may be challenged from a scaling
> perspective,

More fundamentally I personally do not believe p2mp signalling of any
sort is a good fit for very sparse VPNs model (and I think the ratio of
number of given VPN instance on the hosts to the total number of hosts
proves that we are dealing here with very very sparse VPNs environment).

> wrt to NVO3. Then again, the same arguments were made about BGP back
> during the early days of the L2VPN & L3VPN, when those WG's were
> just getting off the ground and look where the industry is now.

I am looking and not seeing (except RTC) much protocol improvements
being proposed in IETF to the above.

Consider EVPN - within given VPN you will get 1000s of C-MACs send to
all PEs where for actual data plane conversations you will need maybe
just few % of those. I know RAM is cheap, but it is not about RAM. The
more information you send around the worse are scaling properties of the
entire system ... think refreshes, GR, NSR, peer bringup, reflector
scaling etc..

Thx,
R.

PS.

> 2) With respect to the latter point, I can potentially see something
> like draft-marques-l3vpn-end-system-03 as being complementary to RFC
> 5512. Specifically, draft-marques-l3vpn-end-system-03 would run on/in
> a Hypervisor and talk to a ToR (running RFC 5512), as appropriate.

The thing which I like particularly from draft-marques-l3vpn-end-system 
is that it decouples control plane from data plane and defines a 
protocol to distribute information between the two. How one constructs a 
control plane to me is a secondary thing. One could model it as 
traditional VPNvX and someone else design his own distribution.

5512 would helps to negotiate outer transport tunnels encapsulation 
between PEs .. I am not really seeing this as necessity in data 
center/cloud applications.

---

From: Larry Kreeger <[email-masked]>
Date: Mon, 20 Feb 2012 00:35:29 -0800

Hi Shane,

On 2/18/12 8:35 AM, "Shane Amante" <[email-masked]> wrote:

> Hi Robert,
> 
> 1) I was viewing RFC 5512 as something that may be able to be run only down to
> and on ToR switches, not (necessarily/always) down to individual Hypervisors,
> specifically for the scale reasons you cite.

While the overlay solution should allow for the ToR switches to be the
encap/decap points, the other requirement is that it must be possible for
the overlays to terminate in the Virtual Switch inside the hypervisor.  We
certainly wouldn't want to use two different protocols, with one for
hypervisors and one for ToR switches.  In fact, I believe it quite likely
that the same DC will have overlays terminating in both hypervisors and in
physical network switches.

> 2) With respect to the latter point, I can potentially see something like
> draft-marques-l3vpn-end-system-03 as being complementary to RFC 5512.
> Specifically, draft-marques-l3vpn-end-system-03 would run on/in a Hypervisor
> and talk to a ToR (running RFC 5512), as appropriate.
> 
> Regardless, at a high level, my question was really meant to ask the more
> broad question of whether RFC 5512 had been reviewed by the authors/proponents
> of NVO3 to see if it was applicable and, if not, why not. When I reviewed the
> Section 6 of the NVO3 draft, (and attended the NVO3 discussion in Taipei), I
> had not seen it mentioned (even though there are several other
> protocols/mechanisms discussed), which is what prompted my question.
> ... or, said differently, could the NVO3 effort go *faster* (and, hopefully
> avoid inventing a new, almost identical signaling protocol) if it was able to
> leverage RFC 5512 for its signaling control plane[1] and, perhaps, focus
> mostly on (for example) adopting/refining/etc. (something like)
> draft-marques-l3vpn-end-system, as I mention in #2 above (and, others have
> mentioned on the list previously).
> 
> In summary, I was attempting to help narrow the scope of NVO3, by re-using /if
> possible/ what we've already developed and, secondarily, focus on the new
> things that don't exist.
> 
> -shane
> 
> [1] As you mention, BGP may be challenged from a scaling perspective, wrt to
> NVO3. Then again, the same arguments were made about BGP back during the early
> days of the L2VPN & L3VPN, when those WG's were just getting off the ground
> and look where the industry is now.
> 
> 
>> That for amount of information scaling would automatically mean that each
>> host in the data center would need to also support RTC (rfc4684).
>> 
>> And I am not sure that the p2mp information distribution nature of BGP fits
>> well to the set of requirements here. It seems that host to VM
>> controller/orchestration/ect is much more p2p in it's nature. From looking at
>> our data centers I see the common user's VMs residing on very little number
>> of hosts.
>> 
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Shane Amante <[email-masked]>
Date: Tue, 21 Feb 2012 00:22:28 -0700

Hi Larry,

> physical network switches.

Please clarify what you mean above, wrt data-plane encapsulation/de-encapsulation vs. control-plane protocols.

RFC 5512 is strictly a control-plane signaling protocol, which can signal any combination of: GRE/IP, IP/IP or L2TPv3/IP for data-plane encapsulation.

-shane

---

From: <[email-masked]>
Date: Tue, 21 Feb 2012 12:00:17 -0500

Hi Shane,

While I'm not Larry, there is a quick explanation for your query ...

> vs. control-plane protocols.

If a data plane "learning" approach is not being used (compare items 3 and 4 in the
proposed charter), then the encapsulation point has to run enough of the control
plane protocol to enable mapping of "inner" addresses to "outer" addresses for
encapsulation.

Looking beyond that ...

At the end of the day, it may be that the control plane within each encapsulation will
wind up being specific to that encapsulation with the crucial effort for interoperability
being the change communication protocol between end systems (e.g., hypervisors) and the
network (e.g., the use of XMPP proposed in the marques-l3vpn-end-system draft).

Both NVGRE and VXLAN are currently based on data plane "learning" and hence should
benefit from a more scalable protocol that isn't based on data plane "learning".  OTOH,
BGP scaling is much better, so for an MPLS VPN that's already using BGP, I'd just as
soon leave BGP alone until there's good reason to believe that something needs improvement
(and if that BGP implementation wants to use RFC 5512, I see no reason to argue).

OTOOH, I share Shane's reservation/caution about a blanket requirement for hypervisors
and/or hypervisor softswitches to implement some subset of BGP (and the use of XMPP in
the marques-l3vpn-end-system draft appears to reflect a similar reservation/caution).

Speaking only for myself, BGP would not have been my first choice for a control plane
for NVGRE and VXLAN, but (like XMPP for end system communication), it should be kept
in consideration.

Thanks,
--David

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Shane Amante
> Sent: Tuesday, February 21, 2012 2:22 AM
> To: Larry Kreeger
> Cc: [email-masked]; Robert Raszuk
> Subject: Re: [nvo3] Applicability of RFC 5512 to NVO3?
> 
> Hi Larry,
> 
> RFC 5512 is strictly a control-plane signaling protocol, which can signal any combination of: GRE/IP,
> IP/IP or L2TPv3/IP for data-plane encapsulation.
> 
> -shane
> 
> 
>

---

From: Larry Kreeger <[email-masked]>
Date: Tue, 21 Feb 2012 19:54:26 -0800

On 2/20/12 11:22 PM, "Shane Amante" <[email-masked]> wrote:

> Please clarify what you mean above, wrt data-plane
> encapsulation/de-encapsulation vs. control-plane protocols.
> 
> RFC 5512 is strictly a control-plane signaling protocol, which can signal any
> combination of: GRE/IP, IP/IP or L2TPv3/IP for data-plane encapsulation.

An Overlay Border Point (OBP), as it is called in
draft-kreeger-nvo3-overlay-cp-00 performs the encap/decap function.  In
order to do this it needs a table mapping inner Virtual Network addresses to
outer Underlying Network addresses.  The document discusses the requirements
for a control plane protocol to populate this mapping table.  I understood
that you were suggesting using RFC5512 as this protocol, but you mentioned
doing it only on ToR and not hypervisors; However, it would need to also run
on hypervisors since they will likely be the main place OBPs would exist.

 - Larry
>>> https://tools.ietf.org/html/draft-narten-nvo3-overlay-problem-statement-01#s
>>> ec
>>> tion-6