Subject: [armd] some ideas on ARP optimization for one broadcast domain spanning across multiple sites

From: Linda Dunbar <[email-masked]>
Date: Mon, 13 Sep 2010 17:44:56 -0500

Dear All, 

Just want to check your feedback on some ideas to reduce the ARP (IPv4)
storm within one broadcast domain. 

Many hosts frequently send out gratuitous ARP for all other hosts in the
broadcast domain to be aware of correct IP/MAC-VLAN mapping for the host.
When one subnet is across multiple sites (or across many top of rack
switches), there are several proposals on sending consolidated IP/MAC-VLAN
among sites. Cisco's OTV suggests using IS/IS to send one site's IP/MAC-VLAN
mapping to all other sites which belong to the same subnet. Juniper's
Ethernet VPN suggests using BGP to send one site's IP/MAC-VLAN mapping to
all other sites which belong to the same subnet. 

What if each site keeps their hosts' MAC address as private? The site
gateway switches (or Top of Rack switches) can proxy the ARP requests from
other sites with its own MAC address. By doing so, site gateway switch (or
Top of Rack Switches) only need to send a group of IP addresses together
with its own MAC to all other sites. Since IP addresses can be summarized,
the amount of information to be sent to other sites is much smaller. Even if
after many rounds of moving around and the IP addresses in one site can't be
summarized anymore, you still get the saving by not sending all the MAC
addresses. 

If Layer 2 switches enable Source Address learning only from ports towards
server, but disable learning from ports towards network side, then all hosts
can still use their individual SA for data frames. By doing so, Source
Addresses of data frames are all original, which will not break DHCP
behavior. 

Your comments are appreciated. I think writing a draft on this idea is
worthwhile. 

Regards, Linda Dunbar

---

From: Ron Cohen <[email-masked]>
Date: Sun, 19 Sep 2010 13:31:20 +0200

Hi Linda,

My comments:

1. Need to take into account more than one gateway per site, for both
resiliency and load balancing.

2. I'm not convinced that the complexity of summarizing parts-of subnets is
worth the potential bandwidth saving. We need to look at the entire protocol
(including updates upon move etc.) to determine whether summarizing doesn't
consume more bandwidth due to triggered updates.

3. It is not clear from the proposal how each gateway learns the IP
addresses of connected hosts. That is, if a new VM boots up, how is the
gateway notified/learns of its IP/MAC assignment. How does aging work, etc.

4. Please explain which DHCP concerns you refer to and why the proposal of
disabling source learning from uplink ports solves it.

Thank you
Ron

On Tue, Sep 14, 2010 at 12:44 AM, Linda Dunbar <[email-masked]> wrote:

> among sites. Cisco’s OTV suggests using IS/IS to send one site’s IP/MAC-VLAN
> mapping to all other sites which belong to the same subnet. Juniper’s
> Ethernet VPN suggests using BGP to send one site’s IP/MAC-VLAN mapping to
> What if each site keeps their hosts’ MAC address as private? The site
> after many rounds of moving around and the IP addresses in one site can’t be
> _______________________________________________
> armd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/armd
>
>

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 20 Sep 2010 11:57:37 -0500

Ron, 

Thank you for the suggestions. 

Detailed answers to your questions are inserted below:

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of Ron
Cohen
Sent: Sunday, September 19, 2010 6:31 AM
To: Linda Dunbar
Cc: [email-masked]
Subject: Re: [armd] some ideas on ARP optimization for one broadcast domain
spanning across multiple sites

Hi Linda,

My comments:

[Linda] Yes, that is a good suggestion. 

[Linda] Bandwidth saving is not the main issue. We are suggesting the
possibility of having private MAC addresses. Just like private IP address,
each domain (or all hosts under one Top of Rack switch) have a handful of
"public" MAC addresses which are used by core or other TORs. Under this
scenario, the MAC addresses to be exposed to other domains are the public
MAC addresses. The private MAC addresses are solely relevant within the
domain. 

This is just a primitive idea. We can discuss merit of doing this. 

[Linda]  When a VM boots up, the very first thing most VMs do is to send a
gratuitous ARP to announce them. 

The latest IEEE802.1Qag will require VMs to send a port profile to the
directly attached switch. 

[Linda] At the ARMD bar BOF in 78th IETF, researchers from Cambridge
University presented MOOSE architecture, which requires the first (or
directly attached) switch to replace both SA and DA. Internet AD Ralph Droms
commented that the host's SA needs to stay with the data frame, otherwise,
DHCP will be broken. 

Thank you

Ron

Dear All, 

Regards, Linda Dunbar

---

From: Ron Cohen <[email-masked]>
Date: Sun, 26 Sep 2010 16:33:10 +0200

Hi Linda,

The potential conflict with DHCP that I'm familiar with is that DHCP clients
usually send an ARP request to figure out whether an IP addressed offered to
them is indeed free. Some implementation place 0.0.0.0 as the IP address of
the requesting node in these situations. If a proxy-ARP server answers such
an ARP request, the DHCP client will assume the address is already taken and
decline the DHCP offer.

I was hoping that Ralph would clarify his concern with breaking DHCP.
However unless we can all understand the details of the potential problem I
don't see any other choice but to ignore it.

Best,
Ron

On Mon, Sep 20, 2010 at 6:57 PM, Linda Dunbar <[email-masked]> wrote:

>  ------------------------------
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Ron Cohen
> *Sent:* Sunday, September 19, 2010 6:31 AM
> *To:* Linda Dunbar
> *Cc:* [email-masked]
> *Subject:* Re: [armd] some ideas on ARP optimization for one broadcast
> domain spanning across multiple sites
>
>
>
> Hi Linda,
>
>
>
> My comments:
>
>
>
> “public” MAC addresses which are used by core or other TORs. Under this
> commented that the host’s SA needs to stay with the data frame, otherwise,
> DHCP will be broken.
>
>
>
> Thank you
>
> Ron
>
>
>
>
>