Subject: [Idr] BGP Multipath source port entropy L2/L3/L4 ECMP LB hash

From: Gyan Mishra <[email-masked]>
Date: Wed, 1 Jul 2020 03:36:58 -0400

Dear IDR

BGP Multipath is commonly used for edge peering AFI/SAFI 1/1 2/1.  BGP
Multipath provides ECMP “flow” based load balancing, however the downside
is the load balancing is uneven as generally most vendor ECMP algorithm is
based on Source / Destination XOR.  As a result the load balancing is
uneven and not even close to 50/50.

It would be nice if BGP Multipath had a vxlan evpn like vxlan header source
port entropy L2/L3/L4 like algorithm so the load balancing is close to
50/50 with an even number of N way ECMP eBGP paths.

Does anyone know of any draft in progress related to this topic.  My
thoughts are that maybe a similar analogy is to a L2 VPN Fat Pseudowire
concept that uses the inner payload L3/L4 for load balancing that maybe I
could develop a specification if one does not exist to provide similar lb
entropy to BGP Multipath.

Kind Regards

Gyan
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*M 301 502-134713101 Columbia Pike *Silver Spring, MD

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 1 Jul 2020 11:32:13 +0200

Hi Gyan,

How you do load balancing is an internal matter to the box. What fields you
use to hash the packets into one of the outbound links is your vendor local
decision.

If a vendor just uses IP src/dst it usually have a knob to extend this to
other fields in IP header. If not open a sys wish with them.

Bottom line I see no room for a draft here. This is not a protocol issue -
it is purely a local implementation issue in fact not even related to BGP
:).

Best,
R.

On Wed, Jul 1, 2020 at 9:37 AM Gyan Mishra <[email-masked]> wrote:

> It would be nice if BGP Multipath had a vxlan evpn like vxlan header
> source port entropy L2/L3/L4 like algorithm so the load balancing is close
> to 50/50 with an even number of N way ECMP eBGP paths.
>
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 1 Jul 2020 08:55:45 -0700

Gyan,

For IP - most vendors do 5-tuple hashing by default with potentially more keys configurable, so L4 data, If present is used.  MPLS/GRE/similar encaps could use addition fields for entropy.
Hashing is a data-plane function and has nothing to do with BGP, you could however add a weight to a prefix and influence ratio of how the traffic towards the prefix is load-shared. This is done through link-bw community (non transitive as per spec, however implementations did violate this).
There’s ECMP considerations draft Petr and I have published (educational), mostly about BGP in DC but still a good reading ;-)

Cheers,
Jeff

> On Jul 1, 2020, at 00:37, Gyan Mishra <[email-masked]> wrote:
> 
> 
> Dear IDR
> 
> BGP Multipath is commonly used for edge peering AFI/SAFI 1/1 2/1.  BGP Multipath provides ECMP “flow” based load balancing, however the downside is the load balancing is uneven as generally most vendor ECMP algorithm is based on Source / Destination XOR.  As a result the load balancing is uneven and not even close to 50/50.
> 
> It would be nice if BGP Multipath had a vxlan evpn like vxlan header source port entropy L2/L3/L4 like algorithm so the load balancing is close to 50/50 with an even number of N way ECMP eBGP paths.
> 
> Does anyone know of any draft in progress related to this topic.  My thoughts are that maybe a similar analogy is to a L2 VPN Fat Pseudowire concept that uses the inner payload L3/L4 for load balancing that maybe I could develop a specification if one does not exist to provide similar lb entropy to BGP Multipath.
> 
> Kind Regards 
> 
> Gyan
> -- 
> 
> 
> Gyan Mishra
> Network Solutions Architect 
> M 301 502-1347
> 13101 Columbia Pike 
> Silver Spring, MD
>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 1 Jul 2020 16:04:58 +0000

Weighting the ratio based on capacity. DC FOUs in mind..

https://tools.ietf.org/html/draft-ietf-bess-evpn-unequal-lb-04

Thanks,
                             Jim Uttaro

From: Idr <[email-masked]> On Behalf Of Jeff Tantsura
Sent: Wednesday, July 01, 2020 11:56 AM
To: Gyan Mishra <[email-masked]>
Cc: IDR List <[email-masked]>
Subject: Re: [Idr] BGP Multipath source port entropy L2/L3/L4 ECMP LB hash

Gyan,

Dear IDR

Kind Regards

Gyan
--

[Image removed by sender.]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.verizon.com_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=s7ZzB4JbPv3nYuoSx5Gy8Q&m=WG2_umhJtFdIUqflRfYPtph3kLgLAfqRIfIsCeImQ6c&s=_t0TdXxSNA4Jv02k9BBz5OxH8KUqepJgEX9JgTTrLZ8&e=>

Gyan Mishra

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=DwQFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=s7ZzB4JbPv3nYuoSx5Gy8Q&m=WG2_umhJtFdIUqflRfYPtph3kLgLAfqRIfIsCeImQ6c&s=ooNiZlgI5bFxsFDOOfnaz4CRt0ZB_jMKe3fvW9FSnBs&e=>

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 2 Jul 2020 05:11:48 -0400

Hi Robert

Thanks for the feedback.  That’s what other folks who have replied as well
that the entropy is independent of BGP.  Makes sense.

Thanks

Gyan

On Wed, Jul 1, 2020 at 5:32 AM Robert Raszuk <[email-masked]> wrote:

> Hi Gyan,
>
> How you do load balancing is an internal matter to the box. What fields
> you use to hash the packets into one of the outbound links is your vendor
> local decision.
>
>> *M 301 502-134713101 Columbia Pike
>> <https://www.google.com/maps/search/13101+Columbia+Pike%C2%A0+Silver+Spring,+MD?entry=gmail&source=g>*Silver
>> Spring, MD
>> <https://www.google.com/maps/search/13101+Columbia+Pike%C2%A0+Silver+Spring,+MD?entry=gmail&source=g>
>>

*Gyan Mishra*

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 2 Jul 2020 05:27:43 -0400

Hi Jeff

I believe most vendors have the option for 5 tuple hash for ethernet bundle
per member link load balancing hash.  However most vendor lack ECMP hash
options as that is usually proprietary and not configurable to change.  At
least that is the case with Cisco however maybe other vendors allow you to
change the n-tuple hash algorithm for ECMP.

Agreed that ECMP hashing is a data plane forwarding function and nothing to
do with the BGP control plane.  Makes sense.

The link-bandwidth knob was kind of what I was looking for to achieve
better load balancing with BGP Multipath.

I read through the link-bw extended community draft you mentioned.

https://datatracker.ietf.org/doc/draft-ietf-idr-link-bandwidth/

Excerpt from the draft.

 In the context of BGP/MPLS VPNs [RFC4364
<https://datatracker.ietf.org/doc/html/rfc4364>], link bandwidth
community
   could be used to support inbound load balancing for multihomed sites,
   as follows.  Consider a site that is connected to PE1 and PE2.  Both
   PE1 and PE2 would advertise VPN-IP routes associated with the
   destinations within the site.  One way to enable other PEs to receive
   all these routes is to require the RD of the routes advertised by PE1
   to be different from the RD of the routes advertised by PE2.  The
   VPN-IP routes advertised by PE1 should carry the link bandwidth
   community; likewise for the VPN-IP routes advertised by PE2.  The
   bandwidth value carried in the community could be locally determined
   by PE1 and PE2.  Alternatively CEs of the site, when advertising IP
   routes to PE1 and PE2, could add the link bandwith community to these
   advertisements, in which case PE1 and PE2, when originating VPN-IP
   routes, would use the bandwidth value from the IP routes they
   received from the CEs to construct the link bandwidth community
   carried by these VPN-IP routes.

   An ingress PE, when sending traffic to destinations within the site,
   can use the bandwidth value carried in the community of the routes
   advertised by PE1 and PE2 to perform load sharing, where some of the
   traffic would go via PE1, while other traffic would go via PE2.

   If there are multiple paths to reach a destination and if only some
   of them have link bandwidth community, the load sharing router should
   not perform unequal cost load balancing based on link bandwidths.

      Please forward link to ECMP considerations draft.

Thank you!!

Gyan

On Wed, Jul 1, 2020 at 11:55 AM Jeff Tantsura <[email-masked]>
wrote:

> Gyan,
>
> For IP - most vendors do 5-tuple hashing by default with potentially more
> keys configurable, so L4 data, If present is used.  MPLS/GRE/similar encaps
> could use addition fields for entropy.
> Hashing is a data-plane function and has nothing to do with BGP, you could
> however add a weight to a prefix and influence ratio of how the traffic
> towards the prefix is load-shared. This is done through link-bw community
> (non transitive as per spec, however implementations did violate this).
> There’s ECMP considerations draft Petr and I have published (educational),
> mostly about BGP in DC but still a good reading ;-)
>
> Cheers,
> Jeff
>
> <https://www.google.com/maps/search/13101+Columbia+Pike+Silver+Spring,+MD?entry=gmail&source=g> *Silver
> Spring, MD
> <https://www.google.com/maps/search/13101+Columbia+Pike+Silver+Spring,+MD?entry=gmail&source=g>
>

*Gyan Mishra*

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 2 Jul 2020 05:34:05 -0400

I am familiar with the BGP EVPN ESI load balancing with DF election with
all active multi homed CEs.

It would be nice if their was something similar for IP VPN similar to ESI
groups all active multi home load balancing and DF election for BUM load
balancing.

Thank you!!

Gyan

On Wed, Jul 1, 2020 at 12:05 PM UTTARO, JAMES <[email-masked]> wrote:

> *Weighting the ratio based on capacity. DC FOUs in mind..*
>
>
>
> *https://tools.ietf.org/html/draft-ietf-bess-evpn-unequal-lb-04
> <https://tools.ietf.org/html/draft-ietf-bess-evpn-unequal-lb-04> *
>
>
>
> *Thanks,*
>
> *                             Jim Uttaro*
>
>
>
> *From:* Idr <[email-masked]> *On Behalf Of * Jeff Tantsura
> *Sent:* Wednesday, July 01, 2020 11:56 AM
> *To:* Gyan Mishra <[email-masked]>
> *Cc:* IDR List <[email-masked]>
> *Subject:* Re: [Idr] BGP Multipath source port entropy L2/L3/L4 ECMP LB
> hash
>
>
>
> Gyan,
>
>
>
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.verizon.com_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=s7ZzB4JbPv3nYuoSx5Gy8Q&m=WG2_umhJtFdIUqflRfYPtph3kLgLAfqRIfIsCeImQ6c&s=_t0TdXxSNA4Jv02k9BBz5OxH8KUqepJgEX9JgTTrLZ8&e=>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
>
>
> *M 301 502-1347 13101 Columbia Pike
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
> *Silver Spring, MD
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=DwQFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=s7ZzB4JbPv3nYuoSx5Gy8Q&m=WG2_umhJtFdIUqflRfYPtph3kLgLAfqRIfIsCeImQ6c&s=ooNiZlgI5bFxsFDOOfnaz4CRt0ZB_jMKe3fvW9FSnBs&e=>
>
> --

*Gyan Mishra*