Subject: [rbridge] LIDs

From: kreeger at cisco.com (Larry Kreeger (kreeger))
Date: Fri, 3 Nov 2006 15:14:06 -0800

Caitlin Bestler wrote on Friday, November 03, 2006 9:07 AM:

> Larry Kreeger (kreeger) wrote:
>> Hi,
>> 
>> I was catching up on old RBridge threads and saw this one about LIDs.
>> 
>> LIDs can be useful in reducing costs in large switches where the
>> entire (potentially large) local MAC table would not need to be
>> present in every module of the system.  In fact it could probably
>> eliminate the need to keep locally learned MACs in hardware at all.
>> Assuming a switch with 16 slots and
>> 48 ports per linecard, and servers running 16 virtual machines each
>> connected to the ports, the number of local MACs would be 16*48*16=
>> 12,288.  With 48 bits per address, that is ~59kbits.
>> 
>> With LIDs, an entry is need only per port, not per local MAC.
>>  So given a 16 bit LID, only 16 cards * 48 ports * 16 bits =
>> ~12kbits, a savings of ~47kbits. 
>> 
>> The more locally attached MACs, the bigger the savings.
>> 
> 
> Instead of keeping the MAC to Port mapping in the egress RBridge you
> instead keep it in each ingress rbridge that has learned of this
> rbridge. I'm not seeing a net savings in memory costs anywhere. There
> is certainly a potential latency benefit, but is there really one
> that is valid over the desired lifespan of the protocol?    

The ingress RBridge must already have a mapping from MAC to egress
RBridge nickname.  So, the table must only be widened to include the LID
- which is much smaller (depending on the size of the LID) than the MAC
to port mapping needed at the egress RBridge.

Also, the size of the table at the ingress RBridge depends on how many
end stations are being actively communicated with from that RBridge.  If
the ingress RBridge is a distributed system, then each module (e.g.
ASIC) only needs to populate the table with mappings for the active
flows through that part of the system.  If we assume that every end
station is not directly communicating with every other end station, then
different entries would exist in each MAC -> Egress RBridge/LID table.
This allows a large distributed system to scale up without requiring
each ASIC to increase its table size - because the mapping to LID is
spread out versus being concentrated at the inter-RBridge port ASICs of
a large RBridge.

It all depends on ratio of RBridge size (locally MAC addresses) versus
the number of flow active.

 - Larry

---

From: gibanez at it.uc3m.es (Guillermo Ibáñez)
Date: Sun, 05 Nov 2006 21:24:40 +0100

I agree that LIDs can help to improve large switches scalability.
Guillermo

Larry Kreeger (kreeger) escribi?:
> _______________________________________________
> rbridge mailing list
> rbridge at postel.org
> http://mailman.postel.org/mailman/listinfo/rbridge
>