Subject: Re: [rbridge] Updated charter

From: Linda Dunbar <[email-masked]>
Date: Thu, 15 Jul 2010 17:56:54 -0500

I have some concern of the bullet (2) (Addition of optional support for ARP
/ Neighbor Discovery optimization) to be added to TRILL re-charter work
list. 

ARP is for resolving MAC address from host's IP address. It is not specific
to TRILL. Straight Ethernet needs ARP.  
Donald's new draft on ARP optimization
(http://pothole.com./~dee3/drafts/draft-eastlake-trill-rbridge-arp-xx.txt)
describes a generic ARP proxy procedure, i.e. changing an ARP broadcast msg
to a unicast msg if the target IP is cached in its local cache. Otherwise do
nothing. 
ARP proxy function should be allowed on any device. As a matter of fact, ARP
proxy has been deployed in many places. When ARP proxy is enabled on any
devices, RBridge should treat the ARP Broadcast or unicast messages same way
as all other IEEE802.1Q data frames. No special processing should be needed
by RBridge for ARP messages. If an Rbridge needs to add ARP proxy function
on some ports, it is exactly same as RBridge having standard IEEE802.1Q
functions on some ports. But TRILL doesn't need to re-define IEEE802.1Q,
does it? 

Best regards, 

Linda Dunbar 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Erik Nordmark
Sent: Monday, May 10, 2010 1:37 AM
To: Developing a hybrid router/bridge.
Subject: [rbridge] Updated charter

Based on the comments on the mailing list, here is the updated charter.

Description of Working Group:

   The TRILL WG has specified a solution for shortest-path frame
   routing in multi-hop IEEE 802.1-compliant Ethernet networks with
   arbitrary topologies, using an existing link-state routing protocol
   technology and encapsulation with a hop count.
   [Add reference to RFC?]

   The current work of the working group is around operational and
   deployment support for the protocol. This includes a MIB and other
   pieces needed for operations, but also additional ways to extend and
   optimize TRILL for the properties of the networks on which it is 
deployed.

   The WG will work on the following items:

   (1) A MIB for TRILL which specifies the unique pieces needed for the
   protocol while reusing what is in the IS-IS MIB and the IEEE 802.1
   MIB modules for pieces that are not unique to TRILL.

   (2) Addition of optional support for ARP / Neighbor Discovery
   optimization.

   (3) Refinement of the TRILL Header Options feature and specification
   of an initial base set of options.

   (4) Support for RBridge campus VLAN regions, such that the same VLAN
   ID can have different meanings in different regions, requiring
   configuration only at border RBridges.

   (5) Write up how Operations and Management (OAM) will be handled in
   networks using TRILL.

   (6) Determine how TRILL will provide improved support for data
   centers such as support of the IEEE 802.1 "Data Center Bridging"
   standards.

   (7) Document interoperability test results of protocol
   implementations.

Goals and Milestones:
   Done     - Accept base protocol specification as a WG document
   Done     - Accept problem statement and applicability as a WG work
              item
   Done     - Start work with routing area WG(s) to undertake TRILL
              extensions
   Done     - Accept architecture document as a WG work item
   Done     - Accept routing protocol requirements as a WG work item
   Done     - Choose routing protocol(s) that can meet the requirements
   Done     - Submit problem statement and applicability to IESG for
              Informational RFC
   Done     - Submit base protocol specification to IEEE/IETF expert
              review
   Done     - Base protocol specification submitted to the IESG for
              publication as a Proposed Standard RFC
   Done     - First draft showing what is needed for MIB

NEW:
   Done     - Initial WG draft on VLAN Mapping
   Done     - Initial WG draft TRILL Header Options
   Jul 2010 - Initial WG draft on ARP/ND optimizations
   Aug 2010 - Submit TRILL Header Options to IESG for publication as
              Proposed Standard
   Aug 2010 - Submit Rbridge Campus VLAN Regions to IESG for publication
              as Proposed Standard
   Oct 2010 - Submit RBridge MIB to IESG for publication as Proposed
              Standard
   Oct 2010 - Initial WG draft for RBridge support of Data Center
              Bridging
   Oct 2010 - Initial WG draft on RBridge OAM
   Jul 2011 - Re-charter or shut down the WG
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: James Carlson <[email-masked]>
Date: Fri, 16 Jul 2010 07:53:34 -0400

> to a unicast msg if the target IP is cached in its local cache. Otherwise do

No, it doesn't say to change the broadcast to a unicast.  In 6b on page
9, it says to forward it "as if it was [sic]" unicast.

The distinction is important.  It's not mangling data.  It's merely
choosing not to forward to ports on which the system knows that there
are no interested listeners.

(By "changing," I assume you're not referring to 6a, which essentially
describes the possibility of creating an actual reply to an ARP message
if the correct response information is known locally.)

> functions on some ports. But TRILL doesn't need to re-define IEEE802.1Q,

I don't see how anything in that draft is tantamount to redefining
802.1Q.  Or, really, how anything in the document has anything to do
with 802.1Q.

Could you fill in some details?

-- 
James Carlson         42.703N 71.076W         <[email-masked]>
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 16 Jul 2010 09:59:31 -0500

James, 

Please see my comments inserted below:

-----Original Message-----
From: James Carlson [mailto:[email-masked]] 
Sent: Friday, July 16, 2010 6:54 AM
To: Linda Dunbar
Cc: 'Erik Nordmark'; 'Developing a hybrid router/bridge.'; 'Peter
Ashwood-Smith'; 'Jari Arkko'; 'Adrian Farrel'; 'David Harrington'; 'Hares
Susan'
Subject: Re: [rbridge] Updated charter

[Linda] It does the exactly same thing! There are the three steps if target
IP is found in its cache:
*	first finds the unicast destination MAC address from its Cache by
the target IP address, 
*	second, uses the regular RBridge function to find the egress RBridge
address based on the unicast MAC address, and then 
*	third, encapsulates the data frame with the egress Rbridge address
as the DA. 

The first step above is commonly called ARP Proxy, i.e. find the unicast MAC
if the target IP is found in its local cache.  The second and third steps
above are regular RBridge functions. 

ARP Proxy and RBridge are two separate functions. Merging them together just
makes the specification unnecessarily more complicated. 
Today, I can buy off-the-shelf RBridge chip set and I can also buy the ARP
proxy stack. They are two separate modules. They can be from two different
vendors. 
Mixing them together just makes it more difficult for equipment and chip
vendors. 

[Linda] I am trying to say that when a switch port supports two distinct
functions, like BGP and IS/IS, you don't need to create one specification to
merge BGP with IS/IS.

---

From: James Carlson <[email-masked]>
Date: Fri, 16 Jul 2010 11:22:51 -0400

Linda Dunbar wrote:
> [Linda] It does the exactly same thing! There are the three steps if
> target IP is found in its cache:
> ·       first finds the unicast destination MAC address from its Cache
> by the target IP address,
> 
> ·       second, uses the regular RBridge function to find the egress
> RBridge address based on the unicast MAC address, and then
> 
> ·       third, encapsulates the data frame with the egress Rbridge
> address as the DA.
> 
> The first step above is commonly called ARP Proxy, i.e. find the unicast
> MAC if the target IP is found in its local cache.  The second and third
> steps above are regular RBridge functions.

It would help me (at least) to have references to the specific parts of
the document that you're commenting on.  It's unclear to me exactly what
step concerns you, or why it matters whether a given step is commonly
called something else.  (For what it's worth, I'd dispute that usage.
"Proxy ARP" has a distinct meaning in the places where RFC 826 is
commonly used, and it doesn't seem to match exactly what you're describing.)

In any event, there's no section here that talks about changing the
original frame's destination to unicast.  The tunneling mechanism does
indeed carry frames inside the TRILL cloud using unicast between peer
RBridges, but I don't see how that's relevant.

> ARP Proxy and RBridge are two separate functions. Merging them together
> just makes the specification unnecessarily more complicated.

This is a separate draft from TRILL.  It's an optimization that some
vendors may choose to employ and that others might not.

I agree that if it were in the base protocol document, then that'd be a
problem.

But as a separate document, how does it make things more complicated?

> Today, I can buy off-the-shelf RBridge chip set and I can also buy the
> ARP proxy stack. They are two separate modules. They can be from two
> different vendors.

I'm not sure what you're referring to here.  Can you provide clear
pointers?  If the "ARP proxy stack" (whatever that may be) is a separate
module, doesn't that mean that it has to mangle the sender's frames?

The mechanism here is a way to glean IP/MAC pairings such that when you
see a broadcasted request, you know where the "true" destination lies,
and don't need to bother with an actual network-wide broadcast.

This "true" destination relies on knowing exactly what's in the RBridge
forwarding database.  That's something that a distinct "module" can't
possibly know, because it's an internal detail.

(And, for what it's worth, "modules" are completely out of scope in most
IETF discussions.  The documents describe [or are _supposed_ to
describe] the on-the-wire behavior, not the internal system architecture
that may or may not lead to that behavior.)

> Mixing them together just makes it more difficult for equipment and chip

How?

I don't follow.  If you don't like what the draft describes, then nobody
is saying that you need to implement it.  Ignore the draft, and you're done.

It doesn't really matter whether any or all or no RBridges implement
this specification.  They should still interoperate using the base document.

If you see an interoperability issue, though, then that would be good to
document, because that'd be an important problem.

> functions, like BGP and IS/IS, you don't need to create one
> specification to merge BGP with IS/IS.

Can you point to a specific document that should be referenced as "Proxy
ARP" rather than implementing what's described in this draft?

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 16 Jul 2010 15:42:20 -0500

James, 

Section 3 of the draft (ARP Optimization Details) listed 5 steps. The first
step is "observing native ARP request and reply frame" to learn the frame's
IP, VLAN. 

This snooping at line rate is not trivial, requiring the switch port to
filtering all the ARP messages at line rate and store the learned IP
addresses and VLANs. But most ARP requests/replies may not traverse through
TRILL ports on a switch, which really defeats the purpose of this extra
work. 

Let's assume that a switch has 12 ports, with 2 ports facing TRILL
interface, 10 ports facing servers via standard Ethernet interface. 

Target hosts of ARP requests from servers can be one of the servers connect
to the switch. When servers are virtualized, target hosts can even be on the
same physical server, you could even have ARP messages being hairpinned back
(IEEE802.1Qbc & 802.1Qbg). 

If you want to add a function to snoop all the ARP requests and replies (ARP
Proxy) to cache target IP-MAC/VLAN to reduce broadcast messages, it is much
more effective to add this function on the ports facing servers, or in a
centralized CPU, so that the switch's local cache (IP<->MAC/VLAN) can have
all the IP <->MAC/VLAN mapping for all hosts traversing through this switch.
When a target IP is found in the switch's local cache, the unicast MAC/VLAN
is used as DA, so that this frame will only exit one port of the switch,
instead of broadcasting to all ports. If the frame is to exit the TRILL
port, the RBridge function on the port will encapsulate the frame with
appropriate egress RBridge address. Most likely the frame will exit ports
facing other servers connected to the switch. 

Therefore, mixing ARP proxy function with TRILL just creates a more
complicated specification, which doesn't help anyone (system vendors,
component vendors or software stack vendors). Most importantly, ARP proxy
and RBridge are two distinct functions.  This is especially bad when APR
proxy has been deployed in many places. 

For a switch with TRILL ports, ports facing to (virtual) servers, and ports
facing other switches via Ethernet interfaces, mixing ARP proxy with TRILL
makes even less sense. Most traffic will be switched between
Ethernet<->Ethernet and Ethernet <-> TRILL. 
 
If it is still not clear, we can discuss more in Maastricht. 

Linda Dunbar

-----Original Message-----
From: James Carlson [mailto:[email-masked]] 
Sent: Friday, July 16, 2010 10:23 AM
To: Linda Dunbar
Cc: 'Erik Nordmark'; 'Developing a hybrid router/bridge.'; 
Subject: Re: [rbridge] Updated charter

Linda Dunbar wrote:
> .       first finds the unicast destination MAC address from its Cache
> .       second, uses the regular RBridge function to find the egress
> .       third, encapsulates the data frame with the egress Rbridge
> address as the DA.
> 

How?

---

From: James Carlson <[email-masked]>
Date: Sat, 17 Jul 2010 10:19:05 -0400

On 7/16/10 4:42 PM, Linda Dunbar wrote:
> addresses and VLANs.

It already has to snoop at all frames on the wire to learn source MAC
addresses and to do normal forwarding.  In some implementations (such as
the one I worked on at Sun), that's 99%+ of the cost.  Having to look a
little more at normally low-rate ARPs in particular to learn a bit more
is very little work compared to handling the bulk data traffic and (more
importantly) the significant effects that running in promiscuous mode
all the time has.

The trade-offs may be different for other implementations.  That's not a
problem.  Just because there's a draft doesn't mean there's someone
forcing you to implement.

(For what it's worth, performance-based arguments are usually viewed by
IETFers as weak, unless the "performance" issue is rather severe, such
as scaling geometrically or worse, because today's hardware is fleeting
but protocols are forever.  In this case, the work required grows with
the traffic, and only for those implementations that bother to optimize.)

> But most ARP requests/replies may not traverse through

It's an optimization.  It optimizes the cases it's able to, and does
leaves for regular processing the ones it cannot.  How could it be
otherwise?

The very same could be said of all unicast traffic.  Not all MAC SAs
will be seen by all RBridges, so not all RBridges will have exact
knowledge of the locations of all nodes in the network.  Given that,
should we not bother to try to optimize the forwarding of unicast
traffic, because the purpose is "defeated?"

(Actually, I expect the ARP optimization to work a bit better, because
the draft specifies keeping the gratuitous ARPs as regular broadcast,
meaning that they'll always be seen -- much unlike the cached unicast case.)

> facing other servers connected to the switch. 

It sounds like you're advocating modifying the packet in flight, so that
the Ethernet MAC DA becomes unicast rather than broadcast.

That's possible, and it does have the somewhat interesting property that
it could be modularized, but it's not what the draft talks about.

The advantages of what the draft proposes are that it preserves the
frame's integrity (so that it doesn't presume to know whether the
broadcast/unicast distinction is important to the receiver), it doesn't
require modifications of the original frame (which are terribly
expensive in some implementations -- prepend is cheap, but modify costs
a data copy operation), and, perhaps most importantly, it's *exactly*
analogous to the existing multicast optimization.  When the system knows
(from monitoring other protocols, such as IGMP) what ports or paths a
packet should traverse, it uses them, and when it doesn't know, it sends
it through the normal broadcast-like tree.  This just the same scheme
applied to ARP/NDP.  (And, really, what's done for unicast as well.)

As I see it, as an implementer, you're really free to do either one of
these mechanisms (as you propose or as the draft proposes) without
having to coordinate with anyone else, because all of this relies on the
existing unicast infrastructure.  It's not a true protocol issue and
requires no extra signaling.  As such, perhaps it doesn't really need to
be in an IETF draft and could be left to implementations to decide what
to do.  But it's also not something worth fighting over.

At most, perhaps the draft could acknowledge that there's a fourth
option -- mangle^Wset the DA on the packet to the cached value and
forward normally.

> proxy has been deployed in many places. 

I still don't get the "complicated specification" issue.  This is a
separate draft.  I would expect that as it's an optional feature, it'll
remain a separate document "forever."  This isn't like one of those IEEE
lettered documents that eventually gets folded into the base document.

Where is it written that ARP proxy and RBridge "are" or must be distinct
functions?  You seem to be stating that as an article of faith, but I
don't understand the basis.

And I fail to see how deployment of any sort of ARP Ethernet MAC DA
modification mechanism would make the proposed mechanism "bad."  Do they
fail to interoperate in some way?  Demonstrating some problem -- some
way in which "true ARP proxies" fail to interoperate with RBridges using
the ideas in the draft -- would go a long way towards making the
objections clear.

> Ethernet<->Ethernet and Ethernet <-> TRILL. 

I'm afraid I can't parse that objection.  I can't see how the
applicability of the mechanism proposed in the draft would ever differ
from that of any other optimization mechanism.

If there are indeed special usage cases where optimization is unwise,
then savvy implementers should avoid making those unwise optimizations
for the sake of the product they're building.  If they don't, it's not a
big deal, because the marketplace will eventually solve that problem as
well.

> If it is still not clear, we can discuss more in Maastricht. 

It'll be a decidedly one-sided conversation, as I have no plans to
travel.  ;-}

---

From: Eric Gray <[email-masked]>
Date: Sat, 17 Jul 2010 21:02:55 -0400

James,

	See "EG> " below...

--
Eric 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of James Carlson
Sent: Saturday, July 17, 2010 10:19 AM
To: Linda Dunbar
Cc: 'Developing a hybrid router/bridge.'; 'Jari Arkko'
Subject: Re: [rbridge] Updated charter

It already has to snoop at all frames on the wire to learn source MAC
addresses and to do normal forwarding.  In some implementations (such as
the one I worked on at Sun), that's 99%+ of the cost.  

EG> So, the fact is that source MAC learning is already done in bridges
EG> (and has been done in bridges for a very long time).  Finding H/W
EG> that does this _is_ trivial.
EG> 
EG> For sure, if you want to start over - possibly because you want to
EG> do additional stuff, while you're about it - it can be a bit hard.
EG> 
EG> And that makes me want to understand whether your implementation
EG> elected to do things the hard way simply because the implementors 
EG> in your case were already predisposed to doing this extra stuff.

--- [SNIP] ---