Subject: Re: [rbridge] A question about the Rbridge ID

From: Xu Xiaohu <[email-masked]>
Date: Wed, 01 Sep 2010 16:52:31 +0800

Hi all

Another possible side-effect of using nicknames rather than IP addresses as trill rbridge IDs and accordingly using trill header rather than IP header is making the multicast distribution in trill network unscalable, complex and inefficient. 

First, if there are 4096 VLANs in a trill network and each VLAN is mapped to a distinct tree, each transit rbridge would have to calculate 4096 SPF trees, and any certain topology change may cause all these 4096 SPF trees to be recalculated. Is this usage scalable from the perspective of the control plane?  

Second, if multiple VLANs are associated with a single tree in a trill network, each transit rbridge receiving multicast traffic from this tree would have to prune such distribution tree on per-VLAN basis so as to avoid unnecessary bandwidth waste, e.g., for each adjacency and for each tree, each transit rbridge has to mark the set of VLANs reachable downstream. Wouldn't this usage make the hardware multicast forwarding table much complex? 

Third, if an ingress rbridge of a given tree is not close to the root of such tree, the multicast forwarding efficiency of trill network is the same as that of the current STP bridge network, and less efficient than the Source Path Tree (SPT) used in PIM-SM.

In contrast, if IP routing is used between rbridges, MVPN technology could be easily inherited into trill. For example, each customer VLAN is associated with a dedicated provider multicast group address. And the multicast distribution trees on the provider network are built based on the proven PIM-SM technology.

If I misunderstood it in some ways, please feel free to correct me.

Best wishes,
Xiaohu

> -----邮件原件-----
> 发件人: Xu Xiaohu [mailto:[email-masked]]
> 发送时间: 2010年8月30日 9:12
> 收件人: 'Donald Eastlake'
> 抄送: [email-masked]'; [email-masked]'
> 主题: re: [rbridge] A question about the Rbridge ID
> 
> Hi Donald,
> 
> > -----邮件原件-----
> > 发件人: [email-masked] [mailto:[email-masked]] 代
> 表
> > Donald Eastlake
> > 发送时间: 2010年8月29日 6:00
> > 收件人: Xu Xiaohu
> > 抄送: [email-masked]; [email-masked]
> > 主题: Re: [rbridge] A question about the Rbridge ID
> >
> > Hi Xiaohu,
> >
> > I'll try to give an answer to your questions below:
> >
> > On Fri, Aug 27, 2010 at 9:39 PM, Xu Xiaohu <[email-masked]> wrote:
> > > Hi Radia,
> > >
> > > As stated in RBRIDGE specification:
> > >   "To save room in the TRILL header and simplify forwarding lookups, a
> > >   dynamic nickname acquisition protocol is run among the RBridges to
> > >   select 2-octet nicknames for RBridges, unique within the campus,
> > >   which are an abbreviation for the 6-octet IS-IS system ID of the
> > >   RBridge.  The 2-octet nicknames are used to specify the ingress and
> > >   egress RBridges in the TRILL header."
> > >
> > > My question is why not consider to use IP addresses directly as Rbridge
> > > identifiers. To realize free configuration, these IP addresses could also
> > be
> > > automatically generated in the similar way as the TRILL nickname does.
> >
> > An obvious disadvantage is that IP addresses are twice as big as TRILL
> > nicknames for IPv4 (worse for IPv6 but I can't see why you would use
> > IPv6). But the main thing is that TRILL has already made its decision.
> >
> > > Some conceivable benefits of using IP address as Rbridge ID include:
> > >
> > > 1) The core Rbridge could directly use the proven IP forwarding chips.
> >
> > There was consideration of using various existing headers (IP,
> > 802.1ah, MPLS) early on in the development of the TRILL standard but
> > these were all rejected for one reason or another. It is my
> > understanding that leading switching chip companies now all have TRILL
> > chips so chip availability is not a problem.
> 
> Yes, TRILL chips are available. However, the economy of scale for IP chips is
> not available for TRILL chips.
> 
> Thanks for your reply anymore.
> 
> Best wishes,
> Xiaohu
> 
> > > 2) Matured OAM tools for IP, e.g., PING and Traceroute, can be used
> > > directly. Or do you have any plan to develop TRILL nickname based PING or
> > > Traceroute tools?
> >
> > RBridges are designed so that an RBridge level Traceroute could be
> > added fairly easily. There are two drafts that I know of which are
> > being developed for TRILL OAM tools of one sort or another. OAM is
> > listed on the proposed new Charter for the TRILL WG.
> >
> > > 3) Features in the IP header, e.g., QoS, Fragmentation can be inherited
> > > naturally. For example, there is no need to use complex technology to deal
> > > with large packet as stated in Rbridge specification, "... If traffic
> > > engineering tools know which links support larger than minimally acceptable
> > > data packet sizes, paths can be computed that can support large data
> > > packets."
> >
> > Constrained routing does not seem like a complex technology. But, in
> > any case, people will be expecting Layer 2 QoS, that is frame
> > priority, Data Center Bridging ( see
> > http://tools.ietf.org/html/draft-eastlake-trill-rbridge-dcb-00 ), and
> > things like that. It was also a goal to be able to incrementally
> > replace 802.1Q customer bridges with RBridges and it seems like a
> > complex design problem to try to do that with IP facilities. Finally,
> > people engineering Layer 2 networks, particularly in things like Data
> > Centers, generally have large MTUs, sometimes 9K jumbo frames, so
> > exceeding the MTU is rarely a problem and if it is many people think
> > the fragmentation is an inefficient solution.
> >
> > > Do you believe saving room in the packet header is a strong reason for
> > > developing a totally new TRILL header, rather than reusing the proven IP
> > > header for the TRILL Rbridge? Or would you please give some other reasons
> > > for this option?
> >
> > I would say it doesn't matter any more. The TRILL header has already
> > been developed, the TRILL base protocol specification is approved as
> > an IETF standard, hardware has already been developed or adapted for
> > TRILL, TRILL implementations are far enough along that a publicly
> > announced TRILL interoperability event has been held at UNH IOL and it
> > looks like a second one will be scheduled early next year. I don't see
> > any reason for TRILL to change the basic design at this point.
> >
> > > Best wishes,
> > > Xiaohu
> >
> > Thanks,
> > Donald
> >
> > _______________________________________________
> > rbridge mailing list
> > [email-masked]
> > http://mailman.postel.org/mailman/listinfo/rbridge

_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: Radia Perlman <[email-masked]>
Date: Wed, 1 Sep 2010 02:45:17 -0700

You are confused about TRILL trees.

TRILL calculates a few trees, with the default being 2 (or maybe it
was 1...whatever), but it could be more.  I don't think there would
ever be an advantage in calculating more than, say 5.

The ingress RBridge chooses one of the trees for a multidestination
frame.  That determines the distribution path, but although each tree
is spanning, VLANs and IP multicast address can be used to filter.

I don't have time  to explain in an email message, but now with that
hint, try rereading the specification where it explains about trees.

Radia

---

From: Xu Xiaohu <[email-masked]>
Date: Wed, 01 Sep 2010 18:00:17 +0800

Yes, I understand that the whole rbridge network could even calculate just a
single tree. However, the traffic congestion would happen on the root of
that tree.

Xiaohu

> -----邮件原件-----
> 发件人: Radia Perlman [mailto:[email-masked]]
> 发送时间: 2010年9月1日 17:45
> 收件人: Xu Xiaohu
> 抄送: Donald Eastlake; [email-masked]; [email-masked]
> > Another possible side-effect of using nicknames rather than IP addresses
as
> trill rbridge IDs and accordingly using trill header rather than IP header
is
> making the multicast distribution in trill network unscalable, complex and
> inefficient.
> >
> > First, if there are 4096 VLANs in a trill network and each VLAN is
mapped
> to a distinct tree, each transit rbridge would have to calculate 4096 SPF
trees,
> and any certain topology change may cause all these 4096 SPF trees to be
> recalculated. Is this usage scalable from the perspective of the control
plane?
> >
> > Second, if multiple VLANs are associated with a single tree in a trill
network,
> each transit rbridge receiving multicast traffic from this tree would have
to
> prune such distribution tree on per-VLAN basis so as to avoid unnecessary
> bandwidth waste, e.g., for each adjacency and for each tree, each transit
> rbridge has to mark the set of VLANs reachable downstream. Wouldn't this
usage
> make the hardware multicast forwarding table much complex?
> >
> > Third, if an ingress rbridge of a given tree is not close to the root of
such
> tree, the multicast forwarding efficiency of trill network is the same as
that
> of the current STP bridge network, and less efficient than the Source Path
Tree
> (SPT) used in PIM-SM.
> >
> > In contrast, if IP routing is used between rbridges, MVPN technology
could
> be easily inherited into trill. For example, each customer VLAN is
associated
> with a dedicated provider multicast group address. And the multicast
> distribution trees on the provider network are built based on the proven
PIM-SM
> technology.
> >> > 发件人: [email-masked]
[mailto:[email-masked]]
> 代
> >> > >   "To save room in the TRILL header and simplify forwarding
lookups,
> a
> >> > > identifiers. To realize free configuration, these IP addresses
could
> also
> >> Yes, TRILL chips are available. However, the economy of scale for IP
chips
> is
> >> > > directly. Or do you have any plan to develop TRILL nickname based
PING
> or
> >> > > naturally. For example, there is no need to use complex technology
to
> deal
> >> > > developing a totally new TRILL header, rather than reusing the
proven
> IP

---

From: Radia Perlman <[email-masked]>
Date: Wed, 1 Sep 2010 03:33:35 -0700

That's another misconception. With a bidirectional tree (which is what
happens in TRILL as opposed to PIM), each packet is received by each
node once.  The root gets no more traffic than any other node.

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 1 Sep 2010 12:19:24 -0400

Hi,

> Another possible side-effect of using nicknames rather than IP addresses as trill rbridge IDs and accordingly using trill header rather than IP header is making the multicast distribution in trill…

I think multi-destination frame distribution in TRILL is quite simple
and efficient.

> First, if there are 4096 VLANs in a trill network and each VLAN is mapped to a distinct tree, each transit rbridge would have to calculate 4096 SPF trees, and any certain topology change may cause…

First of all, the default number of trees is one. Most people believe
that a small number tree will be enough even for very big networks.

Second, VLANs are not mapped to trees in TRILL. Every tree carries
traffic for all VLANs. There is no facility in the TRILL standard to
restrict the VLANs of traffic using a multi-destinaiton frame
distribution tree. It is the ingress RBridge that makes the decision.

> Second, if multiple VLANs are associated with a single tree in a trill network, each transit rbridge receiving multicast traffic from this tree would have to prune such distribution tree on per-VLA…

As I say, there is no association between VLANs and trees. All trees
are usable for all VLANs.

It is optional feature but a "SHOULD" for a transit RBridge to prune
based on VLAN. It is optional feature but a "SHOULD" for a transit
RBridge to prune based on destination multicast address. Implementing
this pruning for any particular number of VLANs, number of multicast
group, and/or number of tree is an engineering problem. No doubt to be
able to prune all possible VLANs and a very large number of multicast
groups and a very large number of trees would involve some difficulty,
depending on what the value of "large" is. but, as I say I don't know
of anyone planning to use a large number of multi-destination frame
distribution trees. There are a number of techniques available some of
which provide tradeoffs between various network metrics.

> Third, if an ingress rbridge of a given tree is not close to the root of such tree, the multicast forwarding efficiency of trill network is the same as that of the current STP bridge network, and l…

I don't know what you mean by "forwarding efficiency". The root of a
distribution tree could be a leaf node of that tree. The best thing to
do depends on your network environment. If you have lots of scattered
senders and listeners for multicast frames, optimal distribution in
terms of minimal link utilization would requires lots of tree,
although the gain in efficiency over a few trees seems small, but
above you are complaining about how hard it is to maintain lots of
trees. In TRILL, you can specify many or few although, of course, what
you can do depends on your RBridge campus size and topology and your
switches and what trade off you want between better routing and more
rapid re-calculating when there is a topology change. If you have a
few very heavy multicast source RBridge, for example, you can, in
TRILL, have multiple different shortest-path trees calculated rooted
at each, if you network topology provides for that, so you can
multi-path multi-destination frames from these heavy sources while
still being optimal.

> In contrast, if IP routing is used between rbridges, MVPN technology could be easily inherited into trill. For example, each customer VLAN is associated with a dedicated provider multicast group ad…

PIM-SM is clearly less efficient than TRILL in terms of link
utilization because frames frequently get unicast across the network
and then distributed on a tree. I'm not sure I know enough about the
rest of your acronym blizzard to respond properly.

I am not impressed now you keep emphasizing how "proven" certain
technologies are. TRILL is a new and superior technology not bound to
copy existing protocols. I would say that AppleTalk and OSI and ATM
and many other protocols are proven technology that is even older and
so more "proven" than those you cite. That does not mean we should
copy them.

Thanks,
Donald

---

From: Anoop Ghanwani <[email-masked]>
Date: Wed, 1 Sep 2010 11:10:38 -0700

I would further add that the tree could be pruned in such
a way for certain flows (based on IGMP or VLAN) 
that even through the packet is traveling on a tree 
rooted at a certain node, the root itself never even 
receives a copy of the frame.

The root just identifies the tree that the packet
is traveling on.  All packets do not need to be
forwarded to the root.

Anoop

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Radia Perlman
> Sent: Wednesday, September 01, 2010 3:34 AM
> To: Xu Xiaohu
> Cc: Donald Eastlake; [email-masked]; [email-masked]
> Subject: Re: [rbridge] A question about the Rbridge ID
> 
> >> >> > >   dynamic nickname acquisition protocol is run among the
> RBridges
> > to
> >> >> > >   RBridge.  The 2-octet nicknames are used to specify the
> ingress
> > and
> >> >> > > My question is why not consider to use IP addresses directly
> as
> > Rbridge
> >> >> > > automatically generated in the similar way as the TRILL
> nickname
> > does.
> >> >> >
> >> >> > An obvious disadvantage is that IP addresses are twice as big
> as
> > TRILL
> >> >> > > 1) The core Rbridge could directly use the proven IP
> forwarding
> > chips.
> >> >> >
> >> >> > understanding that leading switching chip companies now all
> have
> > TRILL
> >> >> > > engineering tools know which links support larger than
> minimally
> >> acceptable
> >> >> > > data packet sizes, paths can be computed that can support
> large
> > data
> >> >> > > packets."
> >> >> >
> >> >> > > Do you believe saving room in the packet header is a strong
> reason
> > for
> >> >> > > header for the TRILL Rbridge? Or would you please give some
> other
> > reasons
> >> >> > > for this option?
> >> >> >
> >> >> > announced TRILL interoperability event has been held at UNH IOL
> and
> > it
> >> >> > looks like a second one will be scheduled early next year. I
> don't
> > see

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 02 Sep 2010 08:41:27 +0800

Yes, you are right. After sending out that email yesterday, I have realized
that I made a very simple mistake :)

I should have said that the traffic congestion would happen on that single
tree and the distribution path of that single tree can not be ensured
optimal for all traffic.

Xiaohu

表
> Radia Perlman
> 发送时间: 2010年9月1日 18:34
> > Yes, I understand that the whole rbridge network could even calculate
just
RBridges
ingress
reason
other
and

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 1 Sep 2010 21:58:42 -0400

Hi Xiaohu,

> tree and the distribution path of that single tree can not be ensured

Yes, you are right. But with TRILL you can choose how many
distribution tree you have and where their roots are to avoid
congestion and be closer to optimal.

Thanks,
Donald

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 02 Sep 2010 10:21:43 +0800

Hi Donald,

> -----邮件原件-----
> 发件人: Donald Eastlake [mailto:[email-masked]]
> 发送时间: 2010年9月2日 0:19
> 收件人: Xu Xiaohu
as
is
mapped
trees,
plane?
> that a small number tree will be enough even for very big networks.

IMHO, whether or not it is enough depends on many factors, e.g., network
scale, network topology and location distribution of multicast
senders/receivers etc.

Take the following simplest topology as an example, if RB1 is elected as the
root of the only tree on the trill network, RB2 and RB3 each will calculate
a SPF tree with RB1 as root, as a result, the link between RB2 and RB3 will
be blocked in the only distribution tree. Assume the multicast sender is
attached to RB2 while the only multicast receiver is attached to RB3, the
multicast traffic would have to be relayed by the root RB1. It is not
optimal than the tree which is rooted at the ingress RB. If there were
multiple ingress RBs in a large trill network, how could you ensure that the
root of the only tree is close to every ingress RB?

             +-----+
             | RB1 |
             *-----*
            /       \
           /         \
          /           \
         /             \
        /               \
       /                 \
     +/----+          +---\-+
     | RB2 +----------+ RB3 |
     +-----+          +-----+

> distribution tree. It is the ingress RBridge that makes the decision.

Yes, here I just list two BASIC possible options for building optimal
multicast distribution trees; one is to build a tree per VLAN (even per
<VLAN+multicast address> pair) which only carries traffic for the dedicated
VLAN, the other is to build a tree per edge RB which carries traffic for all
VLANs. In the former, the transit RB would no more need to prune the
multicast traffic according to the inner VLAN of the traffic and the set of
VLANs reachable downstream per adjacency. In the latter, the multicast
traffic is forwarded along the optimal distribution path, but the transit RB
would have to prune the multicast traffic according to the inner VLAN of the
traffic and the set of VLANs reachable downstream per adjacency. Certain
topology change will cause each transit RB to update its pruning filters
accordingly for each tree.

Yes, you can choose other variations, e.g., a single tree in a trill network
which carries multicast traffic of all VLANs.

network,
to
usage
> are usable for all VLANs.

OK. 

> groups and a very large number of trees would involve some difficulty,

Agree.

> which provide tradeoffs between various network metrics.

Yes, it is a tradeoff.

such
that
Tree
> distribution tree could be a leaf node of that tree. The best thing to

Maybe you wanted to say the ingress RB could be a leaf node of that tree,
right?

> multi-path multi-destination frames from these heavy sources while

Agree.

could
associated
PIM-SM
> rest of your acronym blizzard to respond properly.

RPT (RP tree) can be switched automatically to the SPT (Source Path Tree).
In particular, if the egress PE knows the addresses of the ingress PEs
belonging to the same MVPN, such egress PE could directly use PIM-SSM to
construct the SPTs rooted at each ingress PE.

> so more "proven" than those you cite. That does not mean we should

If the proven technologies are still valuable, we should reuse them rather
than spending a lot of time to invent and verify a totally new technology,
unless the new technology has many charming advantages over the old ones.

Best wishes,
Xiaohu

lookups,
could
chips
PING
to
proven

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 1 Sep 2010 23:55:51 -0400

Hi Xiaohu,

2010/9/1 Xu Xiaohu <[email-masked]>:
>> > ...

Yes.

> root of the only tree is close to every ingress RB?

Yes, as I say in other messages, in the general case you do not get
shortest paths unless you calculate a tree rooted at each RBridge.
(This is not always true. For example, if RB1 is a leaf node that
connects only to RB2 and RB2 is multiply connected to the rest of the
campus, then a tree rooted at RB1 and a tree rooted RB2 could be
identical. In that case a multi-destination free sent on either tree
would follow the same paths and there would be no need to compute the
two trees rooted at both RB1 and RB2.)

> VLANs.

I don't understand in what sense a tree per VLAN is "optimal". If you
want to use shortest paths, you won't get that with a tree per VLAN. I
also don't understand why you bothered to say that a tree per VLAN is
too many trees to reasonably calculate when TRILL has already decided
that TRILL distribution tree are independent of VLAN.

> In the former, the transit RB would no more need to prune the
> VLANs reachable downstream per adjacency.

Maybe it would eliminate VLAN pruning information, but the tree
calculation effort goes way up and doesn't the Reverse Path Forwarding
Check information also grow even more?

> In the latter, the multicast

It is all a matter of trade-offs. We seem to agree that you need lots
of trees to be optimal, so many that for large networks calculating
and maintaining the combination of the tree forwarding information,
tree pruning information, and reverse path forwarding check
information would be a large burden. So most large networks will have
fewer trees and not be perfectly optimal.

> Maybe you wanted to say the ingress RB could be a leaf node of that tree,

No, I said what I wanted to say.

It is true that an ingress RBridge that ingresses a multi-destinaiton
frame might be a leaf node of the distribution tree it chooses to use
or might not be a leaf node of that tree. It is also true that the
root RBridge of a distribution tree might be a leaf node of that tree
or might not be a leaf node of that tree. Certainly, if the root
RBridge is a leaf node the campus, it must be a leaf node of any tree
for which it is root. Probably such an RBridge is not a particularly
good root and probably such a campus is not a particularly good
topology but TRILL permits such an RBridge to be used as a root if you
want and if you don't configure your RBridges at all, such an RBridge
in such a campus could be chosen by default.

> construct the SPTs rooted at each ingress PE.

You are explaining why PIM-SM is *much* more complicated than TRILL.
Just as before, if you calculate more and more trees, distribution
gets more optimal but the computational burden goes way up.

> unless the new technology has many charming advantages over the old ones.

TRILL is simpler and more efficient and, whatever effort it took to
design the TRILL base protocol specification, it has already been
done.

Thanks,
Donald

---

From: Xu Xiaohu <[email-masked]>
Date: Thu, 02 Sep 2010 14:57:07 +0800

Hi Donald,

> 发送时间: 2010年9月2日 11:56
> > traffic is forwarded along the optimal distribution path, but the
transit
> RB

PIM-based multicast distribution tree is directly calculated upon the
unicast routing table. Hence, no matter how many distribution trees are
there, the PIM router doesn't need to calculate and maintain as many
separate SPF trees as distribution trees as the trill rbridge does. It is
why I argued that trill is much more complicated than PIM-SM or PIM-SSM. For
example, if each VLAN is associated with a distinct IP multicast group
address G in the transit or backbone network, each PE attached to a given
VLAN could join the <*, G> RPT associated to that VLAN, once a packet is
received from that RPT, the egress PE could automatically trigger the
switchover from RPT to SPT. Alternatively, if a PE attached to a VLAN has
already known all other PEs attached to the same VLAN, such PE could
establish <S, G> SPTs from the beginning. For transit devices (i.e., P
routers), the multicast behavior is much simpler.

Best wishes,
Xiaohu

RBridges
ingress
reason
other
and