Subject: Questions about draft-sajassi-l2vpn-vpls-bridge-interop-00

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 16 Nov 2004 16:56:29 -0500

1) Section 8 of draft-sajassi-l2vpn-vpls-bridge-interop-00 says "And if the CE devices belonging to an ESI are IEEE bridges, then a partial-mesh of PWs can cause broadcast storms in the customer and provider networks"

I understand that when a MAC address becomes unreachable, new messages destined for that MAC address will be replicated over all links, but does that qualify as a broadcast storm? Or is there more to it than that?

2) The next sentence in section 8: "Furthermore, it can cause multiple copies of a single frame to be received by the CE and/or PE devices"

How can that happen?

3) A few paragraphs down: "Therefore, the detection time may be longer than the detection time needed for loop prevention inside the customer network"

Apparently, partial mesh can lead to a loop in the customer network. I guess this is a consequence of the interworking between STP and VPLS, but don't understand the details. Is this easy to explain or should I take a course in STP?

4) Section 10 "For example, if a service instance is associated with four CEs at four different sites, then the maximum number of provider bridges (or PEs), that need to participate in that customer MAC address learning, is only three regardless of how many PEs are in the path of that service instance."

I don't understand that. If four CE2 are connected to four PEs with a full-mesh of PWs between the PEs, don't all four PEs need to participate in MAC address learning?

That's all (for now).

Peter

---

From: Ali Sajassi <[email-masked]>
Date: Tue, 16 Nov 2004 22:29:24 -0800

Peter,

Please refer to my comments below ...

At 04:56 PM 11/16/2004 -0500, Busschbach, Peter B (Peter) wrote:
>1) Section 8 of draft-sajassi-l2vpn-vpls-bridge-interop-00 says "And if 
>the CE devices belonging to an ESI are IEEE bridges, then a partial-mesh 
>of PWs can cause broadcast storms in the customer and provider networks"
>
>I understand that when a MAC address becomes unreachable, new messages 
>destined for that MAC address will be replicated over all links, but does 
>that qualify as a broadcast storm? Or is there more to it than that?

It is more than that. The type of broadcast storm that I am talking about 
is a permanent one as the result of a loop inside the customer network. 
When a customer site is dual-homed (even if its CE is singled-homed) or 
when it has backward connections, then a failure of PW in one direction can 
result in a loop within the customer network.

>2) The next sentence in section 8: "Furthermore, it can cause multiple 
>copies of a single frame to be received by the CE and/or PE devices"
>
>How can that happen?

One of the scenario that this can happen is during n-PE redundancy of 
section 9.2 of vpls-ldp draft. In this case, when the PW that carries the 
provider BPDU traffic between two n-PEs fail, then both n-PEs go into 
forwarding state and both forward the traffic of a given customer (VPLS) 
over their corresponding PWs. The receiving PE, receive two copies of each 
packets that gets brodcasted.

>3) A few paragraphs down: "Therefore, the detection time may be longer 
>than the detection time needed for loop prevention inside the customer network"
>
>Apparently, partial mesh can lead to a loop in the customer network. I 
>guess this is a consequence of the interworking between STP and VPLS, but 
>don't understand the details. Is this easy to explain or should I take a 
>course in STP?

Well, lets give it try first :-) In the bridge world, the Ethernet link 
failure is not uni-directional  (if it fails, it fails in both direction). 
Furthermore, the bridge finds out about the link failure immediately and 
does its failure recovery (by switching to alternate links). In IEEE 
bridges, if a bridge stops receiving BPDUs over a link but doesn't get any 
failure indication for that link, it puts that link in the forwarding state 
(from its previous blocked state). Now in case of a uni-directional failure 
of a PW, if the detection & recovery time for a PW failure is greater than 
twice the BPDU hello time, then it can result in a loop inside the customer 
network.

>4) Section 10 "For example, if a service instance is associated with four 
>CEs at four different sites, then the maximum number of provider bridges 
>(or PEs), that need to participate in that customer MAC address learning, 
>is only three regardless of how many PEs are in the path of that service 
>instance."
>
>I don't understand that. If four CE2 are connected to four PEs with a 
>full-mesh of PWs between the PEs, don't all four PEs need to participate 
>in MAC address learning?

Let me give you another example: if three customer CEs are connected via 10 
provider bridges (e.g., there are ten bridges in the path of these three 
CEs), then only a single bridge needs to do MAC address learning. The 
reason for this is that the data path through other nine bridges along the 
end-to-end path is PtP and thus no MAC address learning is needed for those 
nine bridges. IEEE 802.1ad does this by using the GVRP and disabling MAC 
address learning for the VLANs that have only a single ingress and egress 
point on that bridge.

>That's all (for now).

I am glad that you are going through these issues. I think these are 
important issues that providers need to be aware of when offering 
multipoint Ethernet service.

-Ali

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Wed, 17 Nov 2004 10:50:27 -0500

Hi Ali,

Thanks for your response. If you don't mind, I do have a few follow-up questions.

Peter

> -----Original Message-----
> From: Ali Sajassi [mailto:[email-masked]]
> Sent: Wednesday, November 17, 2004 1:29 AM
> To: Busschbach, Peter B (Peter); [email-masked]
> Subject: Re: Questions about 
> draft-sajassi-l2vpn-vpls-bridge-interop-00
> 
[snipped]

> packets that gets brodcasted.

Unfortunately, I don't fully understand section 9.2 of the vpls-ldp draft. Perhaps you could shed your light on that as well :-)   Section 9.2 mentions:

"If an island has more than one PE-rs, then a dedicated full-mesh of pseudowires is used among these PE-rs devices for carrying the SP BPDU packets for  that island."

What is meant by "dedicated"? Since the PE-rs devices are part of the VPLS network, there is by definition a full mesh of PWs between them. Are the dedicated PWs for SP BDPU packets different PWs than the ones that carry the data traffic?

Also, would it not be possible that as a result of running MSTP in the P-VPLAN, the VPLS network itself is blocked? E.g. if an MTU-s is connected to both PE1 and PE2 and the resulting spanning tree is such that the link between PE1 and PE2 is blocked (i.e. the MTU-s is the root), then MTU-s will forward packets with unknown MAC address received from PE1 to PE2 and vice versa. Now, if PE1 and PE2 both block all traffic from the VPLS network, the P-VPLAN is isolated from all remote nodes. On the other hand, if PE1 and PE2 both forward traffic over the VPLS network, there is a loop.

> (from its previous blocked state).

Does a single PW failure in a full-mesh of PWs cause a total interruption of BPDUs? I would think that in a bridge world, every site that participates in a VPLS instance would generate BPDUs. If PE1 stops receiving BDPUs from PE2 due to a PW failure, but if it continues receiving BPDUs from PE3, PE4, etc., why would it change the forwarding state?

> Now in case of a 
> uni-directional failure 
> twice the BPDU hello time,

What is the BPDU hello time?

>  then it can result in a loop 
> inside the customer 
> network.
> 
> 
> CEs), then only a single bridge needs to do MAC address learning.

I still don't understand. If three CEs are connected to three PEs, don't all three PEs need to participate in MAC address learning?

You can avoid any MAC learning in PEs by establishing point-to-point PWs between CEs. However, in a regular VPLS architecture, each PE needs to do MAC learning, right? 

[snipped to the end]

---

From: Ali Sajassi <[email-masked]>
Date: Wed, 17 Nov 2004 22:38:02 -0800

Hi Peter,

[snipped]

>Unfortunately, I don't fully understand section 9.2 of the vpls-ldp draft. 
>Perhaps you could shed your light on that as well :-)   Section 9.2 mentions:
>
>"If an island has more than one PE-rs, then a dedicated full-mesh of 
>pseudowires is used among these PE-rs devices for carrying the SP BPDU 
>packets for  that island."
>
>What is meant by "dedicated"? Since the PE-rs devices are part of the VPLS 
>network, there is by definition a full mesh of PWs between them. Are the 
>dedicated PWs for SP BDPU packets different PWs than the ones that carry 
>the data traffic?

Exactly. These full-mesh of PWs are dedicated to carry the service 
providers BPDUs only and this full-mesh is for the PE-rs devices of that 
island only and not the whole network. So if the whole network consists of 
5 access islands each of which having three PE-rs devices, then each island 
has a full-mesh of three PWs for its BPDU traffic. And if a VPLS instance 
(for a given customer) spans across all the five islands, then that VPLS 
instance needs a full-mesh of 105 PWs (15*7). The idea of having a separate 
full-mesh of PWs for each island's BPDU traffic is to make each island STP 
independent from every other islands.

>Also, would it not be possible that as a result of running MSTP in the 
>P-VPLAN, the VPLS network itself is blocked? E.g. if an MTU-s is connected 
>to both PE1 and PE2 and the resulting spanning tree is such that the link 
>between PE1 and PE2 is blocked (i.e. the MTU-s is the root), then MTU-s 
>will forward packets with unknown MAC address received from PE1 to PE2 and 
>vice versa. Now, if PE1 and PE2 both block all traffic from the VPLS 
>network, the P-VPLAN is isolated from all remote nodes. On the other hand, 
>if PE1 and PE2 both forward traffic over the VPLS network, there is a loop.

This won't happen. The full-mesh of PWs for that islands BPDUs is for the 
virtual bridge port that I draw in the PE model in page 7 of my IETF 
presentation (I have sent my preso to the chairs to be posted). So if you 
take a look at that figure, hopefully things become more clear. When that 
virtual port gets blocked, then all the VPLS instances (or provider VLANs) 
over that port gets blocked as well and therefore, only a single PE-rs will 
be active for any given VPLS instance. Even if there is another link 
between the two PE-rs devices, the MSTP will take care of blocking this 
additional link.

>Does a single PW failure in a full-mesh of PWs cause a total interruption 
>of BPDUs? I would think that in a bridge world, every site that 
>participates in a VPLS instance would generate BPDUs. If PE1 stops 
>receiving BDPUs from PE2 due to a PW failure, but if it continues 
>receiving BPDUs from PE3, PE4, etc., why would it change the forwarding state?

There are scenarios with dual-homing of a customer site or redundant 
connection via another network, where a unidirectional failure of PW can 
result in a loop - it would be easier to show these scenarios via figures. 
Also in the scenario that you describe above, because the CE1 connected to 
PE1 doesn't receive BPDUs directly from the CE2 (connected to PE2), then it 
can miscalculate its path to the root and block its link and open up an 
alternative link through another network which is not desirable.

>What is the BPDU hello time?

typically 2 seconds.

>I still don't understand. If three CEs are connected to three PEs, don't 
>all three PEs need to participate in MAC address learning?
>
>You can avoid any MAC learning in PEs by establishing point-to-point PWs 
>between CEs. However, in a regular VPLS architecture, each PE needs to do 
>MAC learning, right?

Typically yes, all the PEs need to learn MAC addresses but what I am 
talking about is that the MAC address learning can be optimized (per 
802.1ad) such that only the PEs that need to learn MAC addresses, then 
learn MAC addresses. Let me give you another example, if a customer has 
only two CEs connected via a VPLS network, then we can optimized our 
network such that no MAC address learning is needed because the path 
between the two CEs through the network is basically a PtP path. Now if you 
consider three CEs, then there can be only a single Y branch along the path 
between the three CEs and therefore, only the PE with Y branch needs to 
learn MAC address. With this optimization, the question is not if a PE can 
learn customer MAC addresses but whether a PE needs to learn customer MAC 
addresses.

-Ali

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Thu, 18 Nov 2004 15:54:35 -0500

Ali,

Thanks for the detailed reponse. Things are becoming clearer. I discovered that one of the issues that causes much of my confusion is the way Ethernet deals with Spanning Trees. I tend to think of a Spanning Tree as a tree from which entire links are pruned. Instead, in the ethernet world, it is a matter of blocking ports. Someday, hopefully, I will be able to internalize that.

A few things:

Ali> There are scenarios with dual-homing of a customer site or 
Ali> redundant 
Ali> connection via another network, where a unidirectional 
Ali> failure of PW can 
Ali> result in a loop - it would be easier to show these 
Ali> scenarios via figures.

It would be very helpful for me to see a detailed example that shows exactly what can go wrong and why. If at some point in time you develop such material that you are willing to share, I would greatly appreciate it.

Ali> Now if you 
Ali> consider three CEs, then there can be only a single Y 
Ali> branch along the path 
Ali> between the three CEs and therefore, only the PE with Y 
Ali> branch needs to 
Ali> learn MAC address. With this optimization, the question is 
Ali> not if a PE can 
Ali> learn customer MAC addresses but whether a PE needs to 
Ali> learn customer MAC 
Ali> addresses.

With three CEs you can create an architecture in which *none* of the PEs needs to perform MAC address learning: create point-to-point links between the CEs and let one of the CEs function as the hub node.

I understand your point  that you don't necessarily need to do MAC learning in every PE. However, it is not clear to me what the ultimate relevance of that is. With N CEs in a regular VPLS architecture, all N PEs need to do MAC address learning. However, as you point out, you could choose for a different architecture: create a VPLS among a subset of PEs (including the empty subset) and connect all remaining PEs to that subset via point-to-point PWs. These remaining PEs then don't need to perform MAC address learning. By doing this, you trade data plane efficiency off against the number of MAC entries that you store in the network. The question is: do you think this fine-tuning of VPLS is important? It seems to me that the benefit is small and the operational complexity significant.

Peter

P.S.

A completely different question (not just for Ali, but for anyone who reads this email): when I check my own emails in the email archive (e.g. http://www.ietf.org/mail-archive/web/l2vpn/current/msg00793.html ) I see that the lines are not trunkated. Instead, you need to use the scroll bar to scroll to the end of a sentence. This is not the case for emails from others in the same archive. I have tried to solve it and even worked with our IT department, but to no avail.

When you receive my emails, do they appear on your screen as they do in the archive (i.e. with scroll bar)? Any idea what to do about it?

---

From: Ali Sajassi <[email-masked]>
Date: Thu, 18 Nov 2004 18:31:16 -0800

Peter,

At 03:54 PM 11/18/2004 -0500, Busschbach, Peter B (Peter) wrote:
>Ali,
>
>Thanks for the detailed reponse. Things are becoming clearer. I discovered 
>that one of the issues that causes much of my confusion is the way 
>Ethernet deals with Spanning Trees. I tend to think of a Spanning Tree as 
>a tree from which entire links are pruned. Instead, in the ethernet world, 
>it is a matter of blocking ports. Someday, hopefully, I will be able to 
>internalize that.

And the blocking doesn't have to be for all the VLANs - e.g., it can be for 
a group of VLANs by associated that group of VLANs with a one instance of 
spanning tree by running MSTP. Therefore, by allowing multiple spanning 
trees (typically 2 or 3), you can have a nice load balancing across 
different links.

>It would be very helpful for me to see a detailed example that shows 
>exactly what can go wrong and why. If at some point in time you develop 
>such material that you are willing to share, I would greatly appreciate it.

Sure, I will work on it and will add it as an appendix to my bridge-interop 
draft.

>With three CEs you can create an architecture in which *none* of the PEs 
>needs to perform MAC address learning: create point-to-point links between 
>the CEs and let one of the CEs function as the hub node.

True, but this is no longer a VPLS service and instead it is a VPWS service.

>I understand your point  that you don't necessarily need to do MAC 
>learning in every PE. However, it is not clear to me what the ultimate 
>relevance of that is. With N CEs in a regular VPLS architecture, all N PEs 
>need to do MAC address learning. However, as you point out, you could 
>choose for a different architecture: create a VPLS among a subset of PEs 
>(including the empty subset) and connect all remaining PEs to that subset 
>via point-to-point PWs. These remaining PEs then don't need to perform MAC 
>address learning. By doing this, you trade data plane efficiency off 
>against the number of MAC entries that you store in the network. The 
>question is: do you think this fine-tuning of VPLS is important? It seems 
>to me that the benefit is small and the operational complexity significant.

This depends on what flavor of VPLS we are talking about. With VPLS H-VPLS 
with either QinQ or MPLS access network, you will have at least four 
bridging nodes along the path of any two CEs. So if a customer has three 
sites as they are connected to the provider network as follow:

CE1 ----- u-PE1---------n-PE1 -----------------------n-PE2 
--------------u-PE2 ------ CE2
                                                                    |
                                                                    |--------------------u-PE-3 
------ CE3

then the only PEs that need to learn MAC addresses for that customer is 
n-PE2. Also when you use QinQ access network (802.1ad), then you can take 
advantage of 802.1ad protocols for figuring out which PE doesn't need to 
learn customer MAC addresses and disable MAC address learning on those PE 
dynamically without any configuration what so ever. This is specified in 
802.1ad standard. With MPLS access network one can also leverage the 
802.1ad protocol as well and achieve the same result. So even with MPLS 
access network you will setup the PWs for a give customer as before and 
then run the .1ad protocol on your PEs to decided which PEs should disable 
MAC address learning for which customers. This should disabling feature 
should be transparent and orthogonal to your PW setup.

-Ali

>Peter
>
>
>P.S.
>
>A completely different question (not just for Ali, but for anyone who 
>reads this email): when I check my own emails in the email archive (e.g. 
>http://www.ietf.org/mail-archive/web/l2vpn/current/msg00793.html ) I see 
>that the lines are not trunkated. Instead, you need to use the scroll bar 
>to scroll to the end of a sentence. This is not the case for emails from 
>others in the same archive. I have tried to solve it and even worked with 
>our IT department, but to no avail.
>
>When you receive my emails, do they appear on your screen as they do in 
>the archive (i.e. with scroll bar)? Any idea what to do about it?