Subject: [Rift] Fw:New Version Notification for draft-zwx-rift-leaf-ring-00.txt

From: <[email-masked]>
Date: Thu, 8 Jul 2021 09:06:20 +0800 (CST)

Hi, 
A new draft for RIFT deployment and proposed extension is submitted. 
Much appreciate if you spend some time to read this draft. 
Any comments are welcome.
Thanks,
Sandy

------------------原始邮件------------------
发件人：[email-masked]
收件人：徐本崇10065053;徐本崇10065053;魏月华00019655;张征00007940;
日 期 ：2021年07月08日 08:48
主 题 ：New Version Notification for draft-zwx-rift-leaf-ring-00.txt

A new version of I-D, draft-zwx-rift-leaf-ring-00.txt
has been successfully submitted by Zheng Zhang and posted to the
IETF repository.
Name:        draft-zwx-rift-leaf-ring
Revision:    00
Title:        Supporting leaves without northbound neighbors connecting to a fat-tree network using RIFT
Document date:    2021-07-07
Group:        Individual Submission
Pages:        8
URL:            https://www.ietf.org/archive/id/draft-zwx-rift-leaf-ring-00.txt
Status:         https://datatracker.ietf.org/doc/draft-zwx-rift-leaf-ring/
Html:           https://www.ietf.org/archive/id/draft-zwx-rift-leaf-ring-00.html
Htmlized:       https://datatracker.ietf.org/doc/html/draft-zwx-rift-leaf-ring
Abstract:
This document discusses the usage and solution for leaf nodes without
northbound neighbors connecting to a fat-tree network by leaf nodes
having direct northbound neighbors in RIFT.
The IETF Secretariat

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 26 Jul 2021 21:35:18 +0200

Read that with interest. RIFT is obviously quite applicable for the ring
leaf stuff ;-) and I saw the use case before.

Simplest solution IMO (assuming you're running RIFT on the ring as your
draft does) is to let the hierarchy figure out the CLOS variant here. ToFs
start @ 24 per spec so there is plenty space if that's not enough it's easy
to implement RIFT in a way where ToF start @ a higher number.

So to use your example you will end up with a topology that is a very
fractioned CLOS ;-) but will work just fine given that L05 will get default
from both L04 and L06. Observe that if you end up with L05 and L05' @ the
same level you can enable horizontal link (leaf-2-leaf) but that won't do
much
since default will point northbound and if L05 or L05' loses their
northbound link they will just fall down in hierarchy to X-4 and use the
other L05* as default

So I'm not sure what the draft would buy us over naive use of RIFT to stack
more levels on top of each other. May be I'm missing something but we'll
hear the preso I guess.

-- tony

 |               |  |  |  |  |          | |   |
 | +----------------+  |  +-----------------+ |
 | |             |     |     |          | | | |
 | | +------------------------------------+ | |
 | | |           |     |     |          |   | |
++-+-+--+        | +---+---+ |        +-+---+-++
|       |        +-+       +-+        |        |
|  L01  |--------- |  L02  |----------|  L03   |  Level X-1
+---+---+          +-------+          +---+----+
    |                                     |
    |                                     |
++-+-+--+                             ++--+---+
|       |                             |       |  Level X-2
|  L04  |                             |  L06  |
+---+---+                             +-+-----+
    |                                   |
    +----------------+  +---------------+
                     |  |
                   ++-+-+--+
                   |       |
                   |  L05  |  Level X-3
                   +-------+

On Thu, Jul 8, 2021 at 3:07 AM <[email-masked]> wrote:

> URL:
> https://www.ietf.org/archive/id/draft-zwx-rift-leaf-ring-00.txt
> Html:
> https://www.ietf.org/archive/id/draft-zwx-rift-leaf-ring-00.html
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-zwx-rift-leaf-ring
> Abstract:
> The IETF Secretariat_______________________________________________
> RIFT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rift
>

---

From: <[email-masked]>
Date: Tue, 27 Jul 2021 04:10:27 +0800 (CST)

Hi Tony, 
Thank you very much for your comments!
Yes, we thought about the hierarchical solution before, but it's hard. 
In this figure I simplify the network, in the actual deployment, many nodes like L05 need to access the network. 
If we use the hierarchical levels, there may be many levels here and may exceed the max level limit. 
Do you have any other solution suggestion? Much appreciate for it. :-)
Best regards,
Sandy

------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月27日 03:37
主 题 ：Re: [Rift] Fw:New Version Notification for draft-zwx-rift-leaf-ring-00.txt
_______________________________________________
RIFT mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rift

Read that with interest. RIFT is obviously quite applicable for the ring leaf stuff ;-) and I saw the use case before.

Simplest solution IMO (assuming you're running RIFT on the ring as your draft does) is to let the hierarchy figure out the CLOS variant here. ToFs start @ 24 per spec so there is plenty space if that's not enough it's easy to implement RIFT in a way where ToF start @ a higher number.

So to use your example you will end up with a topology that is a very fractioned CLOS ;-) but will work just fine given that L05 will get default from both L04 and L06. Observe that if you end up with L05 and L05' @ the same level you can enable horizontal link (leaf-2-leaf) but that won't do much
since default will point northbound and if L05 or L05' loses their northbound link they will just fall down in hierarchy to X-4 and use the other L05* as default

So I'm not sure what the draft would buy us over naive use of RIFT to stack more levels on top of each other. May be I'm missing something but we'll hear the preso I guess.

-- tony

|               |  |  |  |  |          | |   | | +----------------+  |  +-----------------+ | | |             |     |     |          | | | | | | +------------------------------------+ | | | | |           |     |     |          |   | |++-+-+--+        | +---+---+ |        +-+---+-++|       |        +-+       +-+        |        ||  L01  |--------- |  L02  |----------|  L03   |  Level X-1+---+---+          +-------+          +---+----+    |                                     |          |                                     |      ++-+-+--+                             ++--+---+ |       |                             |       |  Level X-2|  L04  |                             |  L06  |+---+---+                             +-+-----+    |                                   |    +----------------+  +---------------+                                  |  |                   ++-+-+--+                   |       |                   |  L05  |  Level X-3                   +-------+

---

From: Tony Przygienda <[email-masked]>
Date: Mon, 26 Jul 2021 22:58:07 +0200

Hey Sandy, well, rift has for the ToF level an i8 on purpose. That's 127
levels for you (255 if you want stretch it).

`top_of_fabric_level` being 24 is just an arbitrary constant that looks
plenty for lots of customers but there is nothing stopping you from
implementing configurable/much higher value (we do ;-)

if you try to build a RIP (that's what you're doing BTW with the suggested
solution re-generating routes, basically RIP using a redistribution
protocol ;-) you will run into counting-to-infinity on metric most likely
in complex failure scenarios given the many leaf-hops you have to converge
back and forth on a closed ring. And then introduce poisoned horizon to
help with that and so on.

RIFT hierarchy election on the other hand has been tuned for *very* fast
convergence on bring up and fast convergence on failures where a root is
still present. you will only see slow counting down to 0 (and not up to
infinity) when you loose all the ToFs (and you're fried then anyway l;-)
So roughly speaking, unless you ring has more than 200 devices you should
be fine. Unless your scale is larger than this I don't see the draft can be
doing a better job than hierarchy election in RIFT frankly.

I hope it's not too cryptic, it preconditions good understanding of dynamic
IP routing history & concepts, otherwise I highly recommend reading up RIP
deployment problems & why BGP has as-path vectors to prevent most problems
(albeit it still suffers from hunting in some scenarios prevented to an
extent by protocol violations such as accept-own-as basically and 'illegal'
numbering schemes on CLOS) ;-)  AFAIR Radia's Interconnections book is bit
dated but conceptually still very sound BTW.

-- tony

On Mon, Jul 26, 2021 at 10:10 PM <[email-masked]> wrote:

> In this figure I simplify the network, in the actual deployment, many
> nodes like L05 need to access the network.
> So I'm not sure what the draft would buy us over naive use of RIFT to
> stack more levels on top of each other. May be I'm missing something but
> we'll hear the preso I guess.
>
> -- tony
>
> |               |  |  |  |  |          | |   | | +----------------+  |
> +-----------------+ | | |             |     |     |          | | | | | |
> +------------------------------------+ | | | | |           |     |     |
>       |   | |++-+-+--+        | +---+---+ |        +-+---+-++|       |
>   +-+       +-+        |        ||  L01  |--------- |  L02  |----------|
> L03   |  Level X-1+---+---+          +-------+          +---+----+    |
>                                  |          |
>        |      ++-+-+--+                             ++--+---+ |       |
>                          |       |  Level X-2|  L04  |
>        |  L06  |+---+---+                             +-+-----+    |
>                            |    +----------------+  +---------------+
>                             |  |                   ++-+-+--+
>    |       |                   |  L05  |  Level X-3
>  +-------+
>

---

From: <[email-masked]>
Date: Wed, 28 Jul 2021 05:43:41 +0800 (CST)

Hi Tony, 
Thank you for your detailed explanation! 
In the actually deployment, more than 1 hundred node may be existed in the ring, fortunately it may not exceed 128. :-P
Yes. The solution in this draft used the increased metric to make the path election more better. This part may looks like RIP somewhat. 
We think it's necessary because there are two east-west links for a node, the node may receive the same prefix from each link. The node can choose the path with better metric. 
But you are right, in order to avoid the routing loops, more methods may be needed such as split horizon or poison reverse. 
Let forget the proposed solution in this draft, and think about what may happen if we use RIFT existed mechanism. 
For example: 
   |    |
   A----B
   |    |
   C    D
   |    |
   E    F
   |    |
   G----H
In this figure, only A and B have northbound connection with spine node. And the network use ZTP to get level automatically.
If we run ZTP here, A and B may be level 10, C and D may be level 9, E and F may be level 8, and G and H may be level 7. 
In normal condition the network works. 
But if the link between A and C is broken, the nodes level may be adjusted. 
After the ZTP changing, the final level may be: A and B is level 10, D is level 9, F is level 8, H is level 7, G is level 6, E is level 5, and C is level 4.
So there may be neighbor rebuilding, routing re-advertisement and re-convergence.
If we configure the level in the node to make it unchanged, the ZTP function cannot be used here. The above problems may not happen, but looks like there isn't flexible. Right? 
What do you think of it?
Best regards,
Sandy

------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月27日 04:58
主 题 ：Re: [Rift] Fw:New Version Notification for draft-zwx-rift-leaf-ring-00.txt
Hey Sandy, well, rift has for the ToF level an i8 on purpose. That's 127 levels for you (255 if you want stretch it).

`top_of_fabric_level` being 24 is just an arbitrary constant that looks plenty for lots of customers but there is nothing stopping you from implementing configurable/much higher value (we do ;-)

if you try to build a RIP (that's what you're doing BTW with the suggested solution re-generating routes, basically RIP using a redistribution protocol ;-) you will run into counting-to-infinity on metric most likely in complex failure scenarios given the many leaf-hops you have to converge back and forth on a closed ring. And then introduce poisoned horizon to help with that and so on.

RIFT hierarchy election on the other hand has been tuned for *very* fast convergence on bring up and fast convergence on failures where a root is still present. you will only see slow counting down to 0 (and not up to infinity) when you loose all the ToFs (and you're fried then anyway l;-)  So roughly speaking, unless you ring has more than 200 devices you should be fine. Unless your scale is larger than this I don't see the draft can be doing a better job than hierarchy election in RIFT frankly.

I hope it's not too cryptic, it preconditions good understanding of dynamic IP routing history & concepts, otherwise I highly recommend reading up RIP deployment problems & why BGP has as-path vectors to prevent most problems (albeit it still suffers from hunting in some scenarios prevented to an extent by protocol violations such as accept-own-as basically and 'illegal' numbering schemes on CLOS) ;-)  AFAIR Radia's Interconnections book is bit dated but conceptually still very sound BTW.

-- tony

------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月27日 03:37

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 28 Jul 2021 01:23:59 +0200

I agree with that. ZTP level unfortunately breaks the adjacency, it's in
the design from the bottom up given that level is carried in many TIEs and
massive reconfiguration of levels beside startup should be a rare
occurence. So there is merit in your proposal albeit I think that the
version of RIP you'll build will need propagation through 1/2 ring as well
if C is attracting traffic and it has to flip to D.

Let's discuss further in the group, Sandy

On a private note, I don't see why 5G folks like those rings, they are more
fragile than a simple 2x2 CLOS, they have longer paths and every node added
makes everything more fragile & slow. Maybe it's the only-2-links on a
node, I remember during LTE times the cells were very limited.

-- tony

On Tue, Jul 27, 2021 at 11:43 PM <[email-masked]> wrote:

> In the actually deployment, more than 1 hundred node may be existed in the
> ring, fortunately it may not exceed 128. :-P
> Yes. The solution in this draft used the increased metric to make the path
> election more better. This part may looks like RIP somewhat.
> We think it's necessary because there are two east-west links for a node,
> the node may receive the same prefix from each link. The node can choose
> the path with better metric.
> But you are right, in order to avoid the routing loops, more methods may
> be needed such as split horizon or poison reverse.
> In this figure, only A and B have northbound connection with spine node.
> And the network use ZTP to get level automatically.
> If we run ZTP here, A and B may be level 10, C and D may be level 9, E and
> F may be level 8, and G and H may be level 7.
> After the ZTP changing, the final level may be: A and B is level 10, D is
> level 9, F is level 8, H is level 7, G is level 6, E is level 5, and C is
> level 4.
> If we configure the level in the node to make it unchanged, the ZTP
> function cannot be used here. The above problems may not happen, but looks
> like there isn't flexible. Right?
> What do you think of it?
> Best regards,
> Sandy
>
>
> I hope it's not too cryptic, it preconditions good understanding of
> dynamic IP routing history & concepts, otherwise I highly recommend reading
> up RIP deployment problems & why BGP has as-path vectors to prevent most
> problems (albeit it still suffers from hunting in some scenarios prevented
> to an extent by protocol violations such as accept-own-as basically and
> 'illegal' numbering schemes on CLOS) ;-)  AFAIR Radia's Interconnections
> book is bit dated but conceptually still very sound BTW.
>
> -- tony
>

---

From: <[email-masked]>
Date: Wed, 28 Jul 2021 08:27:32 +0800 (CST)

Hi Tony, 
OK. Let's discuss it further in the group. 
You know, the operator like to update the network 'smoothly'. So they may give up the update plan when the new technology needs them to change the topology.  
BTW, if you have any network figure with the suggestion topology, the actually deployment is more better, could you please send it to me?
Thank you very much!
Best regards,
Sandy

------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月28日 07:24
主 题 ：Re: [Rift] Fw:New Version Notification for draft-zwx-rift-leaf-ring-00.txt
I agree with that. ZTP level unfortunately breaks the adjacency, it's in the design from the bottom up given that level is carried in many TIEs and massive reconfiguration of levels beside startup should be a rare occurence. So there is merit in your proposal albeit I think that the version of RIP you'll build will need propagation through 1/2 ring as well if C is attracting traffic and it has to flip to D.

On a private note, I don't see why 5G folks like those rings, they are more fragile than a simple 2x2 CLOS, they have longer paths and every node added makes everything more fragile & slow. Maybe it's the only-2-links on a node, I remember during LTE times the cells were very limited.

-- tony

------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月27日 04:58
------------------原始邮件------------------
发件人：TonyPrzygienda
收件人：张征00007940;
抄送人：[email-masked];
日 期 ：2021年07月27日 03:37