Subject: [Rift] cc: from private thread, flooding rules ...

From: Tony Przygienda <[email-masked]>
Date: Sun, 28 Apr 2019 08:18:58 -0700

Benchong,

I'm copy'ing mailing list from the private thread we started through for
posterity ;-) My answers marked with bold >

And yes, very good catch on flooding in first try, you must have really
thought stuff through carefully ;-)

--- tony

------------------------------
*From:* [email-masked] <[email-masked]>
*Sent:* Sunday, April 28, 2019 2:31 AM
*To:* Jeffrey (Zhaohui) Zhang
*Cc:* [email-masked]; Antoni Przygienda; [email-masked];
[email-masked]
*Subject:* Re:RE: Re:I would drive fwd' adoption of drafts ...

1. Something about ZTP

Now the rift protocol ZTP model can auto build rift networks without
configuration(except IP address/IPv6 address/vpn, and ToF level/leaf-2-leaf
as rift-05 5.2.7, and Pod of ToP/Mobility attr).

> *in fact, v6 address can be obtained by ND, yes, ToF must be set (there
is no way what it up or down in a fabric otherwise). Yes, PoD of top is the
same discussion as ToF. I'm not aware we need any configuration for
mobility attributes *

Node or some links status changing may cause level change and then more
nodes need rebuild the neighbors, which break the REQ10 and REQ12.

NODE1-----NODE2

|         |

|         |

NODE3-----NODE4

|

|

NODE5 ......

|

|

    NODE6 ......

For example: NODE1 AND NODE2 level = 10, then after ZTP, NODE3 level = 9,
NODE4 = 9, NODE5 = 8, NODE6 = 7.

 When link between NODE1 and NODE3 break, after ZTP, NODE3 level = 8, NODE5
= 7, NODE6 = 6.

 Link unstable case network unstable. In the above example, Link between
NODE1 and NODE3 unstable will case NODE3/NODE5/NODE6 unstable.

I think rift should support two stage, ZTP and STABLE.

*> your picture fell apart in the formatting so I'm trying to reconstruct
or you have to resend it in fixed width ASCII (or just join the weekly on
Zoom if you can and we'll draw). I don't know how Node4 would be 9 in this
stage, the horizontal link you draw would be really an uplink for node 4
and with that node 4 would be @ level 8. Also I see two links between NOde1
and node 3. If node 2 is level 10 and node 3 has link to node 2 then node 3
would not change level. *

All node learned its level and PoD in ZTP stage. And they should not change
level In STABLE stage when HAL change, or they can get level and PoD
configuration by KV tie in any stage,

*> Yes, it would be possible to configure PoDs by KV but I don't see that
as being much different from configuring it on the device itself. The KV
would need to be directed so the node knows it is being targeted. Level you
cannot get from KV ;-) since you can't form 3-way until you have level
(observe that LIEs carry the level offers and with that levels are
negotiated to form 3-way). And you can't flood KV until you have an
established topology. *

2、KV tie can also take IP address/IPv6 address/vpn from ToF to others, and
the ToF get all networks configuration by Netconf or any other way.

*> yes, all addressing could be configured via KV but that's lots info on a
large fabric and again, I don't see a benefit compared to configuring the
device itself. Generally, information that you want "gathered" @ top of the
fabric like e.g. ARP/IP bindings or properties of nodes in fabric is helped
by KV, stuff that is node specific like "this node needs this ID" is not
very much since you flood part/all of fabric with it with no use & extend
the blast radius. Also, information that needs propagating south through
all or part fabric is helped by KV, e.g. key rollovers. *

KV tie should at least include

type

key-length

key-value

value-length

value

*> well, no ;-) *

   1. *schema itself contains the length already, both key and value are
   strings to be interpreted as seen fit*
   2. *type is not important, it's a string and the implementation can
   parse it any way it wants. There is no value is having e.g. a union
   allowing for all possible thrift types IMO. *
   3.
*As I said since a bit, what we need is a well-known K/V document that
   reserves stuff lots of people want as fair I see like ARP/IP bindings and
   then something where a key can be used by a vendor using his OUI prefix,
   something like well-known "OUI:XXXX:..." *

3.There are some other questions

C.3.2.2

6.  if DBTIE.HEADER < HEADER then

           I)    if originator is this node then bump_own_tie else

                 i.     if this is a N-TIE header from a northbound

                        neighbor then override DBTIE in LSDB with HEADER

                 ii.    else put HEADER into REQKEYS

C.3.3.2

   a.  for every HEADER in TIRE do

       1.  DBTIE = find HEADER in current LSDB

       2.  if DBTIE not found then do nothing

       3.  if DBTIE.HEADER < HEADER then put HEADER into REQKEYS

Why not put DBTIE.HEADER but put HEADER into REQKEYS

*> well, DBTIE.HEADER is _smaller_ than HEADER but we really want to get
the header (in fact its content) from the neighbor so when we request we
have to request the newest HEADER.  But yes, albeit IMO rather confusing,
putting DBTIE.HEADER into REQKEYS would also work since we would request
from neighbor an obsolete header which he would answer with the newest TIE
version. *

C.3.4.  b.

      3.  if DBTIE.HEADER > TIE.HEADER then

           i.     if DBTIE has content already then TXTIE = TIE

Why not TXTIE = DBTIE but TXTIE = TIE

*> right on ;-) and accolades since you found an oversight in flooding
rules write-down that good amount of people fine-combed already and are
implemented at least twice (and you seem to be doing it at least in a third
version ;-) The protocol would stabilize nevertheless on TIDEs as far I see
but still, a very good catch. Yes, this must be DBTIE (in fact, would you
mind to check Bruno's python code for this omission, please). I will
correct & add you to accolades section ;-) *

---

From: Tony Przygienda <[email-masked]>
Date: Tue, 30 Apr 2019 07:06:06 -0700

On Tue, Apr 30, 2019 at 2:02 AM <[email-masked]> wrote:

> Tony,
> Thanks for your reply.

Xu, my pleasure ;-)

>
> 1. It's not a standard Fat Tree,  NODE3 has only one north neighbor NODE1,
> and NODE4 has a north neighbor NODE2. So when link between NODE1 and NODE3
> be down, HAL of NODE3 changes from 10 to 9.
>

ok, what's the link nod3-node4 and node1-node2? horizontal links? if node3
is down, it cannot have a HAL so I try to still interpret it. Basically, if
you are thinking about Node1 _OR_ Node2 going away then the other node will
always seed. let's assume node1 died, then you'll see

Node2
|
Node4
|
Node3

hierarchy being built which is intended. if e.g. node3 goes down, your
network is partitioned. there will be two networks

node1-node2
 |
node4

rest will not be able to obtain a seed (basically without a ToF node you
can't build up a fabric using ZTP since you simply don't know what is up
and what is down)

>
> NODE1-----NODE2
>
> |                 |
>
> |                 |
>
> NODE3-----NODE4
>
> |
>
> |
>
> NODE5 ......
>
> |
>
> |
>
> NODE6 ......
>
>
> ZTP is very good for network initialization. If ZTP is working all the
> way, it may be used limited for security reason. Any new higher level node
> connect in network or last hightest level neighbor get down may cause
> network rebuild. It will easy to be attacked, so I think it's better to
> work in a node start stage.
>

I think the security section explains very well the trade-off between
security needs and ZTP possible ... This is nothing special for RIFT, it's
universal.

>
> 2. Aboute mobility attributes, the leaf is dispatch address route with 32
> mask and connect route with shorter mask, wether is it support mobility
> prefix can be controlled.
>

I can't parse that really. Does RIFT support overlapping prefixes? Yes,
sure. if the /32 moves then it will be a more specific match. The problem
is obviously here if traffic forwarded out a PoD hits a match southbound on
aggregate while the more specific moved into another PoD. Then it will
greedily blackhole. Again, nothing specific for RIFT really, aggregates
blackhnole if they attract traffic they cannot route. Multiple solutions
exist. Such prefixes could be leaked south e.g. but question is really, WHY
would you need aggregates since RIFT automatically aggregates/de-aggregates
for you in sufficient fashion. An example would be good.

>
> 3. >  Yes, it would be possible to configure PoDs by KV but I don't see
> that  as being much different from configuring it on the device itself. The
> KV  would need to be directed so the node knows it is being targeted. Level
>  you cannot get from KV ;-) since you can't form 3-way until you have
>  level (observe that LIEs carry the level offers and with that levels are
>  established topology.
> If we support get configuration by KV tie, the network can be easy control
> by ToF node manual or automatic. ToF(s) can learn network topology by N-TIE
> and send node level and pods by KV tie to them after ZTP stage to confirm
> the network stable. Or the network can be centrol controlled by TOF, The
> controler don't need rember the ip addr of every node.
>

Well, you CAN'T send level from ToF since you won't have 3-way adjacencies
and with that flooding ;-) You need ZTP working first which will establish
level and then you _could_ send PoD to specific nodes via KV, that's true
but it's really _per_ node while KV is pruned flooding so it's bit tricky
since you need to flood to the "level of the PoD" and then prune. But
again, picutre/example would be good.

>
> 4. About the KV struct,  I am not fit the thrift very well, and I'll learn
> about it:-)
>
sure, thrift is widely deployed, stable, elegant, powerfull and easy so
it's worth learning ;-)  It vacuums the floors as well but that's a hidden
feature :-P

>
>
> 5. I have checked Bruno's python code, there is no this problem
>
> process_rx_tie_packet_info
>
>             elif comparison > 0:
>
>                 # We have a newer version of the TIE, send it
>
>                 start_sending_tie_header = db_tie_packet.header
>
>
thanks for that. Like I said, good amount of people were in the weekly
meetings & we chewed the flooding rules over and over again ;-) while they
have been implemented to make sure we don't miss stuff. So we probably
talked through that in detail, agreed and then it was me who miswrote then
in the throes of editorship'ing ;-) on one of the iterations since both
implementations did the correct thing while the spec was wrong as you
pointed out. Again, great catch ;-)

--- tony