Subject: [Rift] R: Re: Negative Disaggregation

From: Tommaso Caiazzi <[email-masked]>
Date: Fri, 10 Apr 2020 17:58:05 +0200

Dear all,

we are thinking to schedule the call on Tuesday, April 14th at 18:00 Rome time (that is 10:00 CST, 9:00 PDT and 13:00 UYT).

Is it ok for you?

If there are any problems with the time we can reschedule it.

Thanks for all and best regards,

Tommaso and Mariano

---

From: Bruno Rijsman <[email-masked]>
Date: Fri, 10 Apr 2020 10:05:25 -0600

That works for me.  Please send an invite with call-in details.

PS: I could be wrong, but I think that is 11:00 CST

> On Apr 10, 2020, at 9:58 AM, Tommaso Caiazzi <[email-masked]> wrote:
> 
> Dear all, 
>

---

From: Tony Przygienda <[email-masked]>
Date: Fri, 10 Apr 2020 13:05:29 -0700

I'm cool. from bitter experience send a calendar invite. the calendaring
tools are best to make srue all the time savings & zones are calculated
correctly

--- tony

On Fri, Apr 10, 2020 at 8:58 AM Tommaso Caiazzi <[email-masked]>
wrote:

> Dear all,
>
>
>
> we are thinking to schedule the call on Tuesday, April 14th at 18:00 Rome
> time (that is 10:00 CST, 9:00 PDT and 13:00 UYT).
>
> Is it ok for you?
>
>
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 10 Apr 2020 21:14:51 +0000

Good for me as well!

Regards,

Pascal

Le 10 avr. 2020 à 17:58, Tommaso Caiazzi <[email-masked]> a écrit :

Dear all,

---

From: Jeff Tantsura <[email-masked]>
Date: Fri, 10 Apr 2020 14:43:28 -0700

Thanks, works for me.

Cheers,
Jeff

---

From: Bruno Rijsman <[email-masked]>
Date: Thu, 23 Apr 2020 12:13:47 -0600

While testing negative disaggregation with Mariano, I discovered an "interesting" behavior in RIFT and I would like to check whether this behavior is expected or not. I am not saying the behavior is wrong, I was just surprised to see it and wanted to check whether this was intended when the protocol was designed.

Consider the following topology:

 TOF1     TOF2
   \       /
    \     /
     \   /
     SPINE
     /   \
    /     \
   /       \
 LEAF1  LEAF2

All adjacencies are 3way.

The TOFs have top-of-fabric level 24.

The leaves have level 0.

The spine uses ZTP to derive a level, so it ends up with level 23.

Now we break all links between the spine and its TOFs:

 TOF1     TOF2
   \       /
    X     X   <<< These adj go down (as expected)
     \   /
     SPINE
     /   \
    /     \   <<< Surprise: these adj also go down
   /       \
 LEAF1  LEAF2

At this point the spine node does not have any Valid Offered Levels (VOLs) because (see [1]):
(1) The levels from the TOFs are not valid because the adjacencies are down
(2) The levels from leaves are never accepted as valid offers

Hence, the derived level for the spine becomes "undefined".

Once the level of the spine becomes undefined, it stops accepting LIE messages from the leaves because a node whose level is undefined never accepts LIEs (see [2] below).

Since the spine stops accepting LIEs from the leaves, the old LIEs time out and the adjacencies go down.

My question is: it is normal and expected that breaking the links from the spine to the TOFs *also* causes the adjacencies between th spine and the leaves to go down?

[1] On page [86] of draft-11 it says:

 Valid Offered Level (VOL): A neighbor’s level received on a valid
 LIE (i.e. passing all checks for adjacency formation while
 disregarding all clauses involving level values) persisting for
 the duration of the holdtime interval on the LIE. Observe that
 offers from nodes offering level value of 0 do not constitute VOLs
 (since no valid DERIVED_LEVEL can be obtained from those and
 consequently ‘not_a_ztp_offer‘ MUST be ignored). Offers from LIEs
 with ‘not_a_ztp_offer‘ being true are not VOLs either. If a node
 maintains parallel adjacencies to the neighbor, VOL on each
 adjacency is considered as equivalent, i.e. the newest VOL from
 any such adjacency updates the VOL received from the same node.

[2] On page [45] of draft -11 it says:

 4. PROCESS_LIE:
 1. if lie has wrong major version OR our own system ID or
 invalid system ID then CLEANUP else
 2. if lie has non matching MTUs then CLEANUP, PUSH
 UpdateZTPOffer, PUSH MTUMismatch else
 3. if PoD rules do not allow adjacency forming then CLEANUP,
 PUSH PODMismatch, PUSH MTUMismatch else
 4. if lie has undefined level OR my level is undefined OR this
 node is leaf and remote level lower than HAT OR (lie’s level
 is not leaf AND its difference is more than one from my
 level) then CLEANUP, PUSH UpdateZTPOffer, PUSH
 UnacceptableHeader else

---

From: Tony Przygienda <[email-masked]>
Date: Thu, 23 Apr 2020 14:27:18 -0700

Absolutely intendend and expected behavior ;-) You can't figure out
topology from spine down basically without a ToF. ToFs are the only source
of truth where north is and if that isn't around no'one knows which way is
up ...

It's actually easy to put some code in that watches for the condition on a
node & complains that there are no valid offers.

Alternative would be to "store" offers and use them even if a node went
away. Think through that, it's impossible to converge the protocol
correctly then without having something like flooding of offers with seqnr#
and so on ...

--- tony

On Thu, Apr 23, 2020 at 11:13 AM Bruno Rijsman <[email-masked]>
wrote:

> While testing negative disaggregation with Mariano, I discovered an
> "interesting" behavior in RIFT and I would like to check whether this
> behavior is expected or not. I am not saying the behavior is wrong, I was
> just surprised to see it and wanted to check whether this was intended when
> the protocol was designed.
>
> Once the level of the spine becomes undefined, it stops accepting LIE
> messages from the leaves because a node whose level is undefined never
> accepts LIEs (see [2] below).
>
> *My question is: it is normal and expected that breaking the links from
> the spine to the TOFs *also* causes the adjacencies between th spine and
> the leaves to go down?*
>
>  the duration of the holdtime interval on the LIE. *Observe that*
> * offers from nodes offering level value of 0 do not constitute VOLs*
> * (since no valid DERIVED_LEVEL can be obtained from those and*
> * consequently ‘not_a_ztp_offer‘ MUST be ignored)*. Offers from LIEs
>  4. if lie has undefined level OR *my level is undefined* OR this

---

From: Bruno Rijsman <[email-masked]>
Date: Thu, 23 Apr 2020 15:36:25 -0600

Gotcha.

Thanks for confirming.

PS: It turned out that the issue we were seeing was caused by the fact that Travis CI/CD *still* does not support IPv6 in this day and age.

> On Apr 23, 2020, at 3:27 PM, Tony Przygienda <[email-masked]> wrote:
> 
> Absolutely intendend and expected behavior ;-) You can't figure out topology from spine down basically without a ToF. ToFs are the only source of truth where north is and if that isn't around no'one knows which way is up ... 
> 
> It's actually easy to put some code in that watches for the condition on a node & complains that there are no valid offers. 
> 
> Alternative would be to "store" offers and use them even if a node went away. Think through that, it's impossible to converge the protocol correctly then without having something like flooding of offers with seqnr# and so on ... 
> 
> --- tony 
> 
> On Thu, Apr 23, 2020 at 11:13 AM Bruno Rijsman <[email-masked] <mailto:[email-masked]>> wrote: