Subject: Re: [Rift] RIFT protocol implementation

From: Antoni Przygienda <[email-masked]>
Date: Wed, 18 Mar 2020 16:34:29 +0000

Forwarding the discussion to the rift ietf list for further exposure & since I generally think it will be possibly more productive in a wider forum. Roma Tre University is working on Bruno’s open source and there’s a bunch of interesting tools they’re developing as well as you can read below.

--- tony

From: Mariano Scazzariello <[email-masked]>
Date: Wednesday, March 18, 2020 at 9:10 AM
To: Bruno Rijsman <[email-masked]>
Cc: Antoni Przygienda <[email-masked]>, Leonardo Alberro Zimmermann <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, Giuseppe Di Battista <[email-masked]>
Subject: Re: RIFT protocol implementation

Hi everyone,
thanks for the extremely detailed suggestions! I have a lot of stuff which I can use to work on the implementation. Also thanks Tony for giving me further useful tips.

Also, I'm happy that we agree on almost every implementation detail and I agree with your suggested variations (like the flag for the spf_run_direction method).

About point C, I did not write anything since it seems quite easy to extend the Thrift model. If I have any issues, I'll surely ask you for some help. I also wrote a point D (which is how to handle the negative disaggregation in the RIB/FIB when received), but I read the Pascal slides (I add the link<https://urldefense.com/v3/__https:/bitbucket.org/riftrfc/rift_draft/src/master/negative*20disaggregation.pptx__;JQ!!NEt6yMaO-gk!QBFQP3Ec6u0elEKgEFaAEndcjyQntJXvMg0h2TfAH6kn4JLUaMb3FOYpDQOltw$> here, so everyone can access them easily) and the steps used in that presentation are the same that I thought.

I still have some doubts about the special SPF run:
2) We will also need a new member field orig_neg_disagg_prefixes (once again of type set, I think) that contains the negatively disaggregated prefixes that are autonomously being originated based on the detection of fallen leafs based on the difference between the normal and special SPF run.
You are proposing to postpone the RIB/FIB update after the special SPF run (which can detect additional fallen leaf nodes). This is right,  but should this only occur on ToF nodes? If we consider a node X, which is not a ToF, that receives a negative disaggregation TIE, it should only add the prefix in the prop_neg_disagg_prefixes set and check if it received this prefix from all its parent nodes and propagate it if required. Then it should proceed to update its RIB/FIB, without running the special SPF. Am I right?

About the flooding oscillations, Tony writes "Your best protection is scaled, randomized tests". Here at Roma Tre University we developed a tool called Kathará<https://urldefense.com/v3/__https:/www.kathara.org/__;!!NEt6yMaO-gk!QBFQP3Ec6u0elEKgEFaAEndcjyQntJXvMg0h2TfAH6kn4JLUaMb3FOZujrrECA$> which is able to emulate network scenarios using Docker containers. Recently, we also developed a Fat Tree Generator, that automatically generates a fat tree topology starting from the fundamental parameters of a Fat Tree (K_LEAF, K_TOP and R) that can be run in Kathará. It also auto-configures the routing protocol on each node (of course, we also included the RIFT-Python implementation). So we can generate Fat Tree topologies of any size and run tests on it to verify functional and behavioral aspects (and also gather routing information such as PDU size and count). We are also developing a Fat Tree Test Framework (in collaboration with Leonardo and the team at UY university) which implements typical data center network scenarios (such as link failure, node failure, fallen leaf, partitioned fabric and so on) to run integration tests on it. For example, we can check if the routing table of a node is equal to the expected one after a failure (e.g. loss of a multipath or a prefix). With this tool we can surely run randomized tests (at any scale, since Kathará supports Kubernetes) to ensure that no flooding oscillations occur.

I agree to move the discussion on the RIFT WG mailing list. Maybe someone of you should introduce us and explain what we're doing.

Thanks everyone for your time,
Mariano.

---

From: Bruno Rijsman <[email-masked]>
Date: Wed, 18 Mar 2020 14:04:54 -0600

Adding the RIFT mailing list to the CC list, per Tony’s suggestion.

> On Mar 18, 2020, at 1:45 PM, Bruno Rijsman <[email-masked]> wrote:
> 
> D) Actually, there's a way to add a "negative" route in the Linux kernel table: https://superuser.com/a/628385 <https://superuser.com/a/628385>. In this way, you can add an unreachable prefix in the table. Probably is a good idea to put this into the Kernel class and add a method in the FIB class to call it. 
> 
> I am pretty sure that you cannot use the Linux negative route feature.
> 
> Based on the description in your link the Linux negative route does not what we need for RIFT. 
> 
> It appears to be similar to what Junos (for example) calls a "discard route" (https://www.juniper.net/documentation/en_US/junos/topics/example/policy-discard-interface.html <https://www.juniper.net/documentation/en_US/junos/topics/example/policy-discard-interface.html>)
> 
> It is a normal route, subject to the normal longest prefix match rules, but instead of sending matching packets to a real next-hop interface, it discards the packets (perhaps sending an ICMP unreachable message back to the source of the packet).
> 
> This is not at all what we want in RIFT.
> 
> A negative route in RIFT is NOT a discard route. Instead it means "please don't send the packet to me, send it to someone else instead".
> 
> Consider the example of a leaf node L that has 3 spine routers S1, S2, and S3.
> 
> If (only) S1 advertises a negative disaggregate for prefix P, then leaf L must NOT install discard route for prefix P. Instead it must install a (normal positive) ECMP route for prefix P with next-hops S2 and S3.
> 
> Thus, we see that a negative route in the RIB gets translated into a positive route (with complementary next-hops) in the FIB.
> 
> Actually, I came back to this section after reading some of your later comments. There may, in fact, be a use for a discard route (and possibly this is what you hand in mind in the first place). Whenever a node decides that it wants to negatively disaggregate some given prefix P (either as a result of the special SPF or as a result of getting a negative disaggregate for P from all parents) then it would be useful for that particular node to install a discard route for P to avoid packets to P hitting a more general aggregate (typically the default).
> 
> My general idea for this is:
> When receiving a negative disaggregation, we have the prefixes.
> 
> To be more precise: when we receive a (flooded) negative disaggregation TIE that contains some negatively disaggregated prefixes, we store: 
> 
> 1) that TIE in the TIE database and trigger an SPF run for later
> 
> 2) For each prefix in the TIE, we immediately run to logic to keep track of whether or not we received a negative disaggregation for that prefix from EACH of the parent nodes. If that changes (true to false or vice versa) we update our own locally originated negative disaggregation TIE to start/stop advertising that particular prefix.
> 
> For each one of them:
> Get the less specific prefix. As example, we have a ~8.0.1.0/24, and the less specific in the RIB is 8.0.0.0/16.
> Get the Nodes that can reach the less specific prefix (e.g. 8.0.0.0/16 is reachable by node1, node2 and node3)
> From these nodes, remove the node which sent the negative TIE (e.g. node3)
> Now we know that 8.0.1.0/24 is reachable by node1 and node2
> Save this in the RIB and install it in the FIB
> 
> My current instinct (but I am happy to discuss alternatives) is a little bit different, particularly I am thinking of having negative routes as first-class citizens in the RIB and postponing the translation from negative routes to positive routes to when routes are moved from the RIB to the FIB. This is the approach that Pascal described in his slides (attached).
> 
> 
> 
> 
> Currently when we run an SPF, we calculate one (positive) route to each reachable prefix P (that route is typically an ECMP route).
> 
> We need to enhance SPF, so that we treat negative prefixes as separate destination objects from positive prefixes. In other words, if we have two TIEs: one TIE advertising prefix P as a reachable prefix or as a positively aggregated prefix, and another TIE advertising the same prefix P as a negatively aggregated prefix, then two prefixes P are NOT the same destination (as far as the RIB is concerned). We treat them as different destinations, namely P-positive and P-negative. And (once again, only as far as the RIB is concerned) we install two different destinations in the RIB: P-positive (P) and P-negative (~P).
> 
> This line of thinking has some dramatic consequences on the complexity of the code (PS: if these complexities are more than you bargained for when you signed up to implement negative disaggregation, I can help with the coding).
> 
> The first complication is that the code for the RIB (class RouteTable) needs to be generalized. Currently the index for entries in the RouteTable is simply the prefix. This needs to be generalized to (prefix, is_negative) where is_negative is False (for normal positive routes) or True (for negative routes). All the RIB show commands etc. need to be updated to deal with the existence of positive and negative routes.
> 
> The second complication is that we need some logical to translate negative routes to complementary positive routes, as described in in slides 55 through 60 in Pascal's negative disaggregation slides (attached this time, as my bandwidth has improved now). This is going to be some "interesting" code.
> 
> (Tony - I would appreciate some feedback on what you think of this approach.)
> 
> 
> In case there are no more nodes avaiable (point 3), the point 5 installs the "unreachable" route in the kernel table.
> When receiving a "positive TIE", we should do the opposite.
> 
> No, I don't think this is right.
> 
> Be very careful: "receiving a positive TIE" is not the same as "a negative TIE being withdrawn"
> 
> This is exactly why I think it makes sense to treat P-positive and P-negative as separate first-class objects in the RIB.
> 
> For each prefix:
> Get it in the RIB, add the node which sent the TIE (e.g. 8.0.1.0/24 is reachable by node1 and node2, but now from node3 too)
> If the n. of nodes is equal to the n. of nodes that can reach the less specific prefix
> we remove the prefix from the RIB (and from the FIB)
> Example, 8.0.0.0/16 is reachable by node1, node2 and node3. 8.0.1.0/24 is reachable by node1, node2 and node3. Remove 8.0.1.0/24.
> If not, we should add a multipath in the RIB/FIB
> 
> In addition to Tony I have now also included Pascal in the CC list since his negative disaggregation slides play a central role in this discussion (Pascal — let me know if you also want a copy of e-mails discussing the questions in A and B).
> 
> I have not copied the RIFT mailing list, since this discussion is very code specific. But if someone else feels this would useful, feel free to forward as far as I am concerned.
> 
> At this moment, my bandwidth issues are much improved, but the situation changes from day to day.
> 
> — Bruno
> 
> 
> 
>> On Mar 10, 2020, at 12:01 PM, Mariano Scazzariello <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Hi Bruno,
>> sorry for the delay in the reply but I've been digging into the code to understand the implementation more in detail and where I can "inject" the edits.
>> 
>> Here's a list of my questions and notes, related to your previous list:
>> 
>> A) The special SPF should be run only by ToF nodes? Besides that, I have to run an additional southbound SPF, with E-W links. The code to run the SPF is in Node class, spf_run method. This calls the spf_run_direction, with the DIR_NORTH or DIR_SOUTH constant. I was thinking to add a third spf_run_direction, with DIR_EAST_WEST. However, the code of that method should be changed a bit:
>> 
>> When the spf_add_candidates_from_node is called, after doing preliminary operations, it calls the spf_add_neighbor_candidates which has the spf_direction as a param. At this point, the self.node_neighbors(node_ties, spf_direction) is called to get the current node neighbors. My idea is that, when the spf_direction = DIR_EAST_WEST, the node_neighbors method is called with a list [DIR_SOUTH, DIR_EAST_WEST] instead of a single value in order to get both south and E-W neighbors. This needs a little code change in the node_neighbors method to accept a list AND a string, but should be really easy to do.
>> At the end of the SPF computation in spf_run_direction, the spf_install_routes_in_rib is called. This should be wrapped in an if-statement like: if spf_direction != DIR_EAST_WEST, because we don't want to install any route in the RIB after doing this special SPF
>> At this point, after the third SPF execution, I should be able to do a diff between the south and E-W executions and check which leaf nodes are missing and if so, call a method like send_negative_disaggregation to send the corresponding TIEs. If something is missing, we should also install the unreachable leaf nodes in the table.
>> B) I found that each TIE is stored into a "db" called tie_packet_infos. After adding the corresponding Thrift models (point C) for the negative disaggregation TIEs, I should be able to retrieve them from the db. However, I still don't know how to trigger the "negative TIE check". In Node class, in process_rx_tie_packet_info (as far as I understood, this is triggered by the interface sockets when a packet is received) after storing it with store_tie_packet_info, I need a way to check if the received TIE is a negative disaggregation TIE. How can I add a callback? Should I edit the store_tie_packet_info code? After that, is there a method to get the parent nodes for a given node? Or should I add it? In this case, my general idea is:
>> 
>> When a negative TIE is received, do the disaggregation in the local routing table (see point D)
>> Now we have to decide if the TIE should be flooded:
>> Get all parents. To do so, get all the Node TIEs and filter only the ones with level = current_node_level + 1 (is that correct?)
>> Get all negative TIEs for the prefixes contained in the received TIE, and group them by prefix (a dict like "prefix": N_OF_NEG_TIES_FOR_PREFIX should be enough)
>> For each item in the map check if N_OF_NEG_TIES_FOR_PREFIX == N_PARENTS and if so, store it
>> Propagate (southbound) a negative disaggregation TIE for all the prefixes found in the previous step (if the list is not empty)
>> I don't know if this is correct, maybe I've badly understood the negative disaggregation! Expecially point 3 and 4: Negative TIE should be propagated entirely or we can "filter" the unreachable prefixes and send a different TIE?
>> 
>> Of course, another check should be added when a "positive TIE" is received. I can't find where it is described (in section 4.2.5.2.3 the draft says "Obviously, when one of the northbound switches withdraws its negative   advertisement, the node has to withdraw its transitively provided negative prefix as well." How is it withdrawn?) In this case we have to check if the negative TIEs for a given prefix are lesser than the number of parents and "withdraw" it sending something to our neighbors.
>> 
>> D) Actually, there's a way to add a "negative" route in the Linux kernel table: https://superuser.com/a/628385 <https://superuser.com/a/628385>. In this way, you can add an unreachable prefix in the table. Probably is a good idea to put this into the Kernel class and add a method in the FIB class to call it. 
>> 
>> My general idea for this is:
>> 
>> When receiving a negative disaggregation, we have the prefixes.
>> For each one of them:
>> Get the less specific prefix. As example, we have a ~8.0.1.0/24, and the less specific in the RIB is 8.0.0.0/16.
>> Get the Nodes that can reach the less specific prefix (e.g. 8.0.0.0/16 is reachable by node1, node2 and node3)
>> From these nodes, remove the node which sent the negative TIE (e.g. node3)
>> Now we know that 8.0.1.0/24 is reachable by node1 and node2
>> Save this in the RIB and install it in the FIB
>> In case there are no more nodes avaiable (point 3), the point 5 installs the "unreachable" route in the kernel table.
>> When receiving a "positive TIE", we should do the opposite.
>> For each prefix:
>> Get it in the RIB, add the node which sent the TIE (e.g. 8.0.1.0/24 is reachable by node1 and node2, but now from node3 too)
>> If the n. of nodes is equal to the n. of nodes that can reach the less specific prefix
>> we remove the prefix from the RIB (and from the FIB)
>> Example, 8.0.0.0/16 is reachable by node1, node2 and node3. 8.0.1.0/24 is reachable by node1, node2 and node3. Remove 8.0.1.0/24.
>> If not, we should add a multipath in the RIB/FIB
>> Sorry for the long email! Of course, take your time to answer! 
>> 
>> Hope to hear news from you really soon!
>> Mariano.
>> 
>> Il 06/03/2020 01:43, Bruno Rijsman ha scritto:
>>> Hello Mariano,
>>> 
>>> Here is some initial guidance on how to implement negative disaggregation.
>>> 
>>> All references to section numbers are based on draft-ietf-rift-rift-10.
>>> 
>>> Here are the work items:
>>> 
>>> (A) New code to decide to originate a negative disaggregate according to the rules in the 2nd paragraph in section 4.2.5.2.3. This essentially boils down to running an additional "special" SPF where the E-W are considered and compare the result with the normal SPF where the E-W links are not considered. Any extra leaf nodes that are discovered in the special SPF run are fallen leafs that need to be negatively disaggregated. To be honest, the language in the draft on this topic is quite complex and vague, so it would be good to validate my understanding on the mailing list and/or with Tony. One challenge to consider is that the code doing SPFs in the presence of E-W links in RIFT-Python is not tested at all — it needs unit tests. Also, the rules for flooding TIE involving E-W links are very complicated and fragile (this was one of the major areas of interop and stability issues when testing interop with Juniper) - you need to tread very carefully here.
>>> 
>>> (B) New code to decide to originate (propagate might be a better word) negative disaggregates according to the rules in the 3rd paragraph in section 4.2.5.2.3. This will involve keeping track of a reference count of how many parents advertised a negative disaggregate, to propagate the negative disaggregate when the reference count reaches "all parents", and the un-propagate it when the reference count goes down from "all parents".
>>> 
>>> (C) Some small amount of new code for actually encoding, decoding, and flooding negative disaggregate TIEs. This should be almost zero work, since it is no different from the existing code for other types of TIEs.
>>> 
>>> (D) New code to process received negative disaggregates and install the corresponding routes in the FIB, as described in section 4.2.6. The main issue is that there is no such thing as a negative entry in the FIB. Thus, at some point we need to convert the "negative" routes into complementary "positive routes" (compare figure 20 with figure 21). The question is: where should this logic live? Should it live in the RIFT-Python abstraction of the FIB (file fib.py)? Should it live in the RIFT-Python abstraction of the RIB (file rib.py)? Or should it live higher than that, e.g. in the general vicinity of function spf_install_routes_in_rib? I am leaning towards putting in the FIB itself, but this needs more thought and would be a good discussion for a conference call.
>>> 
>>> In general, whatever new code you write, make sure you also create the corresponding automated unit test cases with good coverage of the new code.
>>> 
>>> I am still in pretty remote area, so let’s do e-mail for now until we find a good slot for a call.
>>> 
>>> -- Bruno
>>> 
>>> 
>>>> On Mar 3, 2020, at 3:13 AM, Mariano Scazzariello <[email-masked] <mailto:[email-masked]>> wrote:
>>>> 
>>>> Hi Bruno,
>>>> thanks for the update! I'll start digging in the RIFT Python code today, so I'll be able to better understand your suggestions.
>>>> 
>>>> Hope to hear news from you really soon!
>>>> Mariano.
>>>> 
>>>> Il 03/03/2020 01:23, Bruno Rijsman ha scritto:
>>>>> Hi Mariano,
>>>>> 
>>>>> I am currently in the Northern highlands of Guatemala and tomorrow (2 March) I will start a trek that will take me into some remote areas without Internet or phone.
>>>>> 
>>>>> For sure I will be able to do some conference calls when I return to Antigua in the first week of April.
>>>>> 
>>>>> And quite likely, I will able to do a conference call earlier than that when I reach some more populated areas in the last 2 weeks of March, but I am not sure of the exact dates (I will keep you posted).
>>>>> 
>>>>> I will soon send a separate e-mail with my thoughts about disaggregation so that you have something to go on before the conference call.
>>>>> 
>>>>> -- Bruno
>>>>> 
>>>>>> On Feb 28, 2020, at 10:54 AM, Mariano Scazzariello <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>> 
>>>>>> Hi Bruno,
>>>>>> I'm currently in Italy (so GMT+1). I'm available from Monday to Friday, 10.00-19.00 (GMT+1, of course!)
>>>>>> 
>>>>>> So let's schedule a call (even with Leonardo, since he'll help me in the implementation) when you're available.
>>>>>> 
>>>>>> Thanks,
>>>>>> Mariano.
>>>>>> 
>>>>>> Il 28/02/2020 13:40, Bruno Rijsman ha scritto:
>>>>>>> Hi Mariano,
>>>>>>> 
>>>>>>> I would be happy to do a Skype call to explain the structure of the RIFT code and to give some hints on how to implement negative disaggregation.
>>>>>>> 
>>>>>>> Right now I am in Guatemala, often in places without Internet, so it make take some time to find a good timeslot.
>>>>>>> 
>>>>>>> What timezone are you in, and what are good times for you?
>>>>>>> 
>>>>>>> -- Bruno
>>>>>>> 
>>>>>>> On Fri, Feb 28, 2020, 5:08 AM Mariano Scazzariello <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>>> Hi Bruno,
>>>>>>> 
>>>>>>> as pointed out in the last email by Leonardo, I'll implement the Negative Disaggregation because we are testing the protocol and we need it as soon as possible.
>>>>>>> 
>>>>>>> I'm confident with Python language, but I didn't read any rift-python source files so I don't know how it's structured.
>>>>>>> To avoid conflicts with your code and to speed up the development, I'm asking you if it is possible to organize a call (maybe on Skype) to have some help starting the project.
>>>>>>> 
>>>>>>> Waiting for a response with further details, thanks in advance.
>>>>>>> Mariano
>>>>>>> 
>>>>>>> Il 27/02/2020 19:47, Leonardo Alberro Zimmermann ha scritto:
>>>>>>>> Hi Bruno! sorry for the delay, we have been arguing with the group about the options that we have.
>>>>>>>> 
>>>>>>>> see in line >>>
>>>>>>>> 
>>>>>>>> El 19/2/20 a las 14:00, Bruno Rijsman escribió:
>>>>>>>>> Hi Leonardo,
>>>>>>>>> 
>>>>>>>>> Good to hear from you again!
>>>>>>>>> 
>>>>>>>>> I a very pleased to hear that you are still using my RIFT implementation.
>>>>>>>>> 
>>>>>>>>> About a year ago, I also thought about implementing a RIFT Wireshark dissector myself, but it never bubbled up to the top of my list. I did spend some time on thinking how I would do it, and those thoughts might be useful to you. I was planning to write the dissector in C (as opposed to a dynamically loaded Lua dissector). That was mainly driven by the fact that I am not familiar with Lua. The main challenge that I saw was that I did not want to write the C dissector code for parsing the Thrift packet by hand. I wanted C dissector code for parsing Thrift packets to be generated by the Thrift compiler, just like the RIFT code for parsing Thrift packets is generated. I found a Wireshark C dissector for Thrift in GitHub. I contacted the author and we exchanged some e-mails on how to generalize his code for my RIFT dissector. Those e-mail conversations fizzled out as I never got to spend serious time on this little project. I even had some grandioze ideas of writing a generalized Thrift (yes, Thrift, not RIFT) dissector where the end-user of Wireshark would be able to dynamically load any Thrift model at run-time and then decode whatever packets were described by that particular dynamically loaded Thrift model. That still seems like something worthwhile to do, but maybe "a bridge too far" for RIFT, especially considering that the encapsulation header for RIFT has been becoming increasingly complex over time. But anyway - I would be happy to exchange ideas with whoever is working on the Wireshark dissector on your side if that helps (by email or conference call).
>>>>>>>> >>> Till now we just write a Lua code for the RIFT header, the next step is to take the thrift model and coding that. It would be great to setup a conf call or just create another separate email in order to know about your analysis about the dissector this could speed things up (or change it). We are now thinking on incorporate another efforts to this task.
>>>>>>>>> 
>>>>>>>>> Negative disaggregation is still on my TODO list, at the top of the TODO list in fact. However, I have not been spending any time on RIFT lately as my interests have moved on to other topics (Quantum Networking and Quantum Computing, to be specific). So, concretely, I currently have no plans in the foreseeable future to implement negative disaggregation (not in the next half year, let's say).
>>>>>>>>> 
>>>>>>>>> Still, I can see a couple of options here, depending on your needs and preferences:
>>>>>>>>> 
>>>>>>>>> (1) If you really need negative disaggregation and you have no interest in implementing it yourself, then I could be convinced to go back to RIFT for a little bit and add support for it. I don't think it would be all that much work (about two weeks I think) but the bigger issue is finding a slot of 2 weeks to work on it. I have a few other projects on my plate (a couple of long distance treks and some quantum related work) but I think that I could possibly do it some time in the next 2 or 3 months.
>>>>>>>>> 
>>>>>>>>> (2) If someone in your group would be interested in adding support for negative disaggregation to RIFT-Python themselves (would be a great learning experience for that person) then I am willing to help out with advice, code reviews, and of course taking in pull requests. It will probably turn out to be the same amount of work for me, but at least it will get more people involved in the project.
>>>>>>>>> 
>>>>>>>> >>> Mariano Scazzariello (cc) will take the option (2) and he will implement the "Negative Disaggregation" feature over your excellent rift-phyton code. I will be collaborating with this task. In this line we accept all your advice :)
>>>>>>>>> In one of my previous e-mails I mentioned that I might visit Uruguay this summer. I have been spending more time in Central America (Mexico and Guatemala) than anticipated, so we won't make it to South America this year. But we will definitely be in South America (Argentina, and hence potentially also Uruguay) in the late summer of 2021.
>>>>>>>>> 
>>>>>>>>> Finally - is Juniper being helpful in providing access to their RIFT code? If not, I could contact Tony to help.
>>>>>>>> >>> I forgot to answer the last email with Tony but yes we have access to their RIFT package, thanks!
>>>>>>>>> 
>>>>>>>>> -- Bruno
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Feb 19, 2020, at 10:28 AM, Leonardo Alberro Zimmermann <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Bruno, hope you're doing fine,
>>>>>>>>>> 
>>>>>>>>>> we are collaborating between my group in Montevideo and the Roma 3 group (CC'ed), testing different routing protocols for the datacenter, and in particular your RIFT implementation. Related with RIFT we are working on a wireshark dissector development and preparing a testbd. As we discussed before adding support for negative disaggregation to RIFT was in the TODO list. Now we would like to know if do you have plans for implementing the negative disaggregation "soon".
>>>>>>>>>> 
>>>>>>>>>> Best regards,
>>>>>>>>>> Leonardo.
>>>>>>>>>> 
>>>>>>>>>> El 3/12/19 a las 13:51, Bruno Rijsman escribió:
>>>>>>>>>>> Hello Mariano, Tommaso, and Lorenzo,
>>>>>>>>>>> 
>>>>>>>>>>> I am happy to hear that you find the RIFT work interesting.
>>>>>>>>>>> 
>>>>>>>>>>> I think it is a great idea to integrate RIFT into one or more simulation environments.
>>>>>>>>>>> 
>>>>>>>>>>> There are a couple of related developments that may be of interest to you:
>>>>>>>>>>> 
>>>>>>>>>>> 1) Leonardo Zimmermann (CC’ed) is a professor at the University of the Republic in Uruguay. He is working on a project that involves running FRR in the mini-net simulator for data center (Clos) use cases. AFAIK, he is in the process of integrating the RIFT-Python code into that environment. 
>>>>>>>>>>> 
>>>>>>>>>>> 2) Alistair Woodman (also CC’ed) is on the board of NetDEF, an organization that does a lot of development for FRR. I have been having discussions with Alistair about the possibility of porting RIFT-Python into FRR. Our preferred approach is to port RIFT-Python from Python to C and integrate it into FRR that way. Unfortunately, I don’t have the time                                                     to do that port myself, and Alistair is actively seeking other people who could do it.
>>>>>>>>>>> 
>>>>>>>>>>> — Bruno
>>>>>>>>>>> 
>>>>>>>>>>>> On Dec 3, 2019, at 10:18 AM, MARIANO SCAZZARIELLO <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Dear Bruno, 
>>>>>>>>>>>> 
>>>>>>>>>>>> we are a group of master and PhD students of the Networking Research Group of the Roma Tre University (http://www.dia.uniroma3.it/~compunet/www/view/group.php?id=compunet <http://www.dia.uniroma3.it/~compunet/www/view/group.php?id=compunet>).
>>>>>>>>>>>> 
>>>>>>>>>>>> We read your draft on RIFT: wonderful job! We think that this one of the most interesting things in the Data Center Routing field. Also, we studied your implementation of RIFT in Python that is very interesting.
>>>>>>>>>>>>  
>>>>>>>>>>>> Our group develops a network emulator called Kathará (https://www.kathara.org <https://www.kathara.org/>) that allows to emulate large networks using Docker containers.
>>>>>>>>>>>> Further, we developed a Clos Topology generator that allows to emulate networks whose control plane is implemented with BGP or OpenFabric.
>>>>>>>>>>>>  
>>>>>>>>>>>> We are considering the possibility of implementing RIFT into Kathará. There are several possibilities for that:
>>>>>>>>>>>> We could place your scripts into our containers without fully implementing a control plane
>>>>>>>>>>>> We could re-implement RIFT as a routing daemon into FRRouting
>>>>>>>>>>>> We could embed your scripts into C code inserting the result into FRRouting
>>>>>>>>>>>> 
>>>>>>>>>>>> Do you know about other initiatives for implementing RIFT? What you think about our ideas? Do you have any special suggestions?
>>>>>>>>>>>> Our target is the one of experimenting routing solutions for Data Centers in order to evaluate and compare their reliability and efficiency.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks a lot for your time.
>>>>>>>>>>>> 
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Mariano, Tommaso and Lorenzo.
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> Best regards,
>>>>>>>> Leonardo.
>>>>> 
>>> 
>

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 18 Mar 2020 20:10:39 -0700

Hey Mariano, inline ...

On Wed, Mar 18, 2020 at 6:36 PM Mariano Scazzariello <
[email-masked]> wrote:

> Hi Bruno (and everyone else),
> first of all, thanks for your time to answer my questions.
>
> About the "discard route", yes, my idea is to use this feature when a
> prefix is not reachable from any parent. It can be useful to avoid
> blackholing for a certain prefix.
>
> About treating positive/negative prefixes differently, I agree with your
> idea to store them in the RIB separately and translate the negative routes
> into positive ones only when installing them in the FIB. According to
> slides 55-60 of the Pascal's presentation, I don't think it might be that
> difficult if information is structured in the right way (with proper
> changes to RIB code).
>
> I still have some doubts (sorry!):
> 1- Special SPF should be run on each node that receives a negative
> disaggregation TIE or only on ToF nodes?
>
>    - As discussed previously, ToF nodes run special SPF everytime
>    "normal" SPFs are run
>    - But in the last email you wrote: "1) that TIE in the TIE database
>    and trigger an SPF run for later". With *SPF *you mean the normal SPF
>    or the special one?
>
> 2- I still don't understand how the "withdraw of a negative disaggregation
> TIE" is achieved.
>

Basically an empty TIE is advertised, just like .e.g. positive
disaggreagtion is "withdrawn". It is advised to advertise the empty TIE
with 300 secs liketime only so after 5 mins everyone purges it.

> 3- About "We need to enhance SPF, so that we treat negative prefixes as
> separate destination objects from positive prefixes.", what is the main
> advantage of doing so?
>
> However, I'll start working on the code a bit tomorrow. With the
> information that I have until now, I can surely start coding something. If
> you want, we can schedule a call when something is ready in the code, so we
> can discuss it.
>
>
The difficult part of negative is that you have to "backtrace" your parents
in the prefix lookup to find the gateways to use. Normally only patricia
trie keeps this kind of information but you may find another way.

The discard route is not per spec but an implementation can do that
obviously, probably not even a bad idea to install a discard when
advertising a negative for the prefix.  Pascal?

e bello di avere Italiani coinvolti nel RIFT ;-)

--- tony

---

From: Mariano Scazzariello <[email-masked]>
Date: Thu, 19 Mar 2020 02:36:02 +0100

About treating positive/negative prefixes differently, I agree with your 
idea to store them in the RIB separately and translate the negative 
routes into positive ones only when installing them in the FIB. 
According to slides 55-60 of the Pascal's presentation, I don't think it 
might be that difficult if information is structured in the right way 
(with proper changes to RIB code).

  * As discussed previously, ToF nodes run special SPF everytime
    "normal" SPFs are run
  * But in the last email you wrote: "1) that TIE in the TIE database
    and trigger an SPF run for later". With /SPF /you mean the normal
    SPF or the special one?

2- I still don't understand how the "withdraw of a negative 
disaggregation TIE" is achieved.
3- About "We need to enhance SPF, so that we treat negative prefixes as 
separate destination objects from positive prefixes.", what is the main 
advantage of doing so?

However, I'll start working on the code a bit tomorrow. With the 
information that I have until now, I can surely start coding something. 
If you want, we can schedule a call when something is ready in the code, 
so we can discuss it.

Thanks,
Mariano.

Il 18/03/2020 21:04, Bruno Rijsman ha scritto:
>> On Mar 18, 2020, at 1:45 PM, Bruno Rijsman <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> */D) Actually, there's a way to add a "negative" route in the Linux 
>> kernel table: /**/https://superuser.com/a/628385/**/. In this way, 
>> you can add an unreachable prefix in the table. Probably is a good 
>> idea to put this into the Kernel class and add a method in the FIB 
>> class to call it. /*
>>
>> It appears to be similar to what Junos (for example) calls a "discard 
>> route" 
>> (https://www.juniper.net/documentation/en_US/junos/topics/example/policy-discard-interface.html)
>>
>> It is a normal route, subject to the normal longest prefix match 
>> rules, but instead of sending matching packets to a real next-hop 
>> interface, it discards the packets (perhaps sending an ICMP 
>> unreachable message back to the source of the packet).
>>
>> A negative route in RIFT is NOT a discard route. Instead it means 
>> "please don't send the packet to me, send it to someone else instead".
>>
>> If (only) S1 advertises a negative disaggregate for prefix P, then 
>> leaf L must NOT install discard route for prefix P. Instead it must 
>> install a (normal positive) ECMP route for prefix P with next-hops S2 
>> and S3.
>>
>> Thus, we see that a negative route in the RIB gets translated into a 
>> positive route (with complementary next-hops) in the FIB.
>>
>> Actually, I came back to this section after reading some of your 
>> later comments. There may, in fact, be a use for a discard route (and 
>> possibly this is what you hand in mind in the first place). Whenever 
>> a node decides that it wants to negatively disaggregate some given 
>> prefix P (either as a result of the special SPF or as a result of 
>> getting a negative disaggregate for P from all parents) then it would 
>> be useful for that particular node to install a discard route for P 
>> to avoid packets to P hitting a more general aggregate (typically the 
>> default).
>>
>> */My general idea for this is:/*
>>
>>   * */When receiving a negative disaggregation, we have the prefixes./*
>>
>>
>> To be more precise: when we receive a (flooded) negative 
>> disaggregation TIE that contains some negatively disaggregated 
>> prefixes, we store:
>>
>> 2) For each prefix in the TIE, we immediately run to logic to keep 
>> track of whether or not we received a negative disaggregation for 
>> that prefix from EACH of the parent nodes. If that changes (true to 
>> false or vice versa) we update our own locally originated negative 
>> disaggregation TIE to start/stop advertising that particular prefix.
>>
>>   * */For each one of them:/*
>>      1. */Get the less specific prefix. As example, we have a
>>         ~8.0.1.0/24, and the less specific in the RIB is 8.0.0.0/16./*
>>      2. */Get the Nodes that can reach the less specific prefix (e.g.
>>         8.0.0.0/16 is reachable by node1, node2 and node3)/*
>>      3. */From these nodes, remove the node which sent the negative
>>         TIE (e.g. node3)/*
>>      4. */Now we know that 8.0.1.0/24 is reachable by node1 and node2/*
>>      5. */Save this in the RIB and install it in the FIB/*
>>
>>
>> My current instinct (but I am happy to discuss alternatives) is a 
>> little bit different, particularly I am thinking of having negative 
>> routes as first-class citizens in the RIB and postponing the 
>> translation from negative routes to positive routes to when routes 
>> are moved from the RIB to the FIB. This is the approach that Pascal 
>> described in his slides (attached).
>>
>>
>>
>> Currently when we run an SPF, we calculate one (positive) route to 
>> each reachable prefix P (that route is typically an ECMP route).
>>
>> We need to enhance SPF, so that we treat negative prefixes as 
>> separate destination objects from positive prefixes. In other words, 
>> if we have two TIEs: one TIE advertising prefix P as a reachable 
>> prefix or as a positively aggregated prefix, and another TIE 
>> advertising the same prefix P as a negatively aggregated prefix, then 
>> two prefixes P are NOT the same destination (as far as the RIB is 
>> concerned). We treat them as different destinations, namely 
>> P-positive and P-negative. And (once again, only as far as the RIB is 
>> concerned) we install two different destinations in the RIB: 
>> P-positive (P) and P-negative (~P).
>>
>> This line of thinking has some dramatic consequences on the 
>> complexity of the code (PS: if these complexities are more than you 
>> bargained for when you signed up to implement negative 
>> disaggregation, I can help with the coding).
>>
>> The first complication is that the code for the RIB (class 
>> RouteTable) needs to be generalized. Currently the index for entries 
>> in the RouteTable is simply the prefix. This needs to be generalized 
>> to (prefix, is_negative) where is_negative is False (for normal 
>> positive routes) or True (for negative routes). All the RIB show 
>> commands etc. need to be updated to deal with the existence of 
>> positive and negative routes.
>>
>> The second complication is that we need some logical to translate 
>> negative routes to complementary positive routes, as described in in 
>> slides 55 through 60 in Pascal's negative disaggregation slides 
>> (attached this time, as my bandwidth has improved now). This is going 
>> to be some "interesting" code.
>>
>>   * */In case there are no more nodes avaiable (point 3), the point 5
>>     installs the "unreachable" route in the kernel table./*
>>
>>   * */When receiving a "positive TIE", we should do the opposite./*
>>
>>
>> This is exactly why I think it makes sense to treat P-positive and 
>> P-negative as separate first-class objects in the RIB.
>>
>>   * */For each prefix:/*
>>      1. */Get it in the RIB, add the node which sent the TIE (e.g.
>>         8.0.1.0/24 is reachable by node1 and node2, but now from
>>         node3 too)/*
>>      2. */If the n. of nodes is equal to the n. of nodes that can
>>         reach the less specific prefix/*
>>           o */we remove the prefix from the RIB (and from the FIB)/*
>>               + */Example, 8.0.0.0/16 is reachable by node1, node2
>>                 and node3. 8.0.1.0/24 is reachable by node1, node2
>>                 and node3. Remove 8.0.1.0/24./*
>>      3. */If not, we should add a multipath in the RIB/FIB/*
>>
>>
>> In addition to Tony I have now also included Pascal in the CC list 
>> since his negative disaggregation slides play a central role in this 
>> discussion (Pascal — let me know if you also want a copy of e-mails 
>> discussing the questions in A and B).
>>
>> I have not copied the RIFT mailing list, since this discussion is 
>> very code specific. But if someone else feels this would useful, feel 
>> free to forward as far as I am concerned.
>>
>>> On Mar 10, 2020, at 12:01 PM, Mariano Scazzariello 
>>> <[email-masked] <mailto:[email-masked]>> 
>>> wrote:
>>>
>>> Hi Bruno,
>>> sorry for the delay in the reply but I've been digging into the code 
>>> to understand the implementation more in detail and where I can 
>>> "inject" the edits.
>>>
>>> A) The special SPF should be run only by ToF nodes? Besides that, I 
>>> have to run an additional southbound SPF, with E-W links. The code 
>>> to run the SPF is in /Node /class, /spf_run/ method. This calls the 
>>> /spf_run_direction/, with the /DIR_NORTH/ or /DIR_SOUTH/ constant. I 
>>> was thinking to add a third /spf_run_direction/, with 
>>> /DIR_EAST_WEST/. However, the code of that method should be changed 
>>> a bit:
>>>
>>>   * When the /spf_add_candidates_from_node /is called, after doing
>>>     preliminary operations, it calls the
>>>     /spf_add_neighbor_candidates /which has the /spf_direction/ as a
>>>     param. At this point, the /self.node_neighbors(node_ties,
>>>     spf_direction) /is called to get the current node neighbors. My
>>>     idea is that, when the /spf_direction = //DIR_EAST_WEST/, the
>>>     /node_neighbors/ method is called with a list /[//DIR_SOUTH,
>>>     DIR_EAST_WEST] /instead of a single value in order to get both
>>>     south and E-W neighbors. This needs a little code change in the
>>>     /node_neighbors/ method to accept a list AND a string, but
>>>     should be really easy to do.
>>>   * At the end of the SPF computation in /spf_run_direction/, the
>>>     /spf_install_routes_in_rib/ is called. This should be wrapped in
>>>     an if-statement like: /if spf_direction != DIR_EAST_WEST/,
>>>     because we don't want to install any route in the RIB after
>>>     doing this special SPF
>>>   * At this point, after the third SPF execution, I should be able
>>>     to do a diff between the south and E-W executions and check
>>>     which leaf nodes are missing and if so, call a method like
>>>     /send_negative_disaggregation/ to send the corresponding TIEs.
>>>     If something is missing, we should also install the unreachable
>>>     leaf nodes in the table.
>>>
>>> B) I found that each TIE is stored into a "db" called 
>>> /tie_packet_infos/. After adding the corresponding Thrift models 
>>> (point C) for the negative disaggregation TIEs, I should be able to 
>>> retrieve them from the db. However, I still don't know how to 
>>> trigger the "negative TIE check". In /Node /class, in 
>>> /process_rx_tie_packet_info/ (as far as I understood, this is 
>>> triggered by the interface sockets when a packet is received) after 
>>> storing it with /store_tie_packet_info/, I need a way to check if 
>>> the received TIE is a negative disaggregation TIE. How can I add a 
>>> callback? Should I edit the /store_tie_packet_info /code? After 
>>> that, is there a method to get the parent nodes for a given node? Or 
>>> should I add it? In this case, my general idea is:
>>>
>>>   * When a negative TIE is received, do the disaggregation in the
>>>     local routing table (see point D)
>>>   * Now we have to decide if the TIE should be flooded:
>>>      1. Get all parents. To do so, get all the Node TIEs and filter
>>>         only the ones with level = current_node_level + 1 (is that
>>>         correct?)
>>>      2. Get all negative TIEs for the prefixes contained in the
>>>         received TIE, and group them by prefix (a dict like
>>>         "prefix": N_OF_NEG_TIES_FOR_PREFIX should be enough)
>>>      3. For each item in the map check if N_OF_NEG_TIES_FOR_PREFIX
>>>         == N_PARENTS and if so, store it
>>>      4. Propagate (southbound) a negative disaggregation TIE for all
>>>         the prefixes found in the previous step (if the list is not
>>>         empty)
>>>
>>> I don't know if this is correct, maybe I've badly understood the 
>>> negative disaggregation! Expecially point 3 and 4: Negative TIE 
>>> should be propagated entirely or we can "filter" the unreachable 
>>> prefixes and send a different TIE?
>>>
>>> Of course, another check should be added when a "positive TIE" is 
>>> received. I can't find where it is described (in section 4.2.5.2.3 
>>> the draft says "Obviously, when one of the northbound switches 
>>> withdraws its negative   advertisement, the node has to withdraw its 
>>> transitively provided negative prefix as well." How is it 
>>> withdrawn?) In this case we have to check if the negative TIEs for a 
>>> given prefix are lesser than the number of parents and "withdraw" it 
>>> sending something to our neighbors.
>>>
>>> D) Actually, there's a way to add a "negative" route in the Linux 
>>> kernel table: https://superuser.com/a/628385. In this way, you can 
>>> add an unreachable prefix in the table. Probably is a good idea to 
>>> put this into the /Kernel /class and add a method in the /FIB /class 
>>> to call it.
>>>
>>>   * When receiving a negative disaggregation, we have the prefixes.
>>>   * For each one of them:
>>>      1. Get the less specific prefix. As example, we have a
>>>         ~8.0.1.0/24, and the less specific in the RIB is 8.0.0.0/16.
>>>      2. Get the Nodes that can reach the less specific prefix (e.g.
>>>         8.0.0.0/16 is reachable by node1, node2 and node3)
>>>      3. From these nodes, remove the node which sent the negative
>>>         TIE (e.g. node3)
>>>      4. Now we know that 8.0.1.0/24 is reachable by node1 and node2
>>>      5. Save this in the RIB and install it in the FIB
>>>   * In case there are no more nodes avaiable (point 3), the point 5
>>>     installs the "unreachable" route in the kernel table.
>>>
>>>   * When receiving a "positive TIE", we should do the opposite.
>>>   * For each prefix:
>>>      1. Get it in the RIB, add the node which sent the TIE (e.g.
>>>         8.0.1.0/24 is reachable by node1 and node2, but now from
>>>         node3 too)
>>>      2. If the n. of nodes is equal to the n. of nodes that can
>>>         reach the less specific prefix
>>>           o we remove the prefix from the RIB (and from the FIB)
>>>               + Example, 8.0.0.0/16 is reachable by node1, node2 and
>>>                 node3. 8.0.1.0/24 is reachable by node1, node2 and
>>>                 node3. Remove 8.0.1.0/24.
>>>      3. If not, we should add a multipath in the RIB/FIB
>>>
>>>> (A) New code to decide to originate a negative disaggregate 
>>>> according to the rules in the 2nd paragraph in section 4.2.5.2.3. 
>>>> This essentially boils down to running an additional "special" SPF 
>>>> where the E-W are considered and compare the result with the normal 
>>>> SPF where the E-W links are not considered. Any extra leaf nodes 
>>>> that are discovered in the special SPF run are fallen leafs that 
>>>> need to be negatively disaggregated. To be honest, the language in 
>>>> the draft on this topic is quite complex and vague, so it would be 
>>>> good to validate my understanding on the mailing list and/or with 
>>>> Tony. One challenge to consider is that the code doing SPFs in the 
>>>> presence of E-W links in RIFT-Python is not tested at all — it 
>>>> needs unit tests. Also, the rules for flooding TIE involving E-W 
>>>> links are very complicated and fragile (this was one of the major 
>>>> areas of interop and stability issues when testing interop with 
>>>> Juniper) - you need to tread very carefully here.
>>>>
>>>> (B) New code to decide to originate (propagate might be a better 
>>>> word) negative disaggregates according to the rules in the 3rd 
>>>> paragraph in section 4.2.5.2.3. This will involve keeping track of 
>>>> a reference count of how many parents advertised a negative 
>>>> disaggregate, to propagate the negative disaggregate when the 
>>>> reference count reaches "all parents", and the un-propagate it when 
>>>> the reference count goes down from "all parents".
>>>>
>>>> (C) Some small amount of new code for actually encoding, decoding, 
>>>> and flooding negative disaggregate TIEs. This should be almost zero 
>>>> work, since it is no different from the existing code for other 
>>>> types of TIEs.
>>>>
>>>> (D) New code to process received negative disaggregates and install 
>>>> the corresponding routes in the FIB, as described in section 4.2.6. 
>>>> The main issue is that there is no such thing as a negative entry 
>>>> in the FIB. Thus, at some point we need to convert the "negative" 
>>>> routes into complementary "positive routes" (compare figure 20 with 
>>>> figure 21). The question is: where should this logic live? Should 
>>>> it live in the RIFT-Python abstraction of the FIB (file fib.py)? 
>>>> Should it live in the RIFT-Python abstraction of the RIB (file 
>>>> rib.py)? Or should it live higher than that, e.g. in the general 
>>>> vicinity of function spf_install_routes_in_rib? I am leaning 
>>>> towards putting in the FIB itself, but this needs more thought and 
>>>> would be a good discussion for a conference call.
>>>>
>>>> In general, whatever new code you write, make sure you also create 
>>>> the corresponding automated unit test cases with good coverage of 
>>>> the new code.
>>>>
>>>>> On Mar 3, 2020, at 3:13 AM, Mariano Scazzariello 
>>>>> <[email-masked] 
>>>>> <mailto:[email-masked]>> wrote:
>>>>>
>>>>> Hi Bruno,
>>>>> thanks for the update! I'll start digging in the RIFT Python code 
>>>>> today, so I'll be able to better understand your suggestions.
>>>>>
>>>>>> I am currently in the Northern highlands of Guatemala and 
>>>>>> tomorrow (2 March) I will start a trek that will take me into 
>>>>>> some remote areas without Internet or phone.
>>>>>>
>>>>>> And quite likely, I will able to do a conference call earlier 
>>>>>> than that when I reach some more populated areas in the last 2 
>>>>>> weeks of March, but I am not sure of the exact dates (I will keep 
>>>>>> you posted).
>>>>>>
>>>>>> I will soon send a separate e-mail with my thoughts about 
>>>>>> disaggregation so that you have something to go on before the 
>>>>>> conference call.
>>>>>>
>>>>>> -- Bruno
>>>>>>
>>>>>>> On Feb 28, 2020, at 10:54 AM, Mariano Scazzariello 
>>>>>>> <[email-masked] 
>>>>>>> <mailto:[email-masked]>> wrote:
>>>>>>>
>>>>>>> Hi Bruno,
>>>>>>> So let's schedule a call (even with Leonardo, since he'll help 
>>>>>>> me in the implementation) when you're available.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mariano.
>>>>>>>
>>>>>>>> I would be happy to do a Skype call to explain the structure of 
>>>>>>>> the RIFT code and to give some hints on how to implement 
>>>>>>>> negative disaggregation.
>>>>>>>>
>>>>>>>> Right now I am in Guatemala, often in places without Internet, 
>>>>>>>> so it make take some time to find a good timeslot.
>>>>>>>>
>>>>>>>> On Fri, Feb 28, 2020, 5:08 AM Mariano Scazzariello 
>>>>>>>> <[email-masked] 
>>>>>>>> <mailto:[email-masked]>> wrote:
>>>>>>>>
>>>>>>>>     Hi Bruno,
>>>>>>>>
>>>>>>>>     as pointed out in the last email by Leonardo, I'll
>>>>>>>>     implement the Negative Disaggregation because we are
>>>>>>>>     testing the protocol and we need it as soon as possible.
>>>>>>>>
>>>>>>>>     I'm confident with Python language, but I didn't read any
>>>>>>>>     rift-python source files so I don't know how it's structured.
>>>>>>>>     To avoid conflicts with your code and to speed up the
>>>>>>>>     development, I'm asking you if it is possible to organize a
>>>>>>>>     call (maybe on Skype) to have some help starting the project.
>>>>>>>>
>>>>>>>>>     Hi Bruno! sorry for the delay, we have been arguing with
>>>>>>>>>     the group about the options that we have.
>>>>>>>>>
>>>>>>>>>     see in line >>>
>>>>>>>>>
>>>>>>>>>>     About a year ago, I also thought about implementing a
>>>>>>>>>>     RIFT Wireshark dissector myself, but it never bubbled up
>>>>>>>>>>     to the top of my list. I did spend some time on thinking
>>>>>>>>>>     how I would do it, and those thoughts might be useful to
>>>>>>>>>>     you. I was planning to write the dissector in C (as
>>>>>>>>>>     opposed to a dynamically loaded Lua dissector). That was
>>>>>>>>>>     mainly driven by the fact that I am not familiar with
>>>>>>>>>>     Lua. The main challenge that I saw was that I did not
>>>>>>>>>>     want to write the C dissector code for parsing the Thrift
>>>>>>>>>>     packet by hand. I wanted C dissector code for parsing
>>>>>>>>>>     Thrift packets to be generated by the Thrift compiler,
>>>>>>>>>>     just like the RIFT code for parsing Thrift packets is
>>>>>>>>>>     generated. I found a Wireshark C dissector for Thrift in
>>>>>>>>>>     GitHub. I contacted the author and we exchanged some
>>>>>>>>>>     e-mails on how to generalize his code for my RIFT
>>>>>>>>>>     dissector. Those e-mail conversations fizzled out as I
>>>>>>>>>>     never got to spend serious time on this little project. I
>>>>>>>>>>     even had some grandioze ideas of writing a generalized
>>>>>>>>>>     Thrift (yes, Thrift, not RIFT) dissector where the
>>>>>>>>>>     end-user of Wireshark would be able to dynamically load
>>>>>>>>>>     any Thrift model at run-time and then decode whatever
>>>>>>>>>>     packets were described by that particular dynamically
>>>>>>>>>>     loaded Thrift model. That still seems like something
>>>>>>>>>>     worthwhile to do, but maybe "a bridge too far" for RIFT,
>>>>>>>>>>     especially considering that the encapsulation header for
>>>>>>>>>>     RIFT has been becoming increasingly complex over time.
>>>>>>>>>>     But anyway - I would be happy to exchange ideas with
>>>>>>>>>>     whoever is working on the Wireshark dissector on your
>>>>>>>>>>     side if that helps (by email or conference call).
>>>>>>>>>     >>> Till now we just write a Lua code for the RIFT header,
>>>>>>>>>     the next step is to take the thrift model and coding that.
>>>>>>>>>     It would be great to setup a conf call or just create
>>>>>>>>>     another separate email in order to know about your
>>>>>>>>>     analysis about the dissector this could speed things up
>>>>>>>>>     (or change it). We are now thinking on incorporate another
>>>>>>>>>     efforts to this task.
>>>>>>>>>>
>>>>>>>>>>     Negative disaggregation is still on my TODO list, at the
>>>>>>>>>>     top of the TODO list in fact. However, I have not been
>>>>>>>>>>     spending any time on RIFT lately as my interests have
>>>>>>>>>>     moved on to other topics (Quantum Networking and Quantum
>>>>>>>>>>     Computing, to be specific). So, concretely, I currently
>>>>>>>>>>     have no plans in the foreseeable future to implement
>>>>>>>>>>     negative disaggregation (not in the next half year, let's
>>>>>>>>>>     say).
>>>>>>>>>>
>>>>>>>>>>     (1) If you really need negative disaggregation and you
>>>>>>>>>>     have no interest in implementing it yourself, then I
>>>>>>>>>>     could be convinced to go back to RIFT for a little bit
>>>>>>>>>>     and add support for it. I don't think it would be all
>>>>>>>>>>     that much work (about two weeks I think) but the bigger
>>>>>>>>>>     issue is finding a slot of 2 weeks to work on it. I have
>>>>>>>>>>     a few other projects on my plate (a couple of long
>>>>>>>>>>     distance treks and some quantum related work) but I think
>>>>>>>>>>     that I could possibly do it some time in the next 2 or 3
>>>>>>>>>>     months.
>>>>>>>>>>
>>>>>>>>>>     (2) If someone in your group would be interested in
>>>>>>>>>>     adding support for negative disaggregation to RIFT-Python
>>>>>>>>>>     themselves (would be a great learning experience for that
>>>>>>>>>>     person) then I am willing to help out with advice, code
>>>>>>>>>>     reviews, and of course taking in pull requests. It will
>>>>>>>>>>     probably turn out to be the same amount of work for me,
>>>>>>>>>>     but at least it will get more people involved in the project.
>>>>>>>>>>
>>>>>>>>>     >>> Mariano Scazzariello (cc) will take the option (2) and
>>>>>>>>>     he will implement the "Negative Disaggregation" feature
>>>>>>>>>     over your excellent rift-phyton code. I will be
>>>>>>>>>     collaborating with this task. In this line we accept all
>>>>>>>>>     your advice :)
>>>>>>>>>>     In one of my previous e-mails I mentioned that I might
>>>>>>>>>>     visit Uruguay this summer. I have been spending more time
>>>>>>>>>>     in Central America (Mexico and Guatemala) than
>>>>>>>>>>     anticipated, so we won't make it to South America this
>>>>>>>>>>     year. But we will definitely be in South America
>>>>>>>>>>     (Argentina, and hence potentially also Uruguay) in the
>>>>>>>>>>     late summer of 2021.
>>>>>>>>>>
>>>>>>>>>>     Finally - is Juniper being helpful in providing access to
>>>>>>>>>>     their RIFT code? If not, I could contact Tony to help.
>>>>>>>>>     >>> I forgot to answer the last email with Tony but yes we
>>>>>>>>>     have access to their RIFT package, thanks!
>>>>>>>>>>
>>>>>>>>>>     -- Bruno
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>     On Feb 19, 2020, at 10:28 AM, Leonardo Alberro
>>>>>>>>>>>     Zimmermann <[email-masked]
>>>>>>>>>>>     <mailto:[email-masked]>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>     we are collaborating between my group in Montevideo and
>>>>>>>>>>>     the Roma 3 group (CC'ed), testing different routing
>>>>>>>>>>>     protocols for the datacenter, and in particular your
>>>>>>>>>>>     RIFT implementation. Related with RIFT we are working on
>>>>>>>>>>>     a wireshark dissector development and preparing a
>>>>>>>>>>>     testbd. As we discussed before adding support for
>>>>>>>>>>>     negative disaggregation to RIFT was in the TODO list.
>>>>>>>>>>>     Now we would like to know if do you have plans for
>>>>>>>>>>>     implementing the negative disaggregation "soon".
>>>>>>>>>>>
>>>>>>>>>>>     Best regards,
>>>>>>>>>>>     Leonardo.
>>>>>>>>>>>
>>>>>>>>>>>>     1) Leonardo Zimmermann (CC’ed) is a professor at the
>>>>>>>>>>>>     University of the Republic in Uruguay. He is working on
>>>>>>>>>>>>     a project that involves running FRR in the mini-net
>>>>>>>>>>>>     simulator for data center (Clos) use cases. AFAIK, he
>>>>>>>>>>>>     is in the process of integrating the RIFT-Python code
>>>>>>>>>>>>     into that environment.
>>>>>>>>>>>>
>>>>>>>>>>>>     2) Alistair Woodman (also CC’ed) is on the board of
>>>>>>>>>>>>     NetDEF, an organization that does a lot of development
>>>>>>>>>>>>     for FRR. I have been having discussions with Alistair
>>>>>>>>>>>>     about the possibility of porting RIFT-Python into FRR.
>>>>>>>>>>>>     Our preferred approach is to port RIFT-Python from
>>>>>>>>>>>>     Python to C and integrate it into FRR that way.
>>>>>>>>>>>>     Unfortunately, I don’t have the time to do that port
>>>>>>>>>>>>     myself, and Alistair is actively seeking other people
>>>>>>>>>>>>     who could do it.
>>>>>>>>>>>>
>>>>>>>>>>>>     — Bruno
>>>>>>>>>>>>
>>>>>>>>>>>>>     On Dec 3, 2019, at 10:18 AM, MARIANO SCAZZARIELLO
>>>>>>>>>>>>>     <[email-masked]
>>>>>>>>>>>>>     <mailto:[email-masked]>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>     Dear Bruno,
>>>>>>>>>>>>>
>>>>>>>>>>>>>     we are a group of master and PhD students of the
>>>>>>>>>>>>>     Networking Research Group of the Roma Tre University
>>>>>>>>>>>>>     (http://www.dia.uniroma3.it/~compunet/www/view/group.php?id=compunet).
>>>>>>>>>>>>>
>>>>>>>>>>>>>     We read your draft on RIFT: wonderful job! We think
>>>>>>>>>>>>>     that this one of the most interesting things in the
>>>>>>>>>>>>>     Data Center Routing field. Also, we studied your
>>>>>>>>>>>>>     implementation of RIFT in Python that is very interesting.
>>>>>>>>>>>>>
>>>>>>>>>>>>>     Our group develops a network emulator called Kathará
>>>>>>>>>>>>>     (https://www.kathara.org <https://www.kathara.org/>)
>>>>>>>>>>>>>     that allows to emulate large networks using Docker
>>>>>>>>>>>>>     containers.
>>>>>>>>>>>>>     Further, we developed a Clos Topology generator that
>>>>>>>>>>>>>     allows to emulate networks whose control plane is
>>>>>>>>>>>>>     implemented with BGP or OpenFabric.
>>>>>>>>>>>>>
>>>>>>>>>>>>>     We are considering the possibility of implementing
>>>>>>>>>>>>>     RIFT into Kathará. There are several possibilities for
>>>>>>>>>>>>>     that:
>>>>>>>>>>>>>
>>>>>>>>>>>>>      1. We could place your scripts into our containers
>>>>>>>>>>>>>         without fully implementing a control plane
>>>>>>>>>>>>>      2. We could re-implement RIFT as a routing daemon
>>>>>>>>>>>>>         into FRRouting
>>>>>>>>>>>>>      3. We could embed your scripts into C code inserting
>>>>>>>>>>>>>         the result into FRRouting
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>     Do you know about other initiatives for implementing
>>>>>>>>>>>>>     RIFT? What you think about our ideas? Do you have any
>>>>>>>>>>>>>     special suggestions?
>>>>>>>>>>>>>     Our target is the one of experimenting routing
>>>>>>>>>>>>>     solutions for Data Centers in order to evaluate and
>>>>>>>>>>>>>     compare their reliability and efficiency.
>>>>>>>>>>>>>

---

From: Bruno Rijsman <[email-masked]>
Date: Thu, 19 Mar 2020 14:34:07 -0600

> 2- I still don't understand how the "withdraw of a negative disaggregation TIE" is achieved.
> 
> 
> Basically an empty TIE is advertised, just like .e.g. positive disaggreagtion is "withdrawn". It is advised to advertise the empty TIE with 300 secs liketime only so after 5 mins everyone purges it. 

Sending an empty TIE is a special case (namely the last negatively aggregated prefix is withdrawn).

In the more general case, the negatively aggregated prefix can be withdrawn simply by removing the prefix from the TIE and re-flooding the new TIE.

[Section 4.2.5.2.2.] When the connectivity is restored, a node that disaggregated a prefix withdraws the negative disaggregation by the usual mechanism of re- advertising TIEs omitting the negative prefix.

The receiver will replace the old TIE (with the prefix) with the new TIE (without the prefix) and re-run SPF which will remove the negative route from the RIB.

— Bruno

---

From: Mariano Scazzariello <[email-masked]>
Date: Fri, 20 Mar 2020 19:29:58 +0100

Hi everyone,
we had a call yesterday with Leonardo and the UY guys and we decided to 
create a Trello Dashboard <https://trello.com/b/rp23jdha/python-rift> to 
keep track of the Negative Disaggregation development (and also to split 
tasks easily).

I tried to put all the pieces togheter from previous emails, but surely 
something is missing or it's wrong! You have read/write access to the 
tabs so please double-check and comment what I wrote.

However, I already forked Bruno's repo and implemented the "special SPF" 
with E-W links. It seems to work. I also implemented some tests but 
sometimes the fat tree is not started properly (this happens randomly!). 
I'm thinking to switch this test on Kathará and our tools so I can check 
if they can be useful for RIFT development.

I have a question: after the special SPF, I need to run the method to 
check if there are any fallen leaf nodes. Should I use TYPE_NODE dests 
or TYPE_PREFIX dests to calculate the difference? Currently special SPF 
calculates both, but I guess that I have to use TYPE_PREFIX dests, since 
we want to detect unreachable prefixes.

Thanks,
Mariano.

---

From: Bruno Rijsman <[email-masked]>
Date: Wed, 25 Mar 2020 12:31:15 -0600

This Trello deck looks great, and I see you are already making great progress!

To avoid a massive review and merge at the end, and to have an opportunity to provide feedback and guidance early in the process, I would recommend the following:

Make every Trello card a self-contained work unit, i.e. after EACH Trello card do the following:

- Make sure all existing code still runs.

- Make sure all existing unit tests still pass.

- Make sure there are new unit tests for any new functionality introduced by the Trello card.

- Do a code review (e.g. using https://www.youtube.com/watch?v=HW0RPaJqm4g) for the incremental code changes introduced by the Trello card.

- Issue a pull request to merge the incremental code changes into the main brunorijsman/rift-python repo.

-- Bruno

PS: My Internet is very intermittent, so please be patient for responses to e-mails.

> On Mar 20, 2020, at 12:29 PM, Mariano Scazzariello <[email-masked]> wrote:
> 
> Hi everyone,
> we had a call yesterday with Leonardo and the UY guys and we decided to create a Trello Dashboard <https://trello.com/b/rp23jdha/python-rift> to keep track of the Negative Disaggregation development (and also to split tasks easily).
> I tried to put all the pieces togheter from previous emails, but surely something is missing or it's wrong! You have read/write access to the tabs so please double-check and comment what I wrote.
> 
> However, I already forked Bruno's repo and implemented the "special SPF" with E-W links. It seems to work. I also implemented some tests but sometimes the fat tree is not started properly (this happens randomly!). I'm thinking to switch this test on Kathará and our tools so I can check if they can be useful for RIFT development. 
> 
> I have a question: after the special SPF, I need to run the method to check if there are any fallen leaf nodes. Should I use TYPE_NODE dests or TYPE_PREFIX dests to calculate the difference? Currently special SPF calculates both, but I guess that I have to use TYPE_PREFIX dests, since we want to detect unreachable prefixes.
> 
> Thanks,
> Mariano.
>

---

From: Bruno Rijsman <[email-masked]>
Date: Wed, 25 Mar 2020 12:32:37 -0600

> I have a question: after the special SPF, I need to run the method to check if there are any fallen leaf nodes. Should I use TYPE_NODE dests or TYPE_PREFIX dests to calculate the difference? Curren…

Correct, you are looking for differences in TYPE_PREFIX to detect unreachable prefixes.

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 25 Mar 2020 15:06:46 -0700

Well, there are bunch aspects here. @ small scale none of that matters and
PREFIX is more convenient obviously. @ large scale an implementation
checking node differences first (or computing the normal prefixes & fallen
prefixes independently) has good potential for optimizations ...

--- tony

On Wed, Mar 25, 2020 at 12:54 PM Bruno Rijsman <[email-masked]>
wrote:

> check if there are any fallen leaf nodes. Should I use TYPE_NODE dests or
> TYPE_PREFIX dests to calculate the difference? Currently special SPF
> calculates both, but I guess that I have to use TYPE_PREFIX dests, since we
> want to detect unreachable prefixes.
>
>

---

From: Leonardo Alberro Zimmermann <[email-masked]>
Date: Thu, 26 Mar 2020 11:17:04 -0300

Hi everyone,

as a part of this project we are developing a wireshark dissector for 
RIFT. The dissection for the security envelope is ready and tested. Now 
we are working on the Serialized RIFT Model Object dissection and for 
preparing the testing we are looking for a "known" trace, i.e we need a 
few packets and the exact values of these fields in the Thrift model. So 
if anyone can help us we'll be grateful.

Regards,
Leonardo.

El 18/3/20 a las 13:34, Antoni Przygienda escribió:
>
> Forwarding the discussion to the rift ietf list for further exposure & 
> since I generally think it will be possibly more productive in a wider 
> forum. Roma Tre University is working on Bruno’s open source and 
> there’s a bunch of interesting tools they’re developing as well as you 
> can read below.
>
> --- tony
>
> *From: *Mariano Scazzariello <[email-masked]>
> *Date: *Wednesday, March 18, 2020 at 9:10 AM
> *To: *Bruno Rijsman <[email-masked]>
> *Cc: *Antoni Przygienda <[email-masked]>, Leonardo Alberro Zimmermann 
> <[email-masked]>, "[email-masked]" 
> <[email-masked]>, "[email-masked]" 
> <[email-masked]>, Giuseppe Di Battista 
> <[email-masked]>
> *Subject: *Re: RIFT protocol implementation
>
> Hi everyone,
> thanks for the extremely detailed suggestions! I have a lot of stuff 
> which I can use to work on the implementation. Also thanks Tony for 
> giving me further useful tips.
>
> Also, I'm happy that we agree on almost every implementation detail 
> and I agree with your suggested variations (like the flag for the 
> spf_run_direction method).
>
> About point C, I did not write anything since it seems quite easy to 
> extend the Thrift model. If I have any issues, I'll surely ask you for 
> some help. I also wrote a point D (which is how to handle the negative 
> disaggregation in the RIB/FIB when received), but I read the Pascal 
> slides (I add the link 
> <https://urldefense.com/v3/__https:/bitbucket.org/riftrfc/rift_draft/src/master/negative*20disaggregation.pptx__;JQ!!NEt6yMaO-gk!QBFQP3Ec6u0elEKgEFaAEndcjyQntJXvMg0h2TfAH6kn4JLUaMb3FOYpDQOltw$> 
> here, so everyone can access them easily) and the steps used in that 
> presentation are the same that I thought.
>
> *2) We will also need a new member field orig_neg_disagg_prefixes 
> (once again of type set, I think) that contains the negatively 
> disaggregated prefixes that are autonomously being originated based on 
> the detection of fallen leafs based on the difference between the 
> normal and special SPF run.*
> You are proposing to postpone the RIB/FIB update after the special SPF 
> run (which can detect additional fallen leaf nodes). This is right,  
> but should this only occur on ToF nodes? If we consider a node X, 
> which is not a ToF, that receives a negative disaggregation TIE, it 
> should only add the prefix in the /prop_neg_disagg_prefixes /set and 
> check if it received this prefix from all its parent nodes and 
> propagate it if required. Then it should proceed to update its 
> RIB/FIB, without running the special SPF. Am I right?
>
> About the flooding oscillations, Tony writes "Your best protection is 
> scaled, randomized tests". Here at Roma Tre University we developed a 
> tool called Kathará 
> <https://urldefense.com/v3/__https:/www.kathara.org/__;!!NEt6yMaO-gk!QBFQP3Ec6u0elEKgEFaAEndcjyQntJXvMg0h2TfAH6kn4JLUaMb3FOZujrrECA$> 
> which is able to emulate network scenarios using Docker containers. 
> Recently, we also developed a Fat Tree Generator, that automatically 
> generates a fat tree topology starting from the fundamental parameters 
> of a Fat Tree (K_LEAF, K_TOP and R) that can be run in Kathará. It 
> also auto-configures the routing protocol on each node (of course, we 
> also included the RIFT-Python implementation). So we can generate Fat 
> Tree topologies of any size and run tests on it to verify functional 
> and behavioral aspects (and also gather routing information such as 
> PDU size and count). We are also developing a Fat Tree Test Framework 
> (in collaboration with Leonardo and the team at UY university) which 
> implements typical data center network scenarios (such as link 
> failure, node failure, fallen leaf, partitioned fabric and so on) to 
> run integration tests on it. For example, we can check if the routing 
> table of a node is equal to the expected one after a failure (e.g. 
> loss of a multipath or a prefix). With this tool we can surely run 
> randomized tests (at any scale, since Kathará supports Kubernetes) to 
> ensure that no flooding oscillations occur.
>
> I agree to move the discussion on the RIFT WG mailing list. Maybe 
> someone of you should introduce us and explain what we're doing.
>
> _______________________________________________
> RIFT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rift

---

From: Tony Przygienda <[email-masked]>
Date: Thu, 26 Mar 2020 09:23:39 -0700

Leonardo, that is a great project. Observe that depending on language
backend/thrift compiler different streams may represent the same encoding
so it's good to test it against more than one implementation. We should get
streams that has all possible LIEs and TIEs in it.

If you run juniper standalone public package
https://www.juniper.net/us/en/dm/free-rift-trial/ you can easily build a
2x2 yaml topology and just snoop on a UDP port. I suggest snooping a v6/v4
links from a spine to a leaf. that will give you all the LIEs and node TIEs
and default prefixes. If you break a link via CLI you will get
disaggregation as well. If you struggle with that, ping me ...

Having said that you can do all that with open source as well (but observe
we didn't pull open source to -11 draft yet which had some schema changes
albeit minor). you can pull the schema yourself with Bruno's help or I'll
get to it one of these days ;-) ...

Architecturally, since RIFT will evolve and possibly move to newer major
schemas, I would suggest you provision for the possibility to put into
wireshark multiple schema files, basically you'd need to compile multiple
directories in differnt namespaces and use the correct one depending on the
major you found on packet envelope (you can always use the newest minor you
support on the major, they are always compatible).

--- tony

On Thu, Mar 26, 2020 at 7:17 AM Leonardo Alberro Zimmermann <
[email-masked]> wrote:

> RIFT. The dissection for the security envelope is ready and tested. Now we
> are working on the Serialized RIFT Model Object dissection and for
> preparing the testing we are looking for a "known" trace, i.e we need a few
> packets and the exact values of these fields in the Thrift model. So if
> anyone can help us we'll be grateful.
>
> Regards,
> Leonardo.
> forum. Roma Tre University is working on Bruno’s open source and there’s a
> bunch of interesting tools they’re developing as well as you can read
> below.
>
>
>
> --- tony
>
>
>
> <[email-masked]>
> *To: *Bruno Rijsman <[email-masked]> <[email-masked]>
> *Cc: *Antoni Przygienda <[email-masked]> <[email-masked]>, Leonardo
> Alberro Zimmermann <[email-masked]> <[email-masked]>,
> "[email-masked]" <[email-masked]>
> <[email-masked]> <[email-masked]>,
> "[email-masked]" <[email-masked]>
> <[email-masked]> <[email-masked]>, Giuseppe Di
> Battista <[email-masked]>
> thanks for the extremely detailed suggestions! I have a lot of stuff which
> I can use to work on the implementation. Also thanks Tony for giving me
> further useful tips.
>
> Also, I'm happy that we agree on almost every implementation detail and I
> agree with your suggested variations (like the flag for the
> extend the Thrift model. If I have any issues, I'll surely ask you for some
> help. I also wrote a point D (which is how to handle the negative
> disaggregation in the RIB/FIB when received), but I read the Pascal slides
> (I add the link
> *2) We will also need a new member field orig_neg_disagg_prefixes (once
> again of type set, I think) that contains the negatively disaggregated
> prefixes that are autonomously being originated based on the detection of
> fallen leafs based on the difference between the normal and special SPF
> run.*
> You are proposing to postpone the RIB/FIB update after the special SPF run
> (which can detect additional fallen leaf nodes). This is right,  but should
> this only occur on ToF nodes? If we consider a node X, which is not a ToF,
> that receives a negative disaggregation TIE, it should only add the prefix
> in the *prop_neg_disagg_prefixes *set and check if it received this
> prefix from all its parent nodes and propagate it if required. Then it
> should proceed to update its RIB/FIB, without running the special SPF. Am I
> right?
>
> scaled, randomized tests". Here at Roma Tre University we developed a tool
> called Kathará
> generates a fat tree topology starting from the fundamental parameters of a
> Fat Tree (K_LEAF, K_TOP and R) that can be run in Kathará. It also
> auto-configures the routing protocol on each node (of course, we also
> included the RIFT-Python implementation). So we can generate Fat Tree
> topologies of any size and run tests on it to verify functional and
> behavioral aspects (and also gather routing information such as PDU size
> and count). We are also developing a Fat Tree Test Framework (in
> collaboration with Leonardo and the team at UY university) which implements
> typical data center network scenarios (such as link failure, node failure,
> fallen leaf, partitioned fabric and so on) to run integration tests on it.
> For example, we can check if the routing table of a node is equal to the
> expected one after a failure (e.g. loss of a multipath or a prefix). With
> this tool we can surely run randomized tests (at any scale, since Kathará
> supports Kubernetes) to ensure that no flooding oscillations occur.
>
> I agree to move the discussion on the RIFT WG mailing list. Maybe someone
> of you should introduce us and explain what we're doing.
>
> RIFT mailing [email-masked]://www.ietf.org/mailman/listinfo/rift
>
>