Subject: [Rift] [RIFT] Routing in Fat Trees "chaos testing" hackathon at IETF 104 in Prague

From: Bruno Rijsman <[email-masked]>
Date: Mon, 25 Feb 2019 22:14:59 -0300

Hi everyone,
 
At the Routing In Fat Trees (RIFT) IETF 104 hackathon in Prague we will be working on "chaos monkey" testing of two RIFT implementations: the RIFT-Python open source implementation and a vendor RIFT implementation.

This hackathon is inspired by the concept of chaos engineering and in particular by the chaos monkey tool, which intentionally and randomly shuts down servers in a production environment. This is intended to build confidence in the system's capability to withstand turbulent and unexpected conditions.

In this hackathon we will do something similar for RIFT networks. We will build a large fat tree network of RIFT routers. Then we will randomly insert failures into the network and make sure that the network behaves correctly. We will make sure the RIFT routers don't crash or otherwise behave badly. And we will make sure that the routers forwarding tables and link-state databases converge correctly.

We will use (and possibly enhance) the chaos testing framework as well as the visualization and verification tools that are already part of the RIFT-Python open source project.

Even if you are not much interested in the RIFT routing protocol, but you are interested in testing large network topologies using container-like technology, or in visualization tools, or in chaos testing, this might be the right hackathon for you.

A detailed description of the planned activities at the hackathon is available at: http://bit.ly/rift-hackathon-ietf-104 <http://bit.ly/rift-hackathon-ietf-104>

The project is listed on the IETF 104 hackathon wiki page: https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon <https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon> (info copied below)
 
Feel free to join us.

Bruno Rijsman

Routing In Fat Trees (RIFT) Chaos Monkey Testing

* Champions
  * Tony Przygienda <prz at juniper.net> (will join in person)
  * Bruno Rijsman <brunorijsman at gmail.com> (will join remotely)
* Project(s)
  * ​RIFT is a STD track WG working on a modern IP fabric/DC routing protocol
  * ​Detailed description of the planned activities at the hackathon at 
    http://bit.ly/rift-hackathon-ietf-104
  * We have two implementations now:
    * RIFT-Python open source
    * Vendor implementation
    * The open source one has a chaos monkey framework in place we can use 
      to stress protocol implementations and interoperability between them. 
      Unusual hackathon but that will be very, very useful for further 
      development and quality of implementations while breaking completely 
      new grounds (chaos testing on model based protocol specifications).
* Preconditions:
  * Python knowledge
  * Familiarity with the open source RIFT code on ​https://github.com/brunorijsman/rift-python
  * RIFT Specification in some detail ​RIFT draft
  * Familiarity with creating virtual machines in AWS

---

From: Tony Przygienda <[email-masked]>
Date: Tue, 26 Feb 2019 07:33:09 -0800

I'll be setting up a 16 or so cores AWS account next week that we can use
for larger runs during hackathon. For development purposes I think e'one
can chuck with a micro freebie ...

Ubuntu I assume?

thanks

--- tony

On Mon, Feb 25, 2019 at 5:15 PM Bruno Rijsman <[email-masked]>
wrote:

> Hi everyone,
>
> At the Routing In Fat Trees (RIFT) IETF 104 hackathon in Prague we will be
> working on "chaos monkey" testing of two RIFT implementations: the
> RIFT-Python open source implementation and a vendor RIFT implementation.
>
> This hackathon is inspired by the concept of chaos engineering and in
> particular by the chaos monkey tool, which intentionally and randomly shuts
> down servers in a production environment. This is intended to build
> confidence in the system's capability to withstand turbulent and unexpected
> conditions.
>
> In this hackathon we will do something similar for RIFT networks. We will
> build a large fat tree network of RIFT routers. Then we will randomly
> insert failures into the network and make sure that the network behaves
> correctly. We will make sure the RIFT routers don't crash or otherwise
> behave badly. And we will make sure that the routers forwarding tables and
> link-state databases converge correctly.
>
> We will use (and possibly enhance) the chaos testing framework as well as
> the visualization and verification tools that are already part of the
> RIFT-Python open source project.
>
> Even if you are not much interested in the RIFT routing protocol, but you
> are interested in testing large network topologies using container-like
> technology, or in visualization tools, or in chaos testing, this might be
> the right hackathon for you.
>
> *A detailed description of the planned activities at the hackathon is
> available at: http://bit.ly/rift-hackathon-ietf-104
> <http://bit.ly/rift-hackathon-ietf-104>*
>
> The project is listed on the IETF 104 hackathon wiki page:
> https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon (info copied
> below)
>
> Feel free to join us.
>
> Bruno Rijsman
>
>
>   * Familiarity with the open source RIFT code on ​
> https://github.com/brunorijsman/rift-python
> _______________________________________________
> RIFT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rift
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 26 Feb 2019 15:36:47 +0000

Hi Bruno,

This looks like a great project. Thanks for bringing it to the hackathon.

Do you have any special network requirements? Power and access to the IETF network via WiFi will be available at all tables. If you need wired access or anything else, please let me know as soon as possible and we will do our best to accommodate.

Cheers,
Charles

From: hackathon <[email-masked]> on behalf of Bruno Rijsman <[email-masked]>
Date: Monday, February 25, 2019 at 5:15 PM
To: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: [hackathon] [RIFT] Routing in Fat Trees "chaos testing" hackathon at IETF 104 in Prague

Hi everyone,

A detailed description of the planned activities at the hackathon is available at: http://bit.ly/rift-hackathon-ietf-104

The project is listed on the IETF 104 hackathon wiki page: https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon (info copied below)

Feel free to join us.

Bruno Rijsman

* Champions
  * Tony Przygienda <prz at juniper.net<http://juniper.net>> (will join in person)
  * Bruno Rijsman <brunorijsman at gmail.com<http://gmail.com>> (will join remotely)
* Project(s)

---

From: Bruno Rijsman <[email-masked]>
Date: Tue, 26 Feb 2019 12:42:53 -0300

Quick follow-up: I also posted a youtube video of a hands-on walkthrough of RIFT chaos testing: https://youtu.be/GqebgPmA4Xc <https://youtu.be/GqebgPmA4Xc>

> On Feb 25, 2019, at 10:14 PM, Bruno Rijsman <[email-masked]> wrote:
> 
> Hi everyone,
>  
>   * Tony Przygienda <prz at juniper.net <http://juniper.net/>> (will join in person)
>   * Bruno Rijsman <brunorijsman at gmail.com <http://gmail.com/>> (will join remotely)
> * Project(s)
>     http://bit.ly/rift-hackathon-ietf-104 <http://bit.ly/rift-hackathon-ietf-104>
>   * Familiarity with the open source RIFT code on ​https://github.com/brunorijsman/rift-python <https://github.com/brunorijsman/rift-python>

---

From: Bruno Rijsman <[email-masked]>
Date: Tue, 26 Feb 2019 12:44:46 -0300

Hi Charles, 

No, no special requirements.  Just good Internet access, since all of the heavy lifting will happen in the AWS cloud.

— Bruno

> On Feb 26, 2019, at 12:36 PM, Charles Eckel (eckelcu) <[email-masked]> wrote:
> 
> Hi Bruno,
>  
> A detailed description of the planned activities at the hackathon is available at: http://bit.ly/rift-hackathon-ietf-104 <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbit.ly%2Frift-hackathon-ietf-104&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294829286&sdata=a3ClcYZX1avQH27yzXl48Wxo47YPAmUoSCZ7M3aG6JQ%3D&reserved=0>
>  
> The project is listed on the IETF 104 hackathon wiki page: https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrac.ietf.org%2Ftrac%2Fietf%2Fmeeting%2Fwiki%2F104hackathon&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294839291&sdata=L3w0gXD524rbYqEXQw2kbgCGNFj9Nv3W55%2BbDHiNvYA%3D&reserved=0> (info copied below)
>  
> Feel free to join us.
>  
> Bruno Rijsman
>  
>  
>   * Tony Przygienda <prz at juniper.net <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fjuniper.net&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294849296&sdata=pLCTPG%2FVfwguCn9dqi07KH3fAfu1EZcFTSg7%2B5svXDE%3D&reserved=0>> (will join in person)
>   * Bruno Rijsman <brunorijsman at gmail.com <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgmail.com&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294869324&sdata=Nm8L6rVMOWZm3%2Bu7aNim3h%2FqV3UuNQ%2Bj9ixIXpwWkts%3D&reserved=0>> (will join remotely)
> * Project(s)
>     http://bit.ly/rift-hackathon-ietf-104 <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbit.ly%2Frift-hackathon-ietf-104&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294879323&sdata=SyZTYxEQA2B7r%2BxpaDrXIvyqwM58JoLs5FO5YdRRwAA%3D&reserved=0>
>   * Familiarity with the open source RIFT code on ​https://github.com/brunorijsman/rift-python <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbrunorijsman%2Frift-python&data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294889340&sdata=ZiZEg53UuJEb2TTkWOcmWFefP4WyGHALpzOzgtwif3E%3D&reserved=0>
> hackathon mailing list
> [email-masked] <mailto:[email-masked]>
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fhackathon&amp;data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294939371&amp;sdata=1bGO7%2FcCFOJp%2F4s3AbEpMtjoRTl7pqYcLtBSMhGMzWY%3D&amp;reserved=0 <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fhackathon&amp;data=02%7C01%7C%7C8041ec8127374fcea70c08d69c004347%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636867922294939371&amp;sdata=1bGO7%2FcCFOJp%2F4s3AbEpMtjoRTl7pqYcLtBSMhGMzWY%3D&amp;reserved=0>