Subject: [109all] Initial report on yesterday's jabber service interruption

From: IETF Executive Director <[email-masked]>
Date: Fri, 20 Nov 2020 13:43:20 +1300

As previously reported, there was an interruption to the jabber service during yesterday’s break between sessions 2 and 3.  This has now been diagnosed as a DoS condition, quite possibly unintentional, on our jabber server.  

The IETF jabber server is configured to accept 32,000 connections in total and a maximum of 100 messages/sec.  This is an order of magnitude higher than our expected levels during an IETF meeting.  Yesterday the server received a spike in connections at 17,000/sec and a spike in messages at 4000/sec that continued after the limits had been reached and until the server died a few minutes later.  When the server restarted the spike was no longer present.

Work is ongoing to identify the source(s) of the spike and to detect and mitigate any further spikes.

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]