Subject: [106all] Etherpad online

From: Glen <[email-masked]>
Date: Wed, 20 Nov 2019 00:18:50 -0800

All -

Having repaired Etherpad the first time - so that it was now proxying
and functioning correctly - we are now unexpectedly hitting resource
limits on the webserver.   Each person who connects to Etherpad takes
up one connection, and those are in competition with connection
attempts from other requestors for other web resources.  When those
requests spike, Etherpad gets impacted.

Normally of course connection requests stack in queue, and get
processed happily in order, but Etherpad - which requires constant
(re)connections - "notices" that delay, and can't cope with it,
prompting the disconnect messages we're seeing.  Etherpad, by its
design, requires continuous and immediate connectivity, which is not
compatible with a high-volume website like the IETF's.

So although Etherpad has worked well all "day" (evening in the USA)...
when the overnight (in the USA) search engine "run" started (in your
afternoon), all of those search engine runs were starting to push out
the Etherpad connections, and some people started getting
interruptions.

I've increased our immediate capacity to compensate for what I saw,
but of course I cannot prevent an even larger search engine flood from
disrupting Etherpad again.  I've therefore re-opened port 9009 as a
backup measure.  I won't forcibly redirect the zone (owing to the
complaints about Etherpad's lack of SSL), but, if any of you
experience further disconnects, an immediate workaround option would
be to connect to Etherpad at:

http://4.31.198.44:9009/

That is a direct connection, and will not be impacted by any web or
other traffic spikes that our main sites might encounter.

I'm working on a number of upgrades to software used by the IETF, and
will work to improve Etherpad's deployment as a part of that process.

Thank you for your patience.

Glen
--
Glen Barney
IT Director
AMS (IETF Secretariat)