Subject: IETF Server Upgrade/Cutover

From: Glen <[email-masked]>
Date: Thu, 20 Feb 2020 20:39:16 -0800

Dear IETF Community:

On Saturday, February 22, we will be re-executing the cutover of all
IETF services to our new virtual production server group. These new
servers contain the latest version of the Linux operating system on
which they run, a new Linux kernel, new versions of many of the open
source software packages we've come to know and love, and Python 3.6,
opening the door for the Tools Team to work additional magic with the
software packages they continue developing for the community.  Last
month, when we performed this process the first time, the process took
about one hour, and to my delight, almost everything worked without
any issues at all.  We did have some minor issues with the
registration system, now corrected, and we had an errant Python script
that ran overnight causing some data loss, which has also been
corrected.  So I'm actually more confident this time, having seen than
pretty much all of our other services survived the upgrade without a
hitch.  Nonetheless, this operation is significant, and will involve
the shutdown of services, a transfer of data, and the subsequent
activation of the new servers.

While the process itself will begin earlier, there will be a visible
one-hour outage window beginning at 9AM Pacific Time (Noon Eastern,
5PM GMT, overnight elsewhere), Saturday February 22.  During this
window, some services will be interrupted at different intervals.
Email and list transport will stop before 9AM, and remain down for
most of the window, while web and Datatracker will be down for shorter
intervals during the window.  The presence or absence of any one
service should not imply expectations for other services during this
interval.  Once we're finished, we will perform a complete reboot,
after which the new servers will be brought online.  As a part of our
testing, we'll send an announcement out again when the work is
completed.

Although we've all done quite a bit of work on this (and been through
the process once already) there may still be problems we have not yet
been able to identify.  Both AMS and the Tools Team will be performing
additional checks and testing post-cutover, and we will work to
correct any identified issues as quickly as possible.  We ask for your
patience with this process.  As always, if you have questions or
concerns, you can direct them to the IETF-ACTION desk (
[email-masked] ) which I supervise, or you can contact me
directly via email or Jabber at:  [email-masked] , or Telegram or Wire
at @glenams .

Thank you for your patience as we work together to continue to improve
the IETF's online presence, infrastructure and services.

Glen
--
Glen Barney
IT Director
AMS (IETF Secretariat)