Subject: [OPSAWG] I-D Action: draft-ietf-opsawg-sdi-06.txt

From: [email-masked]
Date: Fri, 03 Apr 2020 13:40:32 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Operations and Management Area Working Group WG of the IETF.

        Title           : Secure Device Install
        Authors         : Warren Kumari
                          Colin Doyle
	Filename        : draft-ietf-opsawg-sdi-06.txt
	Pages           : 18
	Date            : 2020-04-03

Abstract:
   Deploying a new network device often requires that an employee
   physically travel to a datacenter to perform the initial install and
   configuration, even in shared datacenters with "smart-hands" type
   support.  In many cases, this could be avoided if there were a
   standard, secure way to initially provision the devices.

   This document extends existing auto-install / Zero-Touch Provisioning
   mechanisms to make the process more secure.

   [ Ed note: Text inside square brackets ([]) is additional background
   information, answers to frequently asked questions, general musings,
   etc.  They will be removed before publication.  This document is
   being collaborated on in Github at: https://github.com/wkumari/draft-
   wkumari-opsawg-sdi.  The most recent version of the document, open
   issues, etc should all be available here.  The authors (gratefully)
   accept pull requests. ]

   [ Ed note: This document introduces concepts and serves as the basic
   for discussion - because of this, it is conversational, and would
   need to be firmed up before being published ]

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-opsawg-sdi/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-opsawg-sdi-06
https://datatracker.ietf.org/doc/html/draft-ietf-opsawg-sdi-06

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-opsawg-sdi-06

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: tom petch <[email-masked]>
Date: Mon, 6 Apr 2020 10:36:34 +0000

Warren

Where I think I get confused with this is its context.  Abstract talks of travelling to a datacentre and elsewhere there are references to a POP, both of which to me have a flavour of a well-staffed high in technical expertise locations where this sort of work is little needed.  I think more of enterprise, where an organisation may have two well equipped data centres and dozens or hundreds of locations with little or no support staff where this issue is paramount.  I think that this is more a question of language than of changing the technical details but it does keep jarring with me.  In the same vein, the references to routers jars with me since while that may be an issue in an operator POP, I see the need to configure other kinds of servers as more pressing.

The other more technical issue is TFTP which yes, I expect will be widely used but which, IMHO, is only ever used over a LAN and so, short of VLAN, which indeed some enterprise do use, implies that the device and config server are on the same LAN, ie in the same building or at least campus.  Again, it is a question of context, is it assumed that device and server are proximal?

I would like to see these two points nailed down more after which I could propose some refinement to the language.

Tom Petch

________________________________________
From: OPSAWG <[email-masked]> on behalf of [email-masked] <[email-masked]>
Sent: 03 April 2020 21:40
To: [email-masked]
Cc: [email-masked]
Subject: [OPSAWG] I-D Action: draft-ietf-opsawg-sdi-06.txt

Abstract:

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg

---

From: Warren Kumari <[email-masked]>
Date: Mon, 6 Apr 2020 11:07:09 -0400

> The other more technical issue is TFTP which yes, I expect will be widely used but which, IMHO, is only ever used over a LAN and so, short of VLAN, which indeed some enterprise do use, implies that…

It is often surprising to me just how much one's experiences and
situations influence one's assumptions and outlook - it seems that I
may have assumed that others have the same experiences / didn't
explain the context well.

It is quite common that operators want to install a peering router /
switch at a location where they have no employees - common examples of
this are IXP / POP / data-center / anywhere where they peer with
others (e.g Ashburn Equinix). While there are (often) technical people
at the location, they are either employed by the datacenter operator
(e.g Equinix, AMS-IX), or by one's "competitors" - this means that you
really don't want to hand them a copy of your config file and ask them
to stick it on the device. Operators generally order a circuit from
inside their network to the location, and then order a device to be
shipped there... and then have to dispatch a person to stick the
initial config on the device. Traditional (unencrypted) autoboot
doesn't solve this, because $whoever could just plug their laptop into
the newly installed circuit, perform the autoboot routine, and have a
copy of the config.

Another very common case is for an ISP (think Verizon, or Telus) to
deliver a circuit to a customer - they have a contractor which
delivers the fiber, and then roll a truck to have someone physically
plug in a PE / CPE router and install the initial config. Again, they
don't really want to hand the config to the user, and also cannot use
the current autoboot solutions for the same reason - the customer
could just plug their own device / laptop into the newly installed
circuit, autoboot and grab the config / join the IGP / whatever.

These are the specific use-case that this is intending to solve - many
/ almost all network devices already support some sort of autoboot
where they DHCP (or similar) and then fetch (TFTP is a very common
mechanism[0], but by no means the only one) an **unencrypted** config
file -- all that this document does is say "keep doing that, but
download an encrypted config instead". It intentionally is kept simple
- vendors already have their own autoboot functionality and this is a
simple patch to that. Vendors also already have a way to put
per-device state (such as a serial number, licence, and often crypto
goop in a TPM, etc) - in some cases all they will need to do is
publish the public key, indexed by the serial number (a number of
vendors have said that this will be trivial). The document is
intentionally kept flexible because so much of this builds on vendor
proprietary solutions; and so instead of saying "Step 1: Do X. Step 2:
Do Y, ... Step N: Do N", it describes the concept, and leaves it to
the (more than capable) vendors to figure out how to implement it in
their system.

This also is only aimed at network devices - other types of servers
may be more pressing, but this document tries not to boil the ocean -
it simply says "instead of grabbing an unencrypted config, grab an
encrypted one, decrypt it and done!"

Hopefully this better sets the scene / explains the context and use
case? I'm not sure if there is a concise way to explain this in the
draft...

W
[0]: I'm using in the document because it is the archetype, and
familiar to the intended audience...

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: tom petch <[email-masked]>
Date: Mon, 6 Apr 2020 15:49:37 +0000

From: Warren Kumari <[email-masked]>
Sent: 06 April 2020 16:07

<tp>
Warren, thanks for that.,  I sort of guessed the first but not the second.   For the second, do you get TFTP over a newly laid fibre?  I was thinking not based on what I have seen but if you do then that is fine..
 
Scenarios I have encountered do not fit either of these two e.g. upgrades to a remote enterprise office with lots of staff but no technical expertise or the installation of a single device that must be secure (e.g.  Hole in the Wall) but that is ok, it is just that I do not quite see the draft excluding them,

I think that it is possible to write an introduction that limits the scope along the lines you sketch out.  Do you have any other scenario?

Tom Petch

W

---

From: Michael Richardson <[email-masked]>
Date: Mon, 06 Apr 2020 14:04:01 -0400

> circuit, autoboot and grab the config / join the IGP / whatever.

I have dealt with this situation.
We used unencrypted TFTP to configure the CPE device over the fiber.
The approach that we took was to have the untagged "VLAN" provide basic DHCP
and the TFTP configuration did only one thing: enable SSH access with a known
password (hashed). (Because the device didn't support RSA keys)

Our installer would genereally be on the "electrician" side of things: able
to drill holes and pull cables, and they often *had* to do exactly that.
When the lights went green, someone from HQ would login and finish
configuration.

So, even if you do have someone to physically plug things in, and they work
for you, they might just not have the skills required.

    > vendors have said that this will be trivial). The document is

That's interesting and positive to learn.
Did these vendors say how they will do this?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Mon, 06 Apr 2020 14:07:51 -0400

Tom, please quote better.

    > <tp>
    > Warren, thanks for that.,  I sort of guessed the first but not the
    > second.   For the second, do you get TFTP over a newly laid fibre?  I
    > was thinking not based on what I have seen but if you do then that is
    > fine..

If you buy an ethernet extension service then you can do TFTP if you want.
If you buy dark fiber, then you get whatever you decide to enable on the port
that you plug the other end into.
Would I want to enable DHCP and TFTP on that port? Not really, but it can be done.
(That's why BRSKI uses IPv6 LL addresses instead)

    > Scenarios I have encountered do not fit either of these two
    > e.g. upgrades to a remote enterprise office with lots of staff but no
    > technical expertise or the installation of a single device that must be
    > secure (e.g.  Hole in the Wall) but that is ok, it is just that I do
    > not quite see the draft excluding them,

I don't think would use SDI for either purpose you state.
Upgrades to equipment either imply it is already bootstrapped.
I don't understand the "Hole in the Wall" reference.

---

From: tom petch <[email-masked]>
Date: Tue, 7 Apr 2020 09:06:43 +0000

From: Warren Kumari <[email-masked]>
Sent: 06 April 2020 16:07
On Mon, Apr 6, 2020 at 6:36 AM tom petch <[email-masked]> wrote:
<tp>
Warren, 
understanding better what you have in mind, I suggest a few changes to the Abstract and Introduction, as below.  My language is probably a bit tighter, omitting some adverbs but that is just a matter of style.  The changes are not great but for me they make the (limited) scope clearer (and yes, all the cases which I had in mind are now excluded:-)

Tom Petch

                        draft-ietf-opsawg-sdi-06

Abstract

   Deploying a new network device in a location 
   where the operator has no staff of its own often requires that an employee
   physically travel to the location to perform the initial install and
   configuration, even in shared datacenters with "smart-hands" type
   support.  In many cases, this could be avoided if there were a
   secure way to initially provision the device.

 1.  Introduction

   In a growing, global network, significant amounts of time and money
   are spent deploying new devices and "forklift" upgrading
   existing devices.  In many cases, these devices are in shared
   datacenters (for example, Internet Exchange Points (IXP) or "carrier
   neutral datacenters"), which have staff on hand that can be
   contracted to perform tasks including physical installs, device
   reboots, loading initial configurations, etc.  There are also a
   number of (often vendor proprietary) protocols to perform initial
   device installs and configurations - for example, many network
   devices will attempt to use DHCP [RFC2131]to get an IP address and
   configuration server, and then fetch and install a configuration when
   they are first powered on.

  The configurations of network devices contain a significant amount of
   security related and / or proprietary information (for example,
   RADIUS [RFC2865] or TACACS+ [I-D.ietf-opsawg-tacacs] secrets).
   Exposing these to a third party to load onto a new device (or using
   an auto-install techniques which fetch an unencrypted config file via
   TFTP [RFC1350]), or something similar, is an unacceptable security risk for many
   operators, and so they send employees to remote locations to
   perform the initial configuration work; this costs, time and money.

   There are some workarounds to this, such as asking the vendor to pre-
   configure the devices before shipping it; asking the smart-hands to
   install a terminal server; providing a minimal, unsecured
   configuration and using that to bootstrap to a complete
   configuration, etc; but these are often clumsy and have security
   issues - for example, in the terminal server case, the console port
   connection could be easily snooped.

   This document layers security onto existing auto-install solutions to
   provide a secure method to initially configure new devices.  It is
   optimized for simplicity, both for the implementor and the operator;
   it is explicitly not intended to be an "all singing, all dancing"
   fully featured system for managing installed / deployed devices, nor

   is it intended to solve all use-cases - rather it is a simple
   targeted solution to solve a common operational issue where the 
   network device has been delivered, fibre laid (as appropriate) 
   but there is no trusted member of the operator's staff to perform
   the initial configuration.
  
   Solutions
   such as Secure Zero Touch Provisioning (SZTP)[RFC8572],
   [I-D.ietf-anima-bootstrapping-keyinfra] and the like are much more
   fully featured, but also more complex to implement and / or are not
   widely deployed yet.

   This solution is specifically designed to be a simple method on top
   of exiting device functionality.  If devices do not support this new
   method, they can continue to use the existing functionality.  In
   addition, operators can choose to use this to protect their
   configuration information, or can continue to use the existing
   functionality.

   The issue of securely installing devices is in no way a new issue,
   nor is it limited to network devices; it occurs when deploying
   servers, PCs, IoT devices, and in many other situations.  While the
   solution described in this document is obvious (encrypt the config,
   then decrypt it with a device key), this document only discusses the
   use for network devices, such as routers and switches.

W

---

From: tom petch <[email-masked]>
Date: Tue, 7 Apr 2020 09:15:38 +0000

From: OPSAWG <[email-masked]> on behalf of Michael Richardson <[email-masked]>
Sent: 06 April 2020 19:07

<tp>
Hole in the Wall is ATM or cash dispenser - an isolated device in a hostile location at the end of a piece of wire where security is paramount.

Apologies if my responses are not as easy to see as they might be; I am currently stuck with a web interface that does lots of things that I wish it did not and does not do what I had taken for granted for decades, like indenting all old text with '> '.  Even the way in which existing e-mails are displayed to me seems random, sometimes using indentation with spaces, sometimes  '> ' (to me the obviously best way to do it) sometimes coloured lines down the side of the screen, all of which seems outside my control - sigh.

Tom Petch

---

From: Warren Kumari <[email-masked]>
Date: Tue, 7 Apr 2020 13:58:33 -0400

> understanding better what you have in mind, I suggest a few changes to the Abstract and Introduction, as below.  My language is probably a bit tighter, omitting some adverbs but that is just a matt…

Thank you - I incorporated your changes verbatim (at least, I tried
to, but it required some rejiggering to get the formatting right), and
posted a new version - this text does a much better job of explaining
the scope.

Thanks,
W
P.S: I also updated the Acknowledgments section to recognize the text
- thank you!

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 7 Apr 2020 18:59:16 +0000

On Apr 7, 2020, at 13:58, Warren Kumari <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Apr 7, 2020 at 5:06 AM tom petch <[email-masked]<mailto:[email-masked]>> wrote:

From: Warren Kumari <[email-masked]<mailto:[email-masked]>>
Sent: 06 April 2020 16:07
On Mon, Apr 6, 2020 at 6:36 AM tom petch <[email-masked]<mailto:[email-masked]>> wrote:

<tp>
Warren,

Yes, this captures what I think is a very typical use case (and one similar to what I had in mind when I first read this).  Though I might have said “layer 1” or “base connectivity” instead of fiber to be agnostic to the medium used for connectivity.

To the other bit of changed text around costing time and money, I think what COVID-19 is showing us is that the dispatching of clued staff may not be possible given local, national, or global restrictions (in addition to being a drain on time and money), but that is more of me thinking aloud.

I think this current text has helped close out some of the remaining issues.

Joe