Subject: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 19 Feb 2019 19:44:21 -0800

Benjamin Kaduk has entered the following ballot position for
draft-ietf-detnet-architecture-11: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

I note that the DETNET WG is explicitly chartered with a work item for the
"overall architecture: This work encompasses ... and security aspects".
It seems incomplete to specify an architecture for a topic such as
deterministic networking without specifically considering what threats are
and are not in scope to be protected against.  Some easy questions should
be whether the system is expected to be robust in the face of an attacker
that generates non-DetNet traffic?  Or an attacker that generates DetNet
traffic in excess of reservations?  It can even be a fine engineering goal
to produce a solution that only protects against media corruption and
hardware crashes and leaves active attacks out of scope, but the actual
intended scope of the work needs to be clear.  At the other end of the
spectrum, protecting against as potent an attacker as a malicious traffic
policer is probably a lost cause, especially if the policer is authorized
to direct remote nodes to take action to terminate "misbehaving" flows.

The referenced draft-ietf-detnet-security is not at a comparable maturity
level to this document and also fails to present a clear threat model for
the DetNet architecture.  (The section entitled "Threat Model" reads as
more of a taxonomy of threats than a model for what threats are and are not
to be addressed.)  It also presents the usage of cryptographic mechanisms
as mitigation techniques without provisioning for the prerequisties of such
mechanisms (e.g., using HMAC for message integrity protection without
mention of infrastructure for distributing the keys for keying the HMAC).

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I agree with Alexey that Informational would (also) be a fine status in
which to publish this document.

Abstract

                                 DetNet operates at the IP layer and
   delivers service over sub-network technologies such as MPLS and IEEE
   802.1 Time-Sensitive Networking (TSN).

I don't know what "sub-network technologies" means.  (Should I?  Is it
defined somewhere we can reference?)

More generally, is DetNet supposed to be a "sub-layer" and/or "sub-network"
that lies between specific layers or classes of layer?  Does DetNet
itself have component "sub-layers" that provide distinct DetNet
functionality?  These are good questions to address early on in the
document so the reader is familiar with the concepts as they progress
through the document.

Section 1

                          DetNet is for networks that are under a single
   administrative control or within a closed group of administrative
   control; these include campus-wide networks and private WANs.  DetNet
   is not for large groups of domains such as the Internet.

side note: Campus-wide networks at educational institutions are basically
guaranteed to have untrusted entities participating in them, just as a
backdrop for security considerations.

Section 3.1

                        This mechanism distributes the contents of
   DetNet flows over multiple paths in time and/or space, so that the
   loss of some of the paths does need not cause the loss of any

The failure models for which this statement is absolutely true as opposed
to probabilistically true seem rather unrealistic models of real physical
systems.

Section 3.2.1.1

   The primary means by which DetNet achieves its QoS assurances is to
   reduce, or even completely eliminate packet loss due to output packet
   contention within a DetNet node as a cause of packet loss.  [...]

editing error?

                                     Note that App-flows are generally
   not expected to be responsive to implicit [RFC2914] or explicit
   congestion notification [RFC3168].

I note that the word "implicit" does not appear in RFC 2914; it may be
worth a bit more detailed of a mapping from concept to reference.
(This text/reference also appears in Section 4.3.2.)

Section 3.2.1.2

                                                                   In
   general, users are encouraged to use, instead of, "do this when you
   get the packet," a combination of:

It seems that an architecture would be within its rights to *mandate* such
application design, rather than just encourage it.  What sorts of
exceptions would cause us to not want to mandate this design?

Section 3.2.2.2

Please expand SRLG (it is only used once, so the abbreviation itself may
not be needed at all).

Section 3.2.3

   Out-of-order packet delivery can be a side effect of distributing a
   single flow over multiple paths especially when there is a change
   from one path to another when combining the flow.  [...]

nit: comma before "especially".

   Resource allocation
           The DetNet forwarding sub-layer provides resource allocation.
           See Section 4.5.  The actual queuing and shaping mechanisms
           are typically provided by underlying subnet, these can be

nit: is this usage of "subnet" common?
Also, this comma looks to be a comma splice.

           closely associated with the means of providing paths for
           DetNet flows, the path and the resource allocation are
           conflated in this figure.

nit: Hmm, actually, is this comma *also* a comma splice?

   Operations, Administration, and Maintenance (OAM) leverages in-band
   and out-of-band signaling that validates whether the service is
   effectively obtained within QoS constraints.  [...]

nit: is there a singular/plural mismatch here ("the service"/"service" vs.
"effectively within"/"effectively obtained within")?

Section 4.1.1

This figure would have helped me a lot several sections earlier.

Section 4.1.2

   A "Deterministic Network" will be composed of DetNet enabled end
   systems, DetNet edge nodes, DetNet relay nodes and collectively
   deliver DetNet services.  DetNet relay and edge nodes are

Nit: I think this is intended to be:
A "Deterministic Network" will be composed of DetNet-enabled end
systems, DetNet edge nodes, and DetNet relay nodes, which collectively
deliver DetNet services.  DetNet relay and edge nodes are

                                                             Examples of
   sub-networks include MPLS TE, IEEE 802.1 TSN and OTN.  [...]

nit: are these sub-networks or protocols used by sub-networks?

   Distinguishing the function of two DetNet data plane sub-layers, the
   DetNet service sub-layer and the DetNet forwarding sub-layer, helps
   to explore and evaluate various combinations of the data plane
   solutions available, some are illustrated in Figure 4.  This

nit: this last comma is a comma splice.

   There are many valid options to create a data plane solution for
   DetNet traffic by selecting a technology approach for the DetNet
   service sub-layer and also selecting a technology approach for the
   DetNet forwarding sub-layer.  There are a high number of valid
   combinations.

nit: I think "large number" is more conventional prose.

Section 4.3.1

I think I'm confused about how, for these flows that "require the <foo>
feature", whether that means that the DetNet implementation must provide
<foo>, or that it is required for the application to have implemented the
<foo> feature.  A mapping (if it makes sense) to the categorization of end
systems in Section 4.2.1 would be a big help.

Section 4.3.2

   Asynchronous DetNet flows are characterized by:

   o  A maximum packet size;

   o  An observation interval; and

   o  A maximum number of transmissions during that observation
      interval.

Is there necessarily only a single tier of observation interval/rate?
(E.g., could there be a burst cap in a small interval and then a lower
overall baseline rate over large intervals?)

                     That is, while any useful application is written to
   expect a certain number of lost packets, the real-time applications
   of interest to DetNet demand that the loss of data due to the network
   is a rare event.

(I might even go with "vanishingly rare".)

Section 4.4.1

(Is there a standard reference for "Northbound"?  I know we're all used to
it, but it's probably best to have a reference if we can.)

Section 4.4.2

                                         The deterministic sequence can
   typically be more complex than a direct sequence and include
   redundancy path, with one or more packet replication and elimination
   points.  [...]

nit: "redundancy paths", plural?

Section 4.8

How does *provisioning* require knowledge of *dynamic* state?

Section 4.9

Does aggregation like this pose a risk of all the aggregatees getting
affected when one exceeds their allocation substantially (so as to also
cause the aggregate to exceed the aggregate's allocation)?

Section 6

The ability for an attacker to use QoS markings as part of traffic
correlation/inspection is not new with DetNet, but is probably still worth
mentioning explicitly.

---

From: Lou Berger <[email-masked]>
Date: Wed, 20 Feb 2019 09:33:34 -0500

Hi Benjamin,

I'm responding as doc shepherd (and chair). Thank you for your 
comments.  Please see below for specific responses.

On 2/19/2019 10:44 PM, Benjamin Kaduk wrote:
> and are not in scope to be protected against.

The working group certainly recognizes that security is an important 
topic and necessary part of the work. The WG's  approach to ensure that 
this significant topic is sufficiently covered is to have a  document 
dedicated to the topic.  As you note below this work,  
draft-ietf-detnet-security, is not at the same level maturity - so 
please stay tuned.  Of course, the WG would welcome any input on this 
security draft now or even a full early-review or security area advisor. 
(We have one from the transport area.)

> Some easy questions should
These are all great input for draft-ietf-detnet-security - I'll touch 
with authors to ensure that they are aware of your comments.

I defer to the IESG and our AD.

I think the authors can address/respond to the remaining comments.

...

Thanks,

Lou

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 20 Feb 2019 10:07:44 -0600

Hi Lou,

> security draft now or even a full early-review or security area advisor. 

I do appreciate that the WG has taken the time to work on a dedicated
security document, that goes through the (tedious!) effort to tabulate the
various known attacker capabilities, threats, and their interactions.

What I think is missing from the high-level architecture document is a
sense for what security goals the architecture intends to achieve.  It's
fine to have this be separate from the nitty-gritty stuff in the separate
document, but these high-level questions can have an impact on the
high-level design of the protocol ecosystem as a whole.  We have mounds of
examples of "come up with a design; bolt security on as an afterthought"
working poorly or being nigh-impossible, and I haven't seen any attempt at
justification for why doing so here is likely to be any different.
Including the security concepts from the start and wholly integrating them
into the system tends to produce a much more elegant design and smoother
outcomes all around.

> with authors to ensure that they are aware of your comments.

I noticed a few other things of note while skimming; I'll try to send those
over as well.

> I think the authors can address/respond to the remaining comments.

Sounds good.

-Benjamin

---

From: Lou Berger <[email-masked]>
Date: Wed, 20 Feb 2019 11:34:11 -0500

Hi Benjamin,

     Please see below.

> into the system tends to produce a much more elegant design and smoother

We went back and forth on how much of the security topic to include in 
the architecture document and settled on the current text of section 5 
where just high level security requirements are identified.  From your 
list below, the architecture covers the points you raise (non-detnet 
traffic impacting detnet traffic, and detnet traffic trying to use more 
than its allocation) mainly from a service delivery perspective.  It 
does mention "shaping" as a security concern, the document could expand 
these if helpful. But I think the real discussion point is how much of 
the security details should be in the base (vs dedicated) document  -- 
and that you disagree with where the WG has drawn the line.  Is this 
correct?

Thanks,

Lou

> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet
>

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 20 Feb 2019 15:21:05 -0600

Hi Lou,

> and that you disagree with where the WG has drawn the line.  Is this 

Well, it's not as much that I disagree about where the line is drawn, as
that I disagree with the characterization as "high level security
requirements".

The first paragraph covers topics worth including here, certainly -- an
attacker that can introduce delay can inherently break the DetNet
guarantees, and there is no further defense.  But the text could probably
go further than its current version and explicitly call out that no
technical measures can defend against such delay, other than by
rerouting/avoiding the compromised position.

The second paragraph also covers a topic worth noting, that entropy and
Murphy as as important of risks to consider as active attacks.

The rest of the section/list don't seem to match up with the high-level
view, though -- "protection of the signaling protocol", but protection
against what?  Authentication and authorization are important parts of
security, but authorization is tied to actions and entities, not just
entities in vacuo.  For "identification and shaping of DetNet flows", if
security must cover them, what security properties are needed.  Do we need
to provide integrity protection for the marking, or confidentiality
protection, too?  Does authentication/integrity need to be provided for
in-line or are out-of-band considerations in play?

The rest of the document presents a good high level picture of what pieces
will be involved in a system, and how they fit together.  I'm hoping to see
a high-level picture of what attacks/failures the architecture does and
does not aim to protect against.  It's somewhat implied that we need to
protect against the failure of any single node or link (except at the
boundary) due to random faults; explicitly saying this would be a perfect
hting to include in the security considerations here!  Once we get beyond
entropy as an attack, into more active attack scenarios, though, we need to
consider what capabilities an attacker might have.  Do we protect against
an entity with the ability to inject various forms of traffic?  Modify or
drop valid traffic?  Are there any (classes of) credentials that we assume
must be securely held, or will we attempt to protect against attackers that
have some privilege within the system?  (Note that misbehaving hardware can
sometimes present as similar or identical to such an attacker.)

Does this give a better picture of what I'm thinking of as "high-level
security requirements"?  I don't necessarily need details of specific
hardware or credentials at play, and certainly not for the solutions, but I
want to have a clear picture of what the scope of future security analysis
will be.  "How will I be able to tell if the actual protocols meet the
security requirements of the architecture?"

-Benjamin

---

From: Lou Berger <[email-masked]>
Date: Thu, 21 Feb 2019 15:32:15 -0500

Hi Benjamin,

> security requirements of the architecture?"

I'll get with authors (of both the architecture and security docs) and 
see how much they think can be reasonably be added to this document -- 
to be clear you *are* asking for a change in how the WG decide to 
address this topic and what would go in which document.

Lou

---

From: "Grossman, Ethan A." <[email-masked]>
Date: Mon, 25 Feb 2019 06:31:16 +0000

Hi Benjamin,
As editor and co-author of the DetNet Security draft, I thank you very much for your comments regarding the security aspects of the DetNet Architecture and Security draft. Indeed it is our intent to ensure that security is considered throughout the design process and not "bolted on" at the end; this is why we have been developing the Security draft in parallel with the Architecture, Data Plane and other drafts.  Here are a couple of my initial thoughts on your comments; I present them for discussion, not as a defense of a pre-determined outcome. 

I follow your concern that we have not made a distinction between the laundry list of possible threats versus those threats that designs based on the architecture must address in order to satisfy the security requirements of the architecture. Excellent point, in that I for one have not considered the topic in that light before, and I appreciate the need to do so now. 

Up until this point, i.e. prior to your comments, we have been working based on at least two guiding principles. One is that we are only addressing security considerations which are specific to time-sensitive, routed, centrally-administered networks, so anything that could be considered to be "covered elsewhere" we have intentionally omitted. For example you cite "using HMAC for message integrity protection without mention of infrastructure for distributing the keys for keying the HMAC"; I would expect that this topic could be regarded as "covered elsewhere", but please correct me if I'm missing something here. 

Another main thread running through almost every aspect of DetNet is that as noted in the Use Cases draft, there are many and diverse applications for DetNet, spanning networks contained within a single machine all the way through networks spanning a nation. As such it is difficult to make generalizations about almost anything in DetNet - everything has to be framed as "scalable" so that we don't mandate something that makes sense for one use case but not for another. 

So the intent of the Security draft (in conjunction with security text in the other DetNet drafts) thus far has been to provide some insights into the aspects of security that are introduced with time-sensitivity to routed networks, with a nod as to how they might be addressed, while being almost completely non-prescriptive about it, leaving the details as an exercise for the reader based on their particular use case. 

In other words, as soon as we say "a DetNet must include [any specific security property]", someone will inevitably come up with a use case for which that didn't make sense (insufficient time, insufficient CPU cycles, not required in this case because blah, etc.) For example when I look at your questions like "Do we need to provide integrity protection for the marking, or confidentiality protection, too?  Does  authentication/integrity need to be provided for in-line or are out-of-band considerations in play?" my thought would be to say "well it depends on the use case, i.e. the specific implementation". 

So at this point I don't know whether this stance is valid, or whether it is an abdication of our responsibility, or whether there are some stakes in the ground that we can actually claim will fit all use cases, and thus we should be stating as mandatory. Certainly we would like to avoid the outcome that the architecture leaves open questions as to whether a given implementation satisfies its security requirements. We also want to avoid being overly prescriptive such that "exceptions" would need to be made to accommodate diverse use cases. 

Please let us know how this line of thinking affects your assessment of our situation, if at all. My short term goal here is to unblock publication of the Architecture draft, but not at the expense of adequate coverage and integration of security. 

Thanks again for your review, 
Sincerely,
Ethan (as editor and co-author of the DetNet Security draft and the DetNet Use Cases draft). 

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Lou Berger
Sent: Thursday, February 21, 2019 12:32 PM
To: Benjamin Kaduk <[email-masked]>
Cc: [email-masked]; [email-masked]; The IESG <[email-masked]>; [email-masked]
Subject: Re: [Detnet] Benjamin Kaduk's Discuss on draft-ietf-detnet-architecture-11: (with DISCUSS and COMMENT)

Hi Benjamin,

On 2/20/2019 4:21 PM, Benjamin Kaduk wrote:
>>>>> I note that the DETNET WG is explicitly chartered with a work item 
>>>>> for the "overall architecture: This work encompasses ... and security aspects".
>>>>> deterministic networking without specifically considering what 
>>>>> threats are and are not in scope to be protected against.
>>>> The working group certainly recognizes that security is an 
>>>> important topic and necessary part of the work. The WG's  approach 
>>>> to ensure that this significant topic is sufficiently covered is to 
>>>> have a  document dedicated to the topic.  As you note below this 
>>>> work, draft-ietf-detnet-security, is not at the same level maturity 
>>>> - so please stay tuned.  Of course, the WG would welcome any input 
>>>> on this security draft now or even a full early-review or security area advisor.
>>> I do appreciate that the WG has taken the time to work on a 
>>> dedicated security document, that goes through the (tedious!) effort 
>>> to tabulate the various known attacker capabilities, threats, and their interactions.
>>>
>>> What I think is missing from the high-level architecture document is 
>>> a sense for what security goals the architecture intends to achieve.  
>>> It's fine to have this be separate from the nitty-gritty stuff in 
>>> the separate document, but these high-level questions can have an 
>>> impact on the high-level design of the protocol ecosystem as a 
>>> whole.  We have mounds of examples of "come up with a design; bolt security on as an afterthought"
>>> working poorly or being nigh-impossible, and I haven't seen any 
>>> attempt at justification for why doing so here is likely to be any different.
>>> Including the security concepts from the start and wholly 
>>> integrating them into the system tends to produce a much more 
>>> elegant design and smoother outcomes all around.
>> We went back and forth on how much of the security topic to include 
>> in the architecture document and settled on the current text of 
>> section 5 where just high level security requirements are identified.  
>> From your list below, the architecture covers the points you raise 
>> (non-detnet traffic impacting detnet traffic, and detnet traffic 
>> trying to use more than its allocation) mainly from a service 
>> delivery perspective.  It does mention "shaping" as a security 
>> concern, the document could expand these if helpful. But I think the 
>> real discussion point is how much of the security details should be 
>> in the base (vs dedicated) document  -- and that you disagree with 
>> where the WG has drawn the line.  Is this correct?
> Well, it's not as much that I disagree about where the line is drawn, 
> as that I disagree with the characterization as "high level security 
> requirements".
>
> The first paragraph covers topics worth including here, certainly -- 
> an attacker that can introduce delay can inherently break the DetNet 
> guarantees, and there is no further defense.  But the text could 
> probably go further than its current version and explicitly call out 
> that no technical measures can defend against such delay, other than 
> by rerouting/avoiding the compromised position.
>
> The second paragraph also covers a topic worth noting, that entropy 
> and Murphy as as important of risks to consider as active attacks.
>
> The rest of the section/list don't seem to match up with the 
> high-level view, though -- "protection of the signaling protocol", but 
> protection against what?  Authentication and authorization are 
> important parts of security, but authorization is tied to actions and 
> entities, not just entities in vacuo.  For "identification and shaping 
> of DetNet flows", if security must cover them, what security 
> properties are needed.  Do we need to provide integrity protection for 
> the marking, or confidentiality protection, too?  Does 
> authentication/integrity need to be provided for in-line or are out-of-band considerations in play?
>
> The rest of the document presents a good high level picture of what 
> pieces will be involved in a system, and how they fit together.  I'm 
> hoping to see a high-level picture of what attacks/failures the 
> architecture does and does not aim to protect against.  It's somewhat 
> implied that we need to protect against the failure of any single node 
> or link (except at the
> boundary) due to random faults; explicitly saying this would be a 
> perfect hting to include in the security considerations here!  Once we 
> get beyond entropy as an attack, into more active attack scenarios, 
> though, we need to consider what capabilities an attacker might have.  
> Do we protect against an entity with the ability to inject various 
> forms of traffic?  Modify or drop valid traffic?  Are there any 
> (classes of) credentials that we assume must be securely held, or will 
> we attempt to protect against attackers that have some privilege 
> within the system?  (Note that misbehaving hardware can sometimes 
> present as similar or identical to such an attacker.)
>
> hardware or credentials at play, and certainly not for the solutions, 
> but I want to have a clear picture of what the scope of future 
> security analysis will be.  "How will I be able to tell if the actual 
> protocols meet the security requirements of the architecture?"

I'll get with authors (of both the architecture and security docs) and see how much they think can be reasonably be added to this document -- to be clear you *are* asking for a change in how the WG decide to address this topic and what would go in which document.

Lou