Subject: [Gen-art] Gen-ART and OPS-Dir review of draft-ietf-ospf-node-admin-tag-06

From: "Black, David" <[email-masked]>
Date: Mon, 5 Oct 2015 23:35:17 +0000

This is a combined Gen-ART and OPS-Dir review.  Boilerplate for both follows ...

I am the assigned Gen-ART reviewer for this draft. For background on
Gen-ART, please see the FAQ at:

<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Please resolve these comments along with any other Last Call comments
you may receive.

I have reviewed this document as part of the Operational directorate's ongoing
effort to review all IETF documents being processed by the IESG.  These comments
were written primarily for the benefit of the operational area directors.
Document editors and WG chairs should treat these comments just like any other
last call comments.

Document: draft-ietf-ospf-node-admin-tag-06
Reviewer: David Black
Review Date: October 5, 2015
IETF LC End Date: October 8, 2015
IESG Telechat date: October 15, 2015

Summary:  This draft is on the right track, but has open issues
 		described in the review.

This is a generally well-written draft about adding administrative tags for
operational use to OSPF.  The draft is clear, and the material on how the
tags could be used is helpful, although raises a number of concerns about
the consequences of the intended usage of these tags.

---- Major issues: ----

[1] Operational considerations:  There appears to be more than enough
enabled by this draft to wreak serious operational havoc, but the draft
seems to sidestep all operational topics, primarily by treating all
usage of tags as vendor- or implementation- specific and trusting the
vendors and operators not to foul things up.  I'm not sure that's wise.

See end of OPS-Dir review below for more on this concern.

--- Minor issues: ----

-- 3.2 Elements of procedure:

[A] I see what look like some underspecified requirements:

   Each tag SHOULD be treated as an independent identifier that MAY be
   used in policy to perform a policy action.

   The administrative tag list within the
   TLV SHOULD be considered an unordered list.

Why are those two not "MUST" requirements?  What happens if either
is not done?

[B] Tag set completeness:

   Multiple node administrative tag TLVs MAY appear in an RI LSA or 
   multiple node administrative tag TLVs MAY be contained in different
   instances of the RI LSA.  The node administrative tags associated
   with a node for the purpose of any computation or processing SHOULD
   be a superset of node administrative tags from all the TLVs in all
   instances of the RI LSA originated by that node.

This paragraph is about processing at that node.  It's easy to misread,
as that implication is buried in the word "originated" in the last line. 
Suggested change:

	"for the purpose of any computation or processing SHOULD" ->
	"for the purpose of any computation or processing performed
		at that node SHOULD"

Also, it looks like it's acceptable for other nodes to perform such
computation or processing based on a partial tag set for this node
(e.g., when some other node has not received all the RI LSAs with all
the tags).  That should be stated.

[C] Tag change/removal:

   When there is a change in the node administrative tag TLV or removal/
   addition of a TLV in any instance of the RI-LSA, implementations MUST
   take appropriate measures to update its state according to the
   changed set of tags.  Exact actions depend on features working with
   administrative tags and is outside of scope of this specification.

Inability to interoperably remove a tag value (e.g., distribute the
update that tag X no longer applies to node Q) seems like a significant
omission, but I'm not a routing expert, so I'll defer to the WG's and
ADs' judgment on the importance of this.  At a minimum, the rationale
for not specifying an interoperable tag value removal mechanism ought
to be added to this document.

[D] No management support

>From OPS-Dir Q&A: At a minimum, reporting of tag values ought to be
defined via an OSPF MIB extension or analogous functionality.

--- Nits/editorial comments: ----

-- 1.  Introduction

The Abstract says that the tags are for "route and path selection"; the
Introduction should also say that.  Suggestion - at the end of this sentence
in the first paragraph:

   It is useful to assign a per-node administrative tag to a router in
   the OSPF domain and use it as an attribute associated with the node.

add the text "for route and path selection".  This will help with the
second sentence in the second paragraph:

   Path selection is a functional set
   which applies both to TE and non-TE applications and hence new TLV
   for carrying per-node administrative tags is included in Router
   Information LSA [RFC4970] .

If "path selection" and "functional set" are specific terms with
specialized meaning in OSPF context, that sentence is fine as-is,
otherwise it would read better if it began with:

   Route and path selection functionality applies to both ...

-- 3.1.  TLV format

   Type : TBA, Suggested value 10

Please add an RFC Editor Note asking the RFC Editor to replace this with
the actual IANA-assigned value.

-- 3.2.  Elements of procedure

While it's obvious that tag usage should be confined to the administrative
domain that understands the tag, it's worth stating.  At the end of this
sentence:

   Interpretation of tag values is specific to the administrative domain
   of a particular network operator.

I'd suggest adding:

   , and hence tag values SHOULD NOT be propagated outside the
   administrative domain to which they apply.

-- 4.4.  Mobile back-haul network service deployment

Please expand "eNodeB" acronym on first use.

-- 4.5.  Explicit routing policy

In Figure 3:
- The link from the leftmost pair of A nodes to the pair of T nodes
   do not have link weights.
- The link from the left R node to the left T node does not have a
   link weight
- The following example of an explicitly routed policy:

      - Traffic from A nodes to I nodes must not go through R and T
      	nodes

    prevents the leftmost pair of A nodes from sending traffic to the
    I nodes.  Was this "black hole" intended as part of the example?

Also: "explicitly routed policies" -> "explicit routing policies"

- 5. Security considerations

I'd add discussion that advertisement of tag values for one administrative
domain into another risks misinterpretation of the tag values (if the two
domains have assigned different meanings to the same values), which may
have undesirable and unanticipated side effects.

In addition, injection of tag values from the outside (e.g., forge OSPF
traffic that appears to be from a node in the domain and carries
administrative tag values) is at least a possible denial-of-service attack
vector, and could also be used for more nefarious purposes (e.g., reroute
otherwise unobservable [to the attacker] VPN traffic via a route where
the attacker can observe it).

idnits 2.13.02 did not find any nits.

---- Selected RFC 5706 Appendix A Q&A for OPS-Dir review ----

A.1.2.  Has installation and initial setup been discussed?
A.1.5.  Has the impact on network operation been discussed?
A.1.6.  Have suggestions for verifying correct operation been discussed?

	No - given the impact that these tags could have on route and path
		computation, likely implementations will be powerful "guns"
		with which network operators can readily shoot themselves
		in far more than just their "feet."  These
		considerations would have to be documented based on the
		specific uses made of these tags by specific implementations
		and deployments.  All of that appears to be outside the scope
		of this draft.

A.1.7.  Has management interoperability been discussed?

	No - at a minimum, reporting of tag values ought to be defined
		via an OSPF MIB extension or analogous functionality.
		This is minor issue [D].

A.1.8.  Are there fault or threshold conditions that should be reported?

	Yes, but they're implementation-specific - see response to
		A.1.[2,5,6] above.

A.2.  Management Considerations
   Do you anticipate any manageability issues with the specification?

	Yes, manageability has been largely ignored.

A.3.  Documentation

   Is an operational considerations and/or manageability section part of
   the document?

	No.

   Does the proposed protocol have a significant operational impact on
   the Internet?

	Yes, the anticipated uses will.

   Is there proof of implementation and/or operational experience?

	Nothing was stated in the draft or shepherd write-up.

As an OPS-Dir member, I'm concerned by the above RFC 5706 answers,
and in particular treating all operational issues as vendor- and/or
operator-specific.  One possible alternative would be to scope the
draft down to interoperably specify what's needed for LFA, as
indicated by this answer from the Shepherd write-up:

(9) How solid is the WG consensus behind this document? Does it
    represent the strong concurrence of a few individuals, with others
    being silent, or does the WG as a whole understand and agree with it?

      There is consensus from the WG and others outside the WG that
      this document can progress. It complements work done on LFA 
      manageability in the RTG Working Group.

Another alternative could be Experimental RFC status for the full
tag mechanism (e.g., to figure out what it'll be used for in practice,
how and why) rather than Proposed Standard.

This is major issue [1].

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: Rob Shakir <[email-masked]>
Date: Tue, 6 Oct 2015 16:23:03 +0100

Hi David,

On October 6, 2015 at 00:37:42, Black, David ([email-masked]) wrote:
> draft down to interoperably specify what's needed for LFA

I’ll quote this part of your mail, but respond to the general question around whether this draft should be tighter scoped (and the operational considerations of what one can do with the mechanism).

In general, this mechanism is meant to give ways for an operator to add their own logic on-top of the routing protocol selection of the OSPF protocol. Specifically, this is for the case where there are sets of paths that OSPF considers valid, but the operator has a particular preference for a particular subset to be included or excluded. To this end, the harm that one can do by preferring paths with a particular admin tag (or set of admin tags) on them is simply to select a subset of the paths that would have been used anwyay. 

So, from the examples in the document, the admin tag lets one:

	* Prefer a specific LFA from a set of valid LFAs. The operator has a business reason to prefer specific ones (e.g., OSPF metric is set based on bandwidth, but we want to minimise latency, or we only want to capacity plan for FRR between P nodes never transiting a PE). There are many different reasons that one might have this preference, and they will be different by operator - so, removing the ability of an operator to specify arbitrary tags will make this mechanism ineffective. Whilst there could be unintended consequences of preferring specific LFAs, actually there are probably worse consequences of NOT preferring particular LFAs, such as traffic going via a PE in another country such that latency is increased over all other LFAs. Bruno and Stephane from FT have given many examples of this in the LFA manageability drafts presented in RTGWG. The admin tag will never result in a new LFA that would never have been selected via the standardised mechanisms being selected - so, in my view, has little potential to cause new ‘harm’.
		
	* Prefer a specific path from a set of paths. This is similar to the case above. No new candidate path is being added to the routes that could be used by the admin tag mechanism. It simply says that when a device that must select N ECMPs to install from M candiates (N < M), then the operator can influence which based on the admin tag. All are still valid ECMPs - so new risk is created. Again, selecting a particular subset of that M without any consideration may be more harmful than doing the operator-specific selection. Again, the logic of which are ‘OK’ PEs and which are not is something that is operator specific, and may only apply to a subset of applications using OSPF to select paths (an example of this is devices that do not scale well as MPLS-TE midpoints, but are OK for transit IP traffic - at this point, we may want to influence CSPF to stay away from them, but not IP - the fact that these devices are ‘not-OK-for-transit-MPLS-TE’ is not something that we can easily enumerate and standardise).

Given that we are really selecting candidates from within a set of paths that have already been selected by OSPF as valid, and usable - then I’m not sure that I can understand the logic behind this sentence from your review: "There appears to be more than enough enabled by this draft to wreak serious operational havoc”.

I strongly disagree with the idea that we should try and enumerate what the tags should be, rather than maintain the ability of an operator to add their own specific logic with these tags. This is what is done today with link affinity, or tags elsehwere in the IGP protocols. 

Re-reading the draft, perhaps we need to clarify that this does not change the underlying SPF process that OSPF uses today - would this address your concern?

Regards,
r.

---

From: <[email-masked]>
Date: Tue, 6 Oct 2015 16:39:47 +0000

Hi David,

Thanks for your careful review.

> vendors and operators not to foul things up.  I'm not sure that's wise.

May be the draft is missing some text on some points, but I don't share this _major_ concern (regarding the operation use of tags):
- their use in routing protocols is not new and we have a large operational feedback on this. e.g. 8 years with IS-IS prefix tags (RFC 5130), and especially 20 years with BGP community (RFC 1997) which are heavily used by all network operators, including between Autonomous System/Administrative Domains (which is beyond OSPF usage)
- they are meant to ease network operation. We could probably do everything you can think of without node tags, e.g. by referencing each node individually using its loopback address or router ID. But this would be more error prone and difficult to maintain over time.  I would note that the abstract of RFC 1997 states that BGP community has been defined to "reduce the management complexity of maintaining the Internet". 

And +1 on Rob's comment

Regards,
Bruno

> From: Black, David [mailto:[email-masked]] > Sent: Tuesday, October 06, 2015 1:35 AM
> 

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Black, David" <[email-masked]>
Date: Tue, 6 Oct 2015 16:44:18 +0000

Rob,

> Given that we are really selecting candidates from within a set of paths that
> have already been selected by OSPF as valid, and usable - then I’m not sure
> that I can understand the logic behind this sentence from your review: "There
> appears to be more than enough enabled by this draft to wreak serious
> operational havoc”.

Perhaps, I'm not understanding something, but I thought I saw an unreachability
problem in the example in Section 4.5/Figure 3:

Was that a mistake, and at least one path from the leftmost pair of A nodes
to the I nodes will be selected despite that "explicitly routed policy"?

Thanks,
--David

> -----Original Message-----
> From: Rob Shakir [mailto:[email-masked]]
> Sent: Tuesday, October 06, 2015 11:23 AM
> To: [email-masked]; Black, David; [email-masked]; [email-masked];
> [email-masked]; General Area Review Team ([email-masked]);
> [email-masked]
> Cc: [email-masked]; Black, David; [email-masked]; [email-masked]
> Subject: Re: Gen-ART and OPS-Dir review of draft-ietf-ospf-node-admin-tag-06
> 
> Hi David,
> 
> I’ll quote this part of your mail, but respond to the general question around
> whether this draft should be tighter scoped (and the operational
> considerations of what one can do with the mechanism).
> 
> In general, this mechanism is meant to give ways for an operator to add their
> own logic on-top of the routing protocol selection of the OSPF protocol.
> Specifically, this is for the case where there are sets of paths that OSPF
> considers valid, but the operator has a particular preference for a particular
> subset to be included or excluded. To this end, the harm that one can do by
> preferring paths with a particular admin tag (or set of admin tags) on them is
> simply to select a subset of the paths that would have been used anwyay.
> 
> 	* Prefer a specific LFA from a set of valid LFAs. The operator has a
> business reason to prefer specific ones (e.g., OSPF metric is set based on
> bandwidth, but we want to minimise latency, or we only want to capacity plan
> for FRR between P nodes never transiting a PE). There are many different
> reasons that one might have this preference, and they will be different by
> operator - so, removing the ability of an operator to specify arbitrary tags
> will make this mechanism ineffective. Whilst there could be unintended
> consequences of preferring specific LFAs, actually there are probably worse
> consequences of NOT preferring particular LFAs, such as traffic going via a PE
> in another country such that latency is increased over all other LFAs. Bruno
> and Stephane from FT have given many examples of this in the LFA manageability
> drafts presented in RTGWG. The admin tag will never result in a new LFA that
> would never have been selected via the standardised mechanisms being selected
> - so, in my view, has little potential to cause new ‘harm’.
> 
> 	* Prefer a specific path from a set of paths. This is similar to the
> case above. No new candidate path is being added to the routes that could be
> used by the admin tag mechanism. It simply says that when a device that must
> select N ECMPs to install from M candiates (N < M), then the operator can
> influence which based on the admin tag. All are still valid ECMPs - so new
> risk is created. Again, selecting a particular subset of that M without any
> consideration may be more harmful than doing the operator-specific selection.
> Again, the logic of which are ‘OK’ PEs and which are not is something that is
> operator specific, and may only apply to a subset of applications using OSPF
> to select paths (an example of this is devices that do not scale well as MPLS-
> TE midpoints, but are OK for transit IP traffic - at this point, we may want
> to influence CSPF to stay away from them, but not IP - the fact that these
> devices are ‘not-OK-for-transit-MPLS-TE’ is not something that we can easily
> enumerate and standardise).
> 
> Given that we are really selecting candidates from within a set of paths that
> have already been selected by OSPF as valid, and usable - then I’m not sure
> that I can understand the logic behind this sentence from your review: "There
> appears to be more than enough enabled by this draft to wreak serious
> operational havoc”.
> 
> I strongly disagree with the idea that we should try and enumerate what the
> tags should be, rather than maintain the ability of an operator to add their
> own specific logic with these tags. This is what is done today with link
> affinity, or tags elsehwere in the IGP protocols.
> 
> Re-reading the draft, perhaps we need to clarify that this does not change the
> underlying SPF process that OSPF uses today - would this address your concern?
> 
> Regards,
> r.
> 
>

---

From: Rob Shakir <[email-masked]>
Date: Tue, 6 Oct 2015 18:21:53 +0100

On October 6, 2015 at 17:46:41, Black, David ([email-masked]) wrote:
> to the I nodes will be selected despite that "explicitly routed policy”?

If the operator chooses to deny prefixes being installed in the RIB based on these tags, then yes, they could end up with unreachability problems. However, an operator can do this today with any routing policy (a number of implementations already have inbound route filtering) - we should not prevent this kind of mechanism based on the fact that an erroneous config might be problematic.

In the case that the operator *preferences* things based on the tags, then this would not be an unreachability problem - OSPF would still correctly determine that there is a path between all nodes in the pictured network - and this would be installed in the RIB as per normal operation.

(My memory is not 100% clear on whether this is intended as part of the example, if it is, then the text should be clarified I agree.)

Kind regards,
r.