Subject: [Last-Call] Opsdir last call review of draft-ietf-sidrops-ov-egress-01

From: Linda Dunbar via Datatracker <[email-masked]>
Date: Tue, 17 Mar 2020 14:53:35 -0700

Reviewer: Linda Dunbar
Review result: Not Ready

I have reviewed this document as part of the Ops area directorate's ongoing
effort to review all IETF documents being processed by the IESG.  These
comments were written primarily for the benefit of the Ops area directors.
Document editors and WG chairs should treat these comments just like any other
last call comments.

The document claims that it highlights an important use case of origin
validation in eBGP egress policies, explaining specifics of correct
implementation in this context. But I don't see where the "Use Case" is
described. It seems to me that the document should have more sections to
describe the Use Case and the procedure of Egress Export of the RPKI validated
Origins.

Section 3 Egress Processing only has one sentence stating that
"When applied to egress policy, validation state MUST be determined using the
effective origin AS of the route as it will (or would) be announced to the
peer." What other choices there are ?   Are there any routers that support  RFC
6480 RPKI  not performing this step? how?

My two cents.

Linda Dunbar

---

From: Chris Morrow <[email-masked]>
Date: Wed, 18 Mar 2020 00:11:46 +0000

Just sniping a reply... which may be out to left (or right!) field :)

On Tue, 17 Mar 2020 21:53:35 +0000,
> 6480 RPKI  not performing this step? how?

I think the sense of this sentence is really:

  "Your router may have configured ASXXX and present itself
  (confedaration) as ASYYY to peer1 and ASBBB to peer2
  (merger/acquisition issues). Please make sure that you validate the
  origin accordingly"

There are a few example networks in the real world where this sort of
thing could be super relevant. Particularly look at recent 'network
that merges a bunch' Level3/CenturyLink/Savvis/GlobalCrossing/etc...
AS3356/AS209/AS<iforget>/AS3459/AS3549. A single edge device
terminating customers on the 3356 backbone may plausibly have
customers that haven't swapped over to 3356 and are actually expecting
the remote peer to be any of 6 or more ASN.

It's important that the 'i originated this prefix, I'm 3356, wait
3549, want 3459...' decision is backed up in ROA content as well
as validation logic.

Looking at the abstract to maybe dig a little deeper:
  "For egress policy, it is important that the
   classification uses the effective origin AS of the processed route,
   which may specifically be altered by the commonly available knobs
   such as removing private ASs, confederation handling, and other
   modifications of the origin AS."

So somewhere deep in my network, bb01.pop01 has:
  192.0.2.0/24

staticly routed, redistributed in BGP and marked for external
announcement (community/etc). That route, internally has an origin-AS of:
  65507

because that's one of the several private ASN that make up my
confederation: ASBAR. So, when I announce:
  192.0.2.0/24 - origin 65507

I need to both swap the origin in the bgp announcement and validate
against the local-as the peer sees me as.
   192.0.2/0/24 - origin 3356
   192.0.2.0/24 - origin 3549
   ...etc...

This will require me to make ROA for all of these origins, as well.

I think anyway that's the goal of the document, really.
though, also, I could have missed the forest for the trees here.

-chris

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 18 Mar 2020 00:44:29 +0000

> 6480 RPKI  not performing this step? how?

jumping the gun on the authors here, there's a mismatch between what 
coders implemented and what operators figured would be workable in terms 
of how policy semantics need to be handled on production routers.

This is a fancy way of saying that some router vendors ticked the ROV 
tickbox, but the way they did it was too limited for real life.

RPKI provides a policy management mechanism.  The ID says that this 
needs to be hooked into the three major policy application points which 
are implemented in most, if not all, bgp rib engines.  These points are:

1. bgp ingress, i.e. at the point between the adj-rib-in and loc-rib
2. on redistribution to other routing protocols
3. bgp egress, i.e. at the point between the loc-rib and adj-rib-out

This didn't happen for ROV on all vendor stacks.

It's kinda obvious to operators that it should have been.

Also, it's not the first time that this sort of feature lapse has 
happened in the history of routing management.  There's been a bit of a 
history of this sort of thing happening over the years.

Nick

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Mar 2020 18:16:02 -0700

linda,

thanks for the review

> Section 3 Egress Processing only has one sentence stating that ...

the lack of more words was not an accident; a feature not a bug. :)

> "When applied to egress policy, validation state MUST be determined
> using the effective origin AS of the route as it will (or would) be
> announced to the peer."   What other choices there are ?  Are there any
> routers that support RFC 6480 RPKI not performing this step? how?

unfortunately yes.  see nick's and chris's emails.

first, too many implementations do not even apply rov policy on locally
originated routes, routes from ibgp, etc.

second, for rov processing, many, maybe most, implementations use the
'global' AS of the router (as if there was one giving multi-AS knobs).

chris's email kinda explains that last one pretty clearly.

the purpose of this draft is to explain that the problem exists and to
make the minimal statement of what implementations should do.  i believe
the text (i need to read robert's email three more times) is well
understood by rov implementors, and even serious bgp/rov ops.

randy

---

From: Job Snijders <[email-masked]>
Date: Wed, 18 Mar 2020 01:26:35 +0000

On Wed, Mar 18, 2020 at 12:44:29AM +0000, Nick Hilliard wrote:
> > Section 3 Egress Processing only has one sentence stating that "When
> > applied to egress policy, validation state MUST be determined using
> > the effective origin AS of the route as it will (or would) be
> > announced to the peer." What other choices there are ?   Are there
> > any routers that support  RFC 6480 RPKI  not performing this step?
> > how?
> 
> coders implemented and what operators figured would be workable in
> terms of how policy semantics need to be handled on production
> routers.
> 
> needs to be hooked into the three major policy application points
> which are implemented in most, if not all, bgp rib engines.  These
> points are:
> 
> 2. on redistribution to other routing protocols 3. bgp egress, i.e. at
> the point between the loc-rib and adj-rib-out
> 

Perhaps:

OLD:
    It might be affected by removal of private AS(s), confederation, AS
    migration, etc.  If there are any AS_PATH modifications resulting in
    origin AS change, then these MUST be taken into account.

NEW:
    BGP implementations have to take removal of private AS(s),
    confederation, AS migration, etc into consideration. If there are
    any AS_PATH modifications resulting in an Origin AS change, then
    these MUST be taken into account and only the final Origin AS is to
    be used as input into the Origin Validation procedure.

Kind regards,

Job

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Mar 2020 18:36:23 -0700

> confederation, AS migration, etc into consideration.

and the winds of summer.  we do not need to enumerate all knobs, as
if we could.

randy

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 18 Mar 2020 04:28:11 +0000

Chris, 

Thank you very much for the explanation.
Your explanation is what I was looking for. 
It would be very nice to have your examples included (as Use Cases) in the document. 

Linda Dunbar
-----Original Message-----
From: Chris Morrow <[email-masked]> 
Sent: Tuesday, March 17, 2020 7:12 PM
To: Linda Dunbar <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: Opsdir last call review of draft-ietf-sidrops-ov-egress-01

On Tue, 17 Mar 2020 21:53:35 +0000,
Linda Dunbar via Datatracker <[email-masked]> wrote:
> 
> Reviewer: Linda Dunbar
> Review result: Not Ready
> 
> I have reviewed this document as part of the Ops area directorate's 
> ongoing effort to review all IETF documents being processed by the 
> IESG.  These comments were written primarily for the benefit of the Ops area directors.
> Document editors and WG chairs should treat these comments just like 
> any other last call comments.
> 
> implementation in this context. But I don't see where the "Use Case" 
> is described. It seems to me that the document should have more 
> sections to describe the Use Case and the procedure of Egress Export 
> of the RPKI validated Origins.
> 
> the effective origin AS of the route as it will (or would) be announced to the

There are a few example networks in the real world where this sort of thing could be super relevant. Particularly look at recent 'network that merges a bunch' Level3/CenturyLink/Savvis/GlobalCrossing/etc...
AS3356/AS209/AS<iforget>/AS3459/AS3549. A single edge device terminating customers on the 3356 backbone may plausibly have customers that haven't swapped over to 3356 and are actually expecting the remote peer to be any of 6 or more ASN.

It's important that the 'i originated this prefix, I'm 3356, wait 3549, want 3459...' decision is backed up in ROA content as well as validation logic.

staticly routed, redistributed in BGP and marked for external announcement (community/etc). That route, internally has an origin-AS of:
  65507

I need to both swap the origin in the bgp announcement and validate against the local-as the peer sees me as.
   192.0.2/0/24 - origin 3356
   192.0.2.0/24 - origin 3549
   ...etc...

-chris

---

From: Ben Maddison <[email-masked]>
Date: Fri, 20 Mar 2020 09:57:42 +0000

> and the winds of summer.  we do not need to enumerate all knobs, as
But enumerating *some* helps the reader understand the context in which
this is important/useful.

Cheers,

Ben

---

From: Randy Bush <[email-masked]>
Date: Fri, 20 Mar 2020 09:08:53 -0700

> this is important/useful.

which is why

   When applied to egress policy, the effective origin AS MUST be used
   to determine the Origin Validation state.  The effective origin AS is
   that which will actually be the origin AS in the announcement.  It
   might be affected by removal of private AS(s), confederation, AS
   migration, etc.

randy

---

From: Warren Kumari <[email-masked]>
Date: Fri, 20 Mar 2020 12:33:19 -0400

>    might be affected by removal of private AS(s), confederation, AS

One of the things that I personally like most about this document is
that it concise - it clarifies something for *implementors* to keep in
mind / points out something where they might trip over something and
hurt themselves.

While I generally like examples and detail in RFCs, in this case it
would be "teaching your grandmother to suck eggs"[0] - the readers in
this case are folk who write BGP, the document basically says
"Warning: sharp object, careful with fingers" - having too much detail
decreases the utility in this case.

Again, I generally agree that documents should enumerate all of the
corner cases, have examples, etc - but in this case I think it would
do more harm than good...

W
[0]: https://en.wikipedia.org/wiki/Teaching_grandmother_to_suck_eggs

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Ben Maddison <[email-masked]>
Date: Sat, 21 Mar 2020 07:59:12 +0000

Hi Warren,

> mind / points out something where they might trip over something and
I only partially agree. For three reasons:

- My experience over the last 12 months of running invalid==deny in
production is that some implementors have managed to take every
ambiguity in the various rov-related RFCs and translate it into bugs or
fragile/hostile behavior. I think our lesson should be that even
obvious-seeming spec gaps ought to be filled in this area.

- Operators using implementations of this draft will observe
behavioural differences between similar-seeming policy applied at
different attachment points. This document should help them understand
those differences.

- We may very well see some implementations wind up with separate
policy knobs for "enable rpki-rov" and "enable rpki-rov-egress". Again,
operators will need a spec against which to validate that feature
behaviour if it is claiming RFC compliance.

> do more harm than good...
I'm not suggesting turning this into a use-case doc. But enough color
to make the mental link between real-world policy and protocol concepts
is necessary, imho.

Cheers,

Ben

---

From: Randy Bush <[email-masked]>
Date: Sat, 21 Mar 2020 10:32:53 -0700

> ...

since we're not getting it clearly, so send a short paragraph or two

randy