Subject: Re: [Sidrops] [WGLC] draft-ietf-sidrops-8210bis-03 - Ends 7/December/2021

From: Randy Bush <[email-masked]>
Date: Thu, 02 Dec 2021 08:46:23 -0800

tim,

i am confused, so i keep going back to your messages on ASPA.  i just
don't get it.  you'll have to try a new clue bat.

the RP cache's job is to deal with all the RPKI cruft, crypto, alg
rolls, transfers, whatever.  the job of the cache is to insulate the
router from all that.

in the ASPA case, the cache takes what it knows, sums it up (as in set
union), and whacks it off to the router in completely processed form.

what am i missing here?

randy

---

From: Tim Bruijnzeels <[email-masked]>
Date: Fri, 3 Dec 2021 09:14:20 +0100

Morning Randy,

> On 2 Dec 2021, at 17:46, Randy Bush <[email-masked]> wrote:
> 
> tim,
> 

yes, agree.

> union), and whacks it off to the router in completely processed form.

I am reading the document again, and I see that I was confused.

I read: "Therefore, the cache MUST deliver the entire data of an ASPA
record in a single ASPA PDU." to mean that RPs MUST make a PDU based
on a single ASPA object.

Only to miss that just below it you continue to say:

   The router should only see one ASPA for a particular Customer
   Autonomous System Number active at any time.  This may place a burden
   on the cache to merge multiple ASPA records it has received from the
   global RPKI into one ASPA PDU.

The above is exactly what I was trying to say..

> what am i missing here?

Sorry for my confusion on this, and thank you for making me re-read!

Tim

---

From: Tim Bruijnzeels <[email-masked]>
Date: Fri, 3 Dec 2021 10:28:17 +0100

Randy, all,

after some more coffee..

> On 3 Dec 2021, at 09:14, Tim Bruijnzeels <[email-masked]> wrote:
> 

In my defence ;) I think the above could benefit from some more explicit
language. Would you agree to something like the following (feel free to
rephrase - but to give you something to shoot at):

  The router should only see one ASPA object for a particular
  Customer Autonomous System Number active at any time, as issuing
  CAs must register multiple providers for a single customer in
  a single object (section 3 of [I-D.ietf-sidrops-aspa-profile]).
  However, despite this requirement on CAs, edge conditions may
  result in multiple valid ASPA objects for a single customer
  to appear in the global RPKI. This places a burden on the cache
  to merge multiple records into one ASPA PDU.

Tim

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 09:36:18 -0800

> I am reading the document again, and I see that I was confused.

i must have looked at your messages a dozen times over a week trying to
see what i did not understand. :)

aside from removing the duplicated paragraph,

   The Provider AS Count is the number of Provider Autonomous System
   Number(s) at the end of the PDU, and MUST be one or more.

is there something we could have done to be more clear?

randy

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 09:38:32 -0800

>   However, despite this requirement on CAs,

not sure i want to dive into the complications of aspa profile.

but i think i will steal this if you do not mind

>   edge conditions may result in multiple valid ASPA objects for a
>   single customer to appear in the global RPKI. This places a burden
>   on the cache to merge multiple records into one ASPA PDU.

randy

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 09:46:36 -0800

The router should see at most one ASPA for a particular Customer
   Autonomous System Number active at any time.  As a number of
   conditions in the global RPKI may present multiple valid ASPA objects
   for a single customer to a particular RP cache, this places a burden
   on the cache to merge multiple ASPA records it has received from the
   global RPKI into one ASPA PDU.

ok?

---

From: Job Snijders <[email-masked]>
Date: Fri, 3 Dec 2021 18:49:48 +0100

>    global RPKI into one ASPA PDU.

What to do when the router receives dissimilar ASPA PDUs via different
RPKI-To-Router sessions? Another merge?

Kind regards,

Job

---

From: Tim Bruijnzeels <[email-masked]>
Date: Fri, 3 Dec 2021 19:00:26 +0100

> On 3 Dec 2021, at 18:46, Randy Bush <[email-masked]> wrote:
> 

Works for me, thanks!

> 
> _______________________________________________
> Sidrops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidrops

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 10:03:56 -0800

> What to do when the router receives dissimilar ASPA PDUs via different RPKI-To-Router sessions? Another merge?

i thought that a router specified a priority on each member of
a set of RP caches.  having a hard time finding that.

i would be very hesitant to suggest the router form the union,
as it then has to backtrack if one is withdrawn.

yuchh

randy

---

From: "Borchert, Oliver (Fed)" <[email-masked]>
Date: Fri, 3 Dec 2021 18:20:03 +0000

Randy,

Can you explain to me why we treat ASPA PDU different than for instance ROA PDU's with a single AS-prefix tuple?
What is the downside of encoding the ASPA PDU as a single customer-provider tuple?

The router would receive one PDU for each defined Customer-Provider tuple, same as for ROAS where the router receives a one PDU for each AS-prefix tuple.

What is the benefit for the router to add the complexity of the "implicit" withdrawal that I pointed out in one of my earlier emails?

I personally believe it makes more sense to have all data PDU's processed in a unified form. ROA and Key PDU's do that, ASPA is completely different.

Oliver

-----------------------------------------------
Oliver Borchert, Computer Scientist
National Institute of Standards and Technology
(Office) +1.301.975.4856
(GVoice) +1.240.668.4117

On 12/3/21, 12:50 PM, "Sidrops on behalf of Job Snijders" <[email-masked] on behalf of job=[email-masked]> wrote:

    Kind regards,

    Job

    _______________________________________________
    Sidrops mailing list
    [email-masked]
    https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fsidrops&amp;data=04%7C01%7Coliver.borchert%40nist.gov%7Cf6c78c7a110d44b8d26a08d9b6855bc9%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C0%7C637741506203139444%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=nhbJ2jpqGyg7xJhWT%2F32aGT52%2F4cBM3VRCnwJ5%2Frhtc%3D&amp;reserved=0

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 10:20:48 -0800

oliver,

i have been confused here too

> The current version of the draft mentions a router only contains one
> single APSA object per customer. This assumes a specific form of
> implementation. I don't believe a protocol specification should do
> that. One can imagine an implementation that stores the data as link
> relations hence multiple internal objects per customer. Take 10
> implementers and you might end up with 10 solutions.

and those implementation variations are an internal matter for the RP,
and should not affect the router.  the job of the RP is to make life
simple for the router by presenting one and only one current ASPA PDU.

of course, ten router vendors could have ten ways of handling their
internal data structures.  but the ASPA PDU is the narrow waist.

> I always read to keep things simple - Great I like simple. Though, as
> I outlined in my email from last week, the current specification
> introduces implicit withdrawals while announcing an ASPA object. That
> contradicts simplicity - it adds additional complexity to the router.

the alternative is requiring the withdraw to be explicit which places
analogous burdens on the router to handle the error condition if it
receives more than one announcement with no intervening withdraw.  win
some, lose some.

> As I mentioned in my email as well as was mentioned in prior IETF's a
> router should never perform any validation on incomplete data. The
> moment the router starts updating the data, it most likely will
> encounter a break before make situation - at least when it comes to
> ROA's. Therefore the router should always wait until a complete update
> is processed prior acting upon it.
> 
> Therefore, "Keep it simple" should result in a PDU as I proposed with
> only a single Customer Provider. An ASPA object that contains 10
> providers should result in 10 PDU's each with the customer and a
> single provider. Duplicates are treated as established, withdrawals
> are treated as established. No special handling required on the
> router's side.

i hear you; and would seriously consider it if we were doing this de
novo.  but, as you say, router vendors undoubtedly have varied internal
data structures.  as we did not place a receive-all-before-processing
requirement on them in v0 and v1, doing so now would seem to be a major
pola violation.

hence we used SHOULD instead of MUST in

   11.  ROA PDU Race Minimization

   When a cache is sending ROA PDUs to the router, especially during an
   initial full load, two undesirable race conditions are possible:

   Break Before Make:  For some prefix P, an AS may announce two (or
      more) ROAs because they are in the process of changing what
      provider AS is announcing P.  This is is a case of "make before
      break."  If a cache is feeding a router and sends the one not yet
      in service a significant time before sending the one currently in
      service, then BGP data could be marked invalid during the
      interval.  To minimize that interval, the cache SHOULD announce
      all ROAs for the same prefix as close to sequentially as possible.

   Shorter Prefix First:  If an AS has issued a ROA for P0, and another
      AS (likely their customer) has issued a ROA for P1 which is a sub-
      prefix of P0, a router which receives the ROA for P0 before that
      for P1 is likely to mark a BGP prefix P1 invalid.  Therefore, the
      cache SHOULD announce the sub-prefix P1 before the covering prefix
      P0.

how much data would a router have to buffer and for how long and with
what end-of-session timeout?

if we now tried to place a receive-all-before-processing requirement on
routers, would there be effects on other PDUs and requirements?

randy

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 10:51:27 -0800

oliver

> Can you explain to me why we treat ASPA PDU different than for
> instance ROA PDU's with a single AS-prefix tuple?  What is the
> downside of encoding the ASPA PDU as a single customer-provider tuple?

same reason as last time we followed the woozle tracks around this tree.

does a router have to add and substract from a set of valid providers,
or does it simply receive the current set?  does the cache have to
remember what it told the router so it can withdraw appropriately[0]?
does the cache need to accumulate all updates for a customer so it can
send sequentially to reduce race condition exposure with incoming bgp?

randy

[0] - fyi, my routers log symptoms of inappropriate roa withdrawals
      for various RPs

---

From: "Borchert, Oliver (Fed)" <[email-masked]>
Date: Fri, 3 Dec 2021 18:51:34 +0000

I understand the make before break fear in ASPA but it is already a big issue with ROA's.
Maybe even a bigger issue if you drop invalids.

I also do not see that the send shorter prefix first solved the problem for ROA's. This does not solve the issue of two ASes that are allowed to announce the same prefix. Here AS-a and AS-b could announce Prefix-p and if the PDU for [AS-a, Prefix-p] arrives and gets processed ahead of [AS-bm Prefix-p] with the second route being the selected one, the second one (selected) would be immediately become invalid in case the router acts upon each PDU immediately. That would result in a withdrawal or switch followed by a switch back once the second ROA PDU is received. Therefore this too can cause a route flip or complete withdrawal.

Hence, the only proper solution is to wait until all data is updated. And if not then the ROA PDU too should use the format of 
prefix-AS-AS-AS-AS combined with smaller first. 

The main issue I have is that for the same problem space (racing condition) ROA and ASPA PDU's are encoded differently as it is right now. Currently, with ROA's the router can be dumb, add or remove, or send an error for duplicates. For ASPA not so much, it might need to examine the delta. And then what about duplicates here, they are allowed. Different behavior for the same situation which I think is not good. 

The argument for timeout etc. is the same when one needs to reload a complete caches worth of data. What I observed when experimenting with commercial router implementations is that I cannot recall to have seen a "serial-query". I believe to remember only to have seen "reset-query" responses to "serial-notifies" with a table switch once the transmission was done, but I might be mistaken.

It would be nice to hear what the vendors on the list have to say, they have to implement it at the end,

Oliver

On 12/3/21, 1:20 PM, "Randy Bush" <[email-masked]> wrote:

    oliver,

    randy

---

From: Randy Bush <[email-masked]>
Date: Fri, 03 Dec 2021 11:42:05 -0800

> [0] - fyi, my routers log symptoms of inappropriate roa withdrawals

disclosure:  i have two hats

i work for a vendor, arrcus.

i am also work as a research fellow at iij, a large isp.  in that role,
i am part of a research commune which has a few research racks in space
kindly donated by folk such as equinix and google.  these racks have
routers from various vendor friends.

the research racks do not have an arrcus box running rov.  think
cobbler's children.

so the note above is not with my arrcus vendor hat, but with my
researcher hat and on well known reputable hardware from other vendors.

randy