Subject: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

From: Jeffrey Haas <[email-masked]>
Date: Tue, 1 Mar 2022 16:21:51 -0500

Working Group,

One of the topics for our last IDR interim was an attempt to conclude
discussions on BGP autoconfiguration proposals prior to sending adoption
requests to the Working Group.  Just prior to that call going out, we were
requested to add one additional proposal for consideration that hadn't yet
been published: l3nd[1] and l3nd-ulpc[2].

The authors have requested a presentation slot at the upcoming IETF 113 for
this proposal.  Since this is a very late entrant, I'd like to try to get
discussion of the properties of this proposal started on the mailing list
using the same discussion points we've had from prior interim meetings.

The Working Group adopted requirements and some of the prior proposal
discussions are part of the design team document.[3]

Below, find my observations on the draft with questions and commentary
intermixed.  I finish this response with my high level analysis and personal
opinion on what these properties accomplish.

-----

Scoping discussion:

The l3nd proposal appears to be designed to start from similar premises as
l3dl.  l3dl has been previously discussed as a candidate for a layer 2 BGP
autoconfiguration proposal and is covered in [3]A.1.2.

:   This document (set) provides a similar solution at Layer-3,
:   attempting to be as similar as reasonable to L3DL.

The intended scoping is point to point links:

:   In this document, the use case for L3ND is for point to point links
:   in a datacenter Clos ([Clos])

This is somewhat unusual language since the typical deployment interface is
Ethernet.  While such Ethernet connections are typically deployed in a point
to point fashion, Ethernet itself is not specifically point to point.

There is also a pointer that the proposal may be intended for more general
purposes than BGP Autoconfiguration:

:   L3ND might be found to be widely applicable to a range of routing and
:   similar protocols which need Layer-3 neighbor discovery.

The proposal appears to be intended to be able to expanded to non-data
center cases.

:   While L3ND is designed for the MDC, there are no inherent reasons it
:   could not run on a WAN.  The authentication and authorization needed
:   to run safely on a WAN need to be considered, and the appropriate
:   level of security options chosen.

Such wider scoping was permitted by the autoconfiguration analysis but such
work was left as out of scope for the DC case.

l3nd seems to be targeting different levels of scale than prior proposals,
and that appears to motivate a different property:

:   The number of addresses of one Encapsulation type on an interface
:   link may be quite large given a TOR switch with tens of servers, each
:   server having a few hundred micro-services, resulting in an
:   inordinate number of addresses.  And highly automated micro-service
:   migration can cause serious address prefix disaggregation, resulting
:   in interfaces with thousands of disaggregated prefixes.
:
:   To meet such scaling needs, the L3ND protocol is session oriented and
:   uses incremental announcement and withdrawal with session restart, a
:   la BGP ([RFC4271]).

Given that the desired scoping context is point to point (typically a small
number of addresses on a given link), it'd be helpful if the authors discuss
scenarios wherein a large number of BGP sessions are expected to be
discovered on such a link.

-----

Transport layer considerations:

l3nd leverages a multicast hello message and requires a TCP (possibly
protected by TLS) session to do its work.

In comparison to other proposals previously discussed:
- draft-xu-idr-neighbor-autodiscovery was session based, with comparability
  to the LDP hello mechanism.  It uses TCP after multicast discovery.
- l3dl itself is session based and implements its own transport and session
  layers. 
- draft-raszuk-idr-bgp-autodiscovery, determined to be not a fit for the
  core data center case, used "BGP over BGP".
- draft-acee-idr-lldp-peer-discovery operates over layer 2 embedded in the
  LLDP protocol.  It is currently sessionless.  Discussion has suggested
  that if, in the future, security or large packet considerations drive
  requirements to need larger packets that LLDPv2 that is being currently
  stanardized in IEEE can serve those purposes.  LLDPv2 provides a session
  layer.
- draft-minto-idr-bgp-autodiscovery uses layer 3 IP multicast exclusively
  and does not have a session layer.

-----

Protocol state and FSM considerations:

- Similar to BGP, parallel session are a possibility.  I'm unclear how
  connection collisions are handled.
  + The procedures seem to try to cover this by saying "drop hello when
    there is an established session".  However, timing could result in two
    sessions happening in parallel prior to drop.
  + There isn't a clear idea of the FSM having an "established" state, or
    the related "OpenSent/OpenConfirm" states similar ot  BGP that could
    prevent this?

Open message:
- If Attributes are single-octet values of local significance, is there any
  significance to receiving the same Attribute twice?  If not, consider
  adding text about their uniqueness.
  + Also, why so small?  Payload length appears to be 4 octets.  A larger
    value permits more flexibility for local semantics; e.g. bit vectors.
  + Is there significance to resuming a session using the documented
    procedures when the Attributes may not be identical to the ones previously
    received in the interrupted session?
- For clarity, consider labeling Serial Number as "last received serial
  number".

Ack message:
- The nybble next to Etype needs some flavor the usual text to deal with its
  contents.  "Must be zero on transmit, should ignored on receipt"
- The Etype conditions are a bit lax for a useful state machine for what we
  have documentation for:
  + If Open can't be acked successfully, should the session be permitted to
    continue?
  + Is "restart is hopeless" a shutdown?  An admindown?  What should the
    implementation do with its PDUs?  Should it tear down the session?
- 3 seconds seems a bit low for a response timeout.  While the protocol
  seems to be structured toward easy restart, why disconnect so
  aggressively?
- The message exchange protocol is "one outstanding ack" which means there
  is no pipelining of the various messages.  Given this, the lack of a
  serial number being acked in the message is understandable, but seems like
  it'd be good debug info.
  + That lack of pipelining means that full exchange of state will be gated
    on slowest sender on the session; the transmitter or the acknowledger..

Encapsulations:
- The text for addressing conflicts needs to specify the Etype along with
  the Error Code.
  + It can't be 0, because that says No Error.
  + It's probably 1 as a warning?
  + In general, normative text needs to specify Etype + Error Code together.
- 10.1, the procedure for expecting an ACK seems conflicting with the
  retransmission text in 9.1.  In that, it says if you're expecting an ack
  and don't get one, the session should be closed.  However, it's implied
  (but unclearly so) that you should have one pending ack at a time.  If
  retransmits of an encapsulation pdu are possible, this means multiple
  outstanding acks may be needed.  In such a case, serial numbers being
  acknowledged are more desirable.
  + In general, the text here for retransmit vs. ack seems to be a mismatch
    between the expectations of working on a reliable stream protocol like
    TCP.
  + Similarly, the "link is broken below layer 3" comment seems a mismatch
    since TCP shouldn't pay direct attention to that.
- 24 bits is a peculiar length for a counter.  Why this size?

Encapsulation Flags:
- 10.2 typo on Encapsulation
- Primary is interesting operational state.  What should an implementation
  do if it gets conflicting primary entries?

Encapsulations (varying):
- You can pack more than one prefix in a PDU for a given serial number.  In
  the event that a single prefix is an error (see question about about Etype
  for such cases), are all prefixes in the bundle intended to be rejected? 
  + How would the ack receiver know which prefix is the bad one if the whole
    bundle isn't rejected?

MPLS Label List/Encapsulation:
- After our "fun" dealing with label issues covering RFC 3107 and others,
  please make sure to tell people the obvious: the label stack needs to be
  well formed and bottom of stack bit should only be set on last entry.

Hello discussion:
- The English of the first paragraph is in need of significant work for
  clarity.
- I think this is trying to say that in the presence of a LAG that
  transmission over more than one LAG member might be okay.
- I also think that it's trying to say that once you have a session up you
  can stop hellos on the individual member links.
- I think the intent of the second paragraph is that if a link is on a
  multi-access network and non-p2p behavior is desired, keep sending hellos.

ULPC draft:

3.1:
:   A peer receiving BGP ULPC PDUs has only one active BGP ULPC PDU for
:   an particular address family on a specific link at any point in time;
:   receipt of a new BGP ULPC PDU for a particular address family
:   replaces any previous one.

Okay, implicit replace.  That's fine.

:   If there are one or more open BGP sessions, receipt of a new BGP ULPC
:   PDU does not affect these sessions 

Do not disturb existing sessions.  That's fine.

:   and the PDU SHOULD be discarded.

Probably not fine.  If the session bounces, you likely intend for it to
connect to the newly advertised entry.

:   If a peer wishes to replace an open BGP session, they MUST first
:   close the running BGP session and then send a new BGP ULPC PDU.

This seems to assume a tight coupling between the implementations for BGP
and for the discovery mechanism.  A peer bounce without having first
advertised the new address can mean a race condition where the bounced peer
continues trying (and possibly failing) to establish the connection with the
old peer.

It'd be cleaner to remove the prior binding first.

However, there's no withdraw procedure currently specified for these TLVs to
remove peer addresses, only to advertise them.

:   For each BGP peering on a link here MUST be one agreed encapsulation,
:   and the addresses used MUST be in the corresponding L3DP IPv4/IPv6
:   Announcement PDUs.  If the choice is ambiguous, an Attribute may be
:   used to signal preferences.

How?  Among other things, the attributes are bound to a session and not any
individual address.

3.1.2/3.1.3 Prefix-Len is probably not appropriate for the addresses.  If
there's an intent to permit peering from more than one address in the
subnet, how should the implementation pick?  From state learned in l3nd
encapsulation bindings for an overlapping subnet?

3.1.4 Authentication: You're referring to multiple possible ways security
could work.  This likely means multiple authentication types and should be
its own code point.

3.1.5 "No flags are currently defined"... and then defines GTSM.

:   As the ULPC PDU may contain keying material, see Section 3.1.4, it
:   SHOULD BE signed.

What does signed mean in this instance?  Do you mean that the transport
session should be given secrecy properties by TLS?

:   Any keying material in the PDU SHOULD BE salted and hashed.

I'm very confused here.

The material that a BGP peer needs for bringing up a session is the key to
pass to the relevant algorithm like TCP-MD5 or TCP-AO.  Typically when we
discuss salt and hashing, we're discussing the ciphertext from those
operations.  

Are you suggesting you send the ciphertext with the intent that an
implementation can consult its own tables to find what input password for
that salt should be used?  If so, this is a flavor of pre-shared key.  It'd
also require coordination of salt values between the implementations.

If this is an obvious technique that I'm not familiar with, a pointer to the
procedure would be useful.

-----

Misc:
- The embedded attempts at humor decrease the clarity of the document for
  readers that are less familiar with English and should be excised.
- In the PDU diagrams, explicitly document what the field lengths are.
  Don't make people interpret ASCII diagrams.
- The HELLO packet is itself without security.  In circumstances in which a
  man-in-the-middle attack is possible and raw TCP is offered as an option,
  active interception and replacement of the HELLO is sufficient to cause 
  the less secure mechanism to be used.
- Section 7, the text about GTSM as a SHOULD on SYN should simply be GTSM on
  the TCP session.  GTSM applies to all of the packets in a TCP session.
- Section numbering under encapsulations could be more consistent.  For
  example, each section for a PDU type.  MPLS Label List probably shouldn't
  be a section at the same level as the mpls encapsulation pdus that use it.
- The session is long-lived but not protected similarly to BGP; i.e. tcp-md5
  or tcp-ao.  Given that the intent of the mechanism is to setup BGP which
  may use those properties, but uses TLS for protection, this isn't
  surprising.  But it means that the sessions are vulnerable to reset attacks
  those TCP mechanisms are intended to protect against.

-----

Jeff's read of the proposal:

1. The proposal is session based.  While the prior discussion was primarily
in the context of draft-xu, sessions were seen as potentially problematic
for a number of reasons.  They also had some possibly useful properties.
1a. Session based makes large amounts of state easy to deal with.  That
said, the requirements show we need very little state for a given session.
That's true even in the ULPC portion of this proposal.
1b. These sessions are long-lived.  You'll have one for each interface you
want to run auto-discovery on.  This minimally doubles the number of active
sockets your stack will need because for each BGP session you get, you'll
have a lingering discovery session.  Perhaps they don't need to be
long-lived if the only thing you're using this for is BGP
auto-configuration?
1c. If you're going to use some flavor of TLS (including DTLS) as we
discussed during the requirements phase as a possible mechanism, you need a
session.
1d. If you're going to use something that rides on top of TCP, you have all
of the TCP vulnerabilities that TCP-MD5 and TCP-AO are intended to mitigate.
Using those mechanisms to mitigate the mechanism means that you're already
solving the keying issue... and perhaps don't need TLS at all.

2. The hello is completely unprotected.  This means there's an edge case
man-in-the-middle condition.  Protecting it means you've already solved the
keying discussions and wouldn't need TLS.

3. TLS crypto infrastructure is operationally weighty if you're covering the
usual things:
- What certificate authority are you using?
- How is revocation handled, if at all?
- If you want to use certificates for authentication, you now have to get
  those distributed to all of your participating routers.
- Certificates have lifetimes - although potentially very long lived ones.
  You have to now worry not only about usual key rollover considerations
  that all of the likley mechanisms need, but also making sure you don't lose
  your auto-discovered sessions on a bounce because a certificate expired.

I'm loosely aware of the ACME work in IETF and perhaps the intent is that
these things are no longer quite as operationally cumbersome.  However,
that's an entire ecosystem that needs to be integrated into your router
operations if that's the case.

4. TLS provides integrity to the transport session and its contents, even if
it can't protect against attacks on TCP itself.  Privacy is optional,
although given the somewhat odd ULPC security text it seems that a scenario
under considertion is some form of distribution the pre-shared keys or
something similar.  However, the majority of the state for ULPC and l3nd
isn't exactly secret; it's stuff you'll see in ARP and ND if you're on link.

I don't think keychain information is considered something from most models
to need privacy. 

5. The proposal likely shouldn't call itself BGP-like.  The explicit ack
model minimally makes that not the case.  The state machine isn't terribly
comparable.

In particular, the Serial Number restart infrastructure seems to be very
heavy-weight for something running over a reliable transport like TCP.  But
this perception is tied to the idea that the BGP piece of this is small.
And even if there's a modestly large number of interfaces on the link, the
protocol should be able to send the entire bulk of them in a small number of
TCP frames.  (Compare vs. BGP send rates in even low end implementations.)
So, why complicate the mechanism with a highly stateful restart mechanism?
I suspect this makes more sense for the layer 2 proposal.

6. The sheer weight of this thing would make more sense if it's a more
general mechanism that is already deployed and the small BGP piece is along
for the ride.  I understand the motivation for a feature like this at layer
2, especially in the lsvr context.

I don't understand the use case for something like this in its current form
at layer 3.  What's that use case?

7. The crypto itself makes the autoconfiguration mechanism a point of attack
on the system rather than something to mitigate the attack.  The overhead to
deal with an on-link attack on TCP-MD5/-AO (symmetrical ciphers) is lower
than the overhead of TLS negotiation.  Rate limiting the creation of
incoming l3nd sessions might help things to some extent, but then you're
worrying similarly about TCP session exhaustion as well.

-- Jeff

[1] https://datatracker.ietf.org/doc/html/draft-ymbk-idr-l3nd-00
[2] https://www.ietf.org/rfcdiff?url2=draft-ymbk-idr-l3nd-ulpc-02.txt
[3] https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-autoconf-considerations-02

---

From: "Susan Hares" <[email-masked]>
Date: Sat, 5 Mar 2022 12:05:25 -0500

Jeff: 

Thank you for raising your concerns on the state machine. 

Please see the text in draft-ymbk-idr-l3nd-01: 

  The server, the sender of the HELLO with the lower IP address,
   listens on the advertised port for the TLS/TCP session open.  The
   receiver of the acceptable HELLO, the TLS/TCP client, initiates a TLS
   or raw TCP session with the sender of the HELLO, the TLS/TCP server,
   preferably TLS, as advertised.  If TLS, the server has chosen and
   signaled either a self-signed certificate or one configured from the
   operational CA trusted by both parties, as negotiated in the HELLO
   exchange.

   Once the TLS/TCP session is established, if the link is configured as
   point to point, the client side SHOULD stop listening on any port for
   which it has sent a HELLO.  The server side SHOULD stop sending
   HELLOs.

The L3DN - proposes BGP auto configuration that 
supports reliable transport with security. 

Open, Encapsulation, Ack - run over TCP (or TLS) 
that provides reliable ordered delivery. 

Open PDUs are required PDUs  that have: 
1) a nonce (like a message ID)  
"Enables detection of a duplicate OPEN PDU" (p. 11, paragraph 2). 
2) serial number - "representing the senders state at the time of 
Sending the last PDU" [p. 11 last paragraph) 

Encapsulation PDUs are optional  PDUs that have: 
1) serial number - "representing the senders state at the time of 
Sending the last PDU" (p. 15) 
2) Flag for Announce/Withdraw of encapsulations 

Encapsulation PDUs do not have a nonce. 

Ack PDUS - send by receiver of Open/Encapsulation PDUs
To indicate status (Ok, Warning, Broken session, Warn operator)
To acknowledge a specific type of PDU
(Open, Encapsulation PDUs (4-7 
ACKs PDUs.   Do not have a nonce or a serial number. 
 
The WG could suggest adding nonces to Encapsulation PDUS
so all PDUs have a message id, and adding the nonce value to 
ACK PDUs.   There are pros/cons of adding this state for debugging
versus bytes on wire + implementation states. 

We look forward to a discussion on this point. 

L3DN is not a "shout in the dark" bgp auto-configuration protocol. 
I define "shout in the dark" protocol as one which 
1) does not provide reliable transport
2) uses authentication/Hash based security 

Draft-minto-idr-bgp-autodiscovery-01 is a "shout in the dark" 
Protocol for bgp auto-configuration that relies on 
UDP multicast, timers, and periodic sending 
(see page 3, page 6 (authentication TLV). 

Sue 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Tuesday, March 1, 2022 4:22 PM
To: [email-masked]
Subject: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

Working Group,

[snip - to make it easier to respond to.  
My previous message answered the questions 
Regarding scope and transport] 

[Point 1 - Protocol state] 
Protocol state and FSM considerations:

[Sue] The solution used by IETF LDP or IEEE parallel sessions is 
to pre-define the resolution.  Please see the above text from 
draft-ymbk-idr-l3nd-01 that resolves this issues. 
[End point 1] 

[Point 2 - Open, duplicate Attributes] 
Open message:
- If Attributes are single-octet values of local significance, is there any
  significance to receiving the same Attribute twice?  If not, consider
  adding text about their uniqueness.
[Sue: Thank you for mentioning this fact.  We'll add text on this point. ]
[End point 2] 

[Point 3 - size of Attributes] 
  + Also, why so small?  Payload length appears to be 4 octets.  A larger
    value permits more flexibility for local semantics; e.g. bit vectors.
[Sue:  Were open to hearing pros/cons of 1 byte vs. 4 bytes
  from the IDR WG.   If we use fixed lengths (1 or 4) both are useful.] 
[end point 3]

[Point 4 - Resuming session] 
  + Is there significance to resuming a session using the documented
    procedures when the Attributes may not be identical to the ones
previously
    received in the interrupted session?
[Sue:  Yes.  
Nonce - provides detection of duplicate Open  
Serial number - indicates a single state of the Database
which contains a set of optional attributes + encapsulations. ]

Let's look at duplicate Open: 
Nonce is the same, and Serial number is the same. 

Let's look at Attribute change 
Nonce is different, Serial number is different 

Let's look at restart of system 
Nonce is different, Serial number is the same. 

Why care about what is in the DB: 
Example: 
Box1-Box2 (hello, TCP-setup)
Box 1 Open-1: With Attribute RR1 
(Indicates will accept 6 AFI/SAFIs in BGP)   
Nonce: 3030
Serial number: 1
Box2: Ack-Open: Error (1) - Error hint (not RR1 flag, RR2 flag) 

Box2: Open-1: Attribute: RR client 
Nonce: 7070 
Serial number: 50  
Box-1: Ack-Open 

Box-1 Open-2:  RR2 (accepts 5 AFI/SAFIs) 
Nonce: 6060 
Serial number: 2 
Box-2: Ack (0) no error 

Box-1 reboots 
Box1-Box2 (hello, TCP-setup)
Box1: Open-3, Attribute RR2 
Nonce:  100
Serial number: 2 
Box2: Ack-open (ok) 
[Point 4 end] 

For clarity, consider labeling Serial Number as "last received serial
  number".
[Sue:  It is being sent on an open - so this does not make sense. 

Old Text:  The Serial Number is a monotonically increasing 32-bit 
value representing the sender's state at the time of sending the 
last PDU. /
New text: 
The Serial Number is a monotonically increasing 32-bit 
value representing the sender's state at the time of sending the 
this Open PDU. /
[end Point-6]

[Point 7] 
Ack message:
- The nybble next to Etype needs some flavor the usual text to deal with its
  contents.  "Must be zero on transmit, should ignored on receipt"
[Sue: Thank you for catching this error. ] 
[End-Point 7] 

[Point-8]  
- The Etype conditions are a bit lax for a useful state machine for what we
  have documentation for: 
  + If Open can't be acked successfully, should the session be permitted to
    continue?
  + Is "restart is hopeless" a shutdown?  An admin-down?  What should the
    implementation do with its PDUs?  Should it tear down the session?

[Sue: Some of L3DN authors suggested deleting the ACK.  
The Open-Message contain attributes each side is willing to handle. 
The ACK simply gives remote side error indicates which may allow
automation of the fixes.  

My example for point 7 gives a positive example for the "warning" (1). 

An example for the restart, is a detection of misconfiguration
 [You are sending me attributes that I think are broken]  

An example for hopeless restart (admin-shut down) 
is if the Open contain multiple broken attributes.  

The minimum Ack PDU requirements are handle ACKed PDUs, 
Etypes, and Error codes.   The rest is extensible for each implementations. 

{End-Point 8] 

[Point-9] 
- 3 seconds seems a bit low for a response timeout.  While the protocol
  seems to be structured toward easy restart, why disconnect so
  aggressively?
[Sue] The timeout on OPEN receiving an ACK is configurable 
With 30 seconds as a timeout.      
[end-comment]

- The message exchange protocol is "one outstanding ack" which means there
  is no pipelining of the various messages.  Given this, the lack of a
  serial number being acked in the message is understandable, but seems like
  it'd be good debug info.
[Sue:  
The 4 byte Nonce identifies the open message. 
The 4 byte serial number identifies the DB state. 

Are you looking for the Nonce to be echoed in the ACK for the OPEN? 

  + That lack of pipelining means that full exchange of state will be gated
    on slowest sender on the session; the transmitter or the acknowledger..
[Sue]
TCP/TLS provides reliable transmission and ordering on the pipeline 
in each directions.  
OPEN has a nonce (message ID),  Encapsulations (ID 4-7) do not.  
Serial number gives the DB state with attributes + encapsulations. 

Are you suggesting: 
1) Augmentation of the Encapsulation header with nonce (message id) 
2) ACK PDU header contains the nonce of the open/Encapsulation header 
3) We allow (as some IEEE protocol do) the batching of 
ACKs in one TLV? 
 
All this is possible, but it is a trade-off that the authors would like
feedback on. 
[sue]

[Covered in next message] 

Encapsulations:

Encapsulation Flags:

Hello discussion:

[Covered in next message]

[Covered in another message] 

ULPC draft:

3.1:

I'm very confused here.

-----

Misc:
  surprising.  But it means that the sessions are vulnerable to reset
attacks

[Covered in another message] 

-----

  that all of the likley mechanisms need, but also making sure you don't
lose

-- Jeff

[1] https://datatracker.ietf.org/doc/html/draft-ymbk-idr-l3nd-00
[2] https://www.ietf.org/rfcdiff?url2=draft-ymbk-idr-l3nd-ulpc-02.txt
[3]
https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-autoconf-considerat
ions-02

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: "Susan Hares" <[email-masked]>
Date: Sat, 5 Mar 2022 17:10:40 -0500

Jeff: 

Thank you for your comments on the draft-ymbk-l3dn-ulpc-02.txt.  

I am not an author on the L3DN ULPC draft - so  I will provide only high
level comments. 

L3DN + ULPC

Topology: 

Link 1: 

Box-1 ----interface-1 -------box2

  192.2.1.1                   192.2.1.2   (IPv4)

  IPv6-addr1             IPv6-addr2 (IPv6)

Link2: 

Box-2 ----interface-2-------box2   

  192.3.1.1                  192.3.1.2   (IPv4) 

L3DN sets up TCP session across both interfaces. 

On link1, ULPC Information (v4, v6, AS number, Auth, Flags) 

    can be sent between v4 peers, and v6 peers. 

    The data passed is unique per peering session. 

On link 2,  1 peering 

The real key is the peering session. 

The Data passed via ULPC is unique per L3DN peering session. 

Suggestion for change: 

Old text/

 "If there are one or more open BGP sessions, receipt of a new BGP ULPC

   PDU does not affect these sessions and the PDU SHOULD be discarded.

   If a peer wishes to replace an open BGP session, they MUST first

   close the running BGP session and then send a new BGP ULPC PDU."

/

New text: /

  "If there are one or more open BGP sessions, receipt of a new BGP ULPC

   PDU does not affect these sessions.    If a peer wishes to replace an 

   open BGP session, they MUST first close the running BGP session

   and then send a BGP Open based on the data in the BGP ULPC PDU."

/

In section 3.1.2 and 3.1.3,  is the prefix-length allowing the 

BGP session to allow a sub-group to peer 

(e.g.  192.5.10/24) 

Section 3.1.4 - would be helped by providing a code point for key types
since

L3DN can be run over TLs. 

Section 3.1.5 removing "no flags are currently defined." Is useful. 

Miscellaneous comments:  See comments on security inline.  

I'm not sure what you suggesting as a change

Cheers, Sue 

===================

ULPC draft:

[Point 1] - ULPC 

3.1:

 

:   SHOULD BE signed.

I'm very confused here.

Are you suggesting you send the ciphertext with the intent that an
implementation can consult its own tables to find what input password for
that salt should be used?  If so, this is a flavor of pre-shared key.

It'd

-----

Misc:

[Sue: Agreed]

[Sue: Yes, possible - but Hello/ACK under TLS should secure that point. 

The comments on TCP-MD5 and TCP-AO seem orthogonal.  

TCP-MD5 and TCP-AO could we used as well as TLS. ] 

[Sue: yes - GTSM does apply to all packets in TCP session.] 

[Sue: Thank you for editorial comment] 

- The session is long-lived but not protected similarly to BGP; i.e.

tcp-md5  or tcp-ao.  Given that the intent of the mechanism is to setup BGP
which

[Sue:  TLS would be nice for BGP as well. ] 

-----

[Sue: Covered in final proposal] 

In particular, the Serial Number restart infrastructure seems to be very
heavy-weight for something running over a reliable transport like TCP.

But

 

-- Jeff

[3]

https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-autoconf-consider

at

ions-02

Idr mailing list

[email-masked]

Idr mailing list

[email-masked]

---

From: "Susan Hares" <[email-masked]>
Date: Sat, 5 Mar 2022 17:50:48 -0500

Jeff - 

this is the final response message on your feedback on draft-ymbk-idr-l3nd. 

We have two divergent views on BGP Requirements for auto-configuration at
L3. 

a) "Shout-in-the-dark" BGP auto-configuration protocol on the order of
"expanded ARP and ND" using MD5 security

b) Auto-configuration of BGP based on secure, reliable, and extensible
protocol that can be tuned to L3 

The landscape has three types of Massive/Large datacenters

a) DC that will never use BGP auto-configuration, except as a check on their
own configuration

b) DC with L2 infrastructure who wish to use L2 BGP auto-configuration is
best (LLDP/LLDPv2 or L3DL). 

c) DC with L3 infrastructure who wish to use L3 BGP auto-configuration
(L3DN/L3DN-ULPC, Minto) 

It is the operators (and their vendor suppliers) for Case C that the IDR WG
needs feedback from. 

It may be our feedback comes in the way of protocols implemented and
deployed.  

Thanks for your hard work on trying to get every protocol to a highest
quality possible prior to our WG discussion on adoption. 

Cheers,  

Sue 

================

Sue - I respond in line to each comment. 

1. The proposal is session based. 

While the prior discussion was primarily in the context of draft-xu,
sessions were seen as potentially problematic for a number of reasons.  They
also had some possibly useful properties.

[sue:  Yes, other drafts on the design team (e.g.
draft-xu-idr-neighbor-discovery)

had some good points about session based. 

1a. Session based makes large amounts of state easy to deal with.  

That said, the requirements show we need very little state for a given
session.

[Sue: The amount of state depends on the packing of encapsulations per link
and whether you allow "bgp neighbor groups" with subnets instead as well as
IP addresses. 

1b. These sessions are long-lived.  

You'll have one for each interface you want to run auto-discovery on. This
minimally doubles the number of active sockets your stack will need because
for each BGP session you get, you'll have a lingering discovery session.
Perhaps they don't need to be long-lived if the only thing you're using this
for is BGP auto-configuration?

[Sue:  The benefit of long-lived may be more for "bgp neighbor group" or a
period of rapid changes in BGP sessions are needed. 

We need feedback at adoption and WG LC about deployments.]

[Sue: agreed]

[TCP is proposed for "no security" and TLS for security.  One could aid
TCP-MD5 for authentication, but the security group has been discouraging
further use of TCP MD5]. 

[Sue: I do not agree with this viewpoint.  

I think the sequence Hello (multicast)/TCP/TLS (Open) response/ OPEN/ACK)

Substantially reduces the attack surface of the man-in-the-middle over the 

"shout-in-the-dark protocols". 

[Sue: The selection of the TLS certificates to self-certificates or 

getting it from an authority.   You could have two certificates (self +
authority)

and allow yourself to roll from authority to self if the certificate
expired.  

Seems easy enough.  The BGP sessions won't bounce, but the L3DN might
bounce. 

[Sue: ACME is making this easier for new router software to rotate. 

Perhaps we'll see this first in open-source code] 

[Sue: Here's where we may differ.  I think ARP and ND should be replaced by
SEND (secure ND + v4 variant of send). 

I think options for extensibility and security are required any new protocol
related to BGP.  I want to see that the IDR group at least has the option to
consider such protocols in our deliberations.  Some networks running BGP
auto-configuration will not need it, but others will.   

But this perception is tied to the idea that the BGP piece of this is small.
And even if there's a modestly large number of interfaces on the link, the
protocol should be able to send the entire bulk of them in a small number of
TCP frames.  

(Compare vs. BGP send rates in even low end implementations.) So, why
complicate the mechanism with a highly stateful restart mechanism?

[Sue: I think that the minimal set for BGP auto-configuration expressed in
the DT,  is not all the data. 

I think allowing extensibility + security is important.] 

[Sue: Sheer weight is a value judgment based on assumptions. 

Your assumptions are clear:  "Just ARP + ND for BGP without security". 

My assumptions are: secure + extensible with configuration knobs to reduce
timer overhead.

at layer 3.  

What's that use case?

[Sue: Massive data center, but I will let someone working with that
deployment argue.]

[Sue]: I disagree with this viewpoint based what I understand from security
comments. 

If this is the major point, then we should ask Security-directorate for a
comparison. 

-- Jeff

[3]

at

ions-02

Idr mailing list

[email-masked]

Idr mailing list

[email-masked]

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 7 Mar 2022 20:56:12 -0500

[General note: While I'd prefer to have a response for the original single
message in a single response, I'll be responding to the individual replies.
This should help those with threaded mail readers track the responses better.]

Sue,

> Please see the text in draft-ymbk-idr-l3nd-01: 

Similarly, please note that my original post was vs. -00 of the draft.  Have
care at trying to move the goal posts mid-conversation. :-)

>   The server, the sender of the HELLO with the lower IP address,

New procedure since -00.

>    listens on the advertised port for the TLS/TCP session open.

While it's possible that the receiver only creates its TCP listen socket after
hearing a HELLO from another sender, this isn't typical listen socket
coding.  It's far more typical for a TCP listener to have a socket open and
ready to receive incoming sessions.

If the receiver of a HELLO that is also sending a HELLO determines it is the
lower of the IP addresses and instead chooses to dynamically create the
listening socket at that time, it would have the properties of:
1. The listening socket is available less of the time, which may be mildly
protective of attacks against the TCP stack.
2. However, introduces timing considerations before the initiator of the TCP
connection would be able to connect.

If your intent is to have deferred listen socket creation as normative
procedure, you will want to have related normative text for the connector to
try to reduce connection time.  Otherwise, you're looking at slowing down
discovery.

>    which it has sent a HELLO.  The server side SHOULD stop sending

L3DN lacks a state machine with named states similar to BGP, so my comment
here will be somewhat imprecise.

Your text above says HELLO is stopped when TLS or TCP is established.  Those
are slightly different events.  TLS requires TCP, so you could stop HELLO
there.  TLS will go through its Handshake Protocol exchange. (RFC 4346, §7.3)
Was that where HELLO should stop when TLS is enabled?

If HELLO stops at the first TCP connection accepted, it permits an attacker
to stifle HELLO messages from legitimate parties.

Similarly, if only a single incoming TCP connection is permitted until TLS
session is up (handshake has completed), it permits a similar denial of service.

Perhaps related, HELLO perhaps should not stop until L3ND mutually exchanges
and positively ACKs their OPEN messages.  That might make a better sync
point to stop HELLO, although it still means addressing the TCP session
questions.

In BGP parlance, the mutual exchange of OPEN and positive ACK would be the
Established state.  

A few extra notes:
- L3ND doesn't have any normative references to TLS.
- The Security Considerations section recommends at least TLS 1.1
- RFC 8996 deprecates TLS 1.0 and 1.1.  If there's compelling reason to
  support TLS 1.1, it might be worth mentioning why.  It's not as if BGP
  hasn't been using the deprecated TCP-MD5 for years. :-)

> "Enables detection of a duplicate OPEN PDU" (p. 11, paragraph 2). 

I think it's more precise to call the nonce a session ID.  One of its
purposes in the procedure is to permit a pre-existing session to be resumed.

I remain unclear why there would be "duplicate" OPEN PDUs on the same TCP
session.  TCP is a reliable protocol.  Is the expectation that the receiver
is taking a reliable protocol and randomly discarding messages?

> versus bytes on wire + implementation states. 

As a mechanism to identify a restarting session, the nonce is a fine session
ID.  Personally, my question is more about whether the necessary state
retention and refresh mechanisms are worth the additional complexity in the
protocol.  I'd personally recommend that a session restart simply replay all
necessary state fresh.  

However, these are personal preferences.  Correctness of the mechanism is
more important for this discussion.

> 2) uses authentication/Hash based security 

In fairness, I labeled such protocols as shout in the dark but that's a
reasonable summary of it. :-)

[...]
> draft-ymbk-idr-l3nd-01 that resolves this issues. 

As I briefly noted to the publication notice for -02 of this draft, this
changes a property of the protocol: It requires both sides to run the HELLO
mechanism.  A consequence of this is that it's no longer possible for a
single side to act as a HELLO sender - likely in a "server" role.

For point to point deployments, this isn't a big deal.

For multi-access deployments that are intended to result in some form of
full or partial mesh, this still isn't a big deal.

For multi-access deployments that are intended to have a small number of
devices acting as the end point for a BGP session, this is problematic.
You'll have attempts at full mesh peering when perhaps a tiny number of
devices want to peer with each other.

[eliding points that are agreed upon...]

>   from the IDR WG.   If we use fixed lengths (1 or 4) both are useful.] 

My personal opinion is that fixed length is fine for the application.  The
Working Group will likely have opinions about the size.

> Nonce is the same, and Serial number is the same. 

For clarity, I think there's some confusion in using the term "duplicate" in
this context.  It's not a duplicate on the same established session.
Instead, it's a Open that has a nonce that matches a previous transport
session.

If that's the intent, I'd suggest either different verbiage or make sure and
define the context for duplicate.

> Nonce is different, Serial number is different 

That's not the scenario under consideration.  If the nonce changes, it's a
restart of the session.

OPEN messages in the loosely defined state machine are intended to be sent
at the start of the transport session once data can be exchanged.

Serial numbers are loosely specified as advancing per packet.

If no additional OPEN message is sent, how would the Serial Number advance
for a change in Attributes?

I think your intent is likely that if the Attributes do not agree for the
same session (Nonce) that Something Must Happen.  Is it a session bounce?
I'm not sure what your intent is.

> Nonce is different, Serial number is the same. 

If Nonce is different, the text says that it is not a session restart.

> Box-1 Open-2:  RR2 (accepts 5 AFI/SAFIs) 

Why is Box-1 sending OPEN again?

> Box1: Open-3, Attribute RR2 

I think the above example may be a bit off.  Perhaps another try at it?

The intent from the specification seems to be that when a session comes up,
and the data matches a previously exchanged Nonce, then you can use matching
serial numbers to do selective replay.  That's fine.

OPEN messages are not, as best I can tell, exchanged again once the OPEN
messages are mutually exchanged and ACKed.  

> value representing the sender's state at the time of sending the 

I'm not sure that's correct.

In the context of OPEN, you're informing the other side what the last state
you heard from them was.  That state is updated on your receipt of messages
with Serial Numbers.

> The ACK simply gives remote side error indicates which may allow

It's your protocol.  I agree that you can live without ACK in many
circumstances.  You will likely need it as a mechanism to make sure you
mutually exchange OPEN and ACK prior to starting to send other PDUs.

It'd also maintain some level of similarity to the BGP FSM.  If so, consider
borrowing the state names.

While the OPEN message lacks any semblance of the BGP Capability mechanism,
it's pretty easy to visualize that some dependency on behavior based on
received Attributes has similar properties.

> is if the Open contain multiple broken attributes.  

If the intent of restart is hopeless is that things are locked into an
AdminDown type state, there should be normative text as to what that does to
the state machine.  For example, HELLO might not resume once the TCP session
is dropped.

> With 30 seconds as a timeout.      

This was in reference to section 10.1.  The value has been updated to five
seconds... which still seems like a short default. :-)

The retransmit behavior again seems peculiar for a reliable TCP session.
> Are you looking for the Nonce to be echoed in the ACK for the OPEN? 

I'm commenting that the ACK PDU (Section 9) does not contain the Serial
Number that it is acknowledging.

> All this is possible, but it is a trade-off that the authors would like

I'm suggesting that the protocol is "call and response".  You send a
message, you want an ACK for it.  This means that in spite of the fact that
this is TCP and there are buffered windows to transmit into in a reliable
fashion, no PDUs are placed into the TCP stream until the prior PDU receives
an ACK PDU.  This is a one-bit flow-control mechanism.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 7 Mar 2022 21:36:07 -0500

Sue,

>    and then send a BGP Open based on the data in the BGP ULPC PDU."

I still believe you'll introduce timing issues with this ordering.

Peer 1 is connected to Peer 2.
Peer 2 wishes to changes its parameters advertised in ULPC.

If Peer 2 unilaterally closes its peering session prior to sending a message
in ULPC, Peer 1 may try to reconnect to Peer 2.  Peer 2 may reject that
session.  Interleaved within these events, Peer 2 will advertise new ULPC to
Peer 1.  While TCP is reliable, it doesn't provide bounded timing on
transmission of state.  It may happen near instantly.  It may also in the
face of packet loss take a while to get there.

Meanwhile, peer 1 has a BGP state machine that likely has a form of the
backoff (DampPeerOscillations, RFC 4271 §8.2.2).  We don't want to encourage
slow auto configuration, I think.

> BGP session to allow a sub-group to peer 

Help me understand such peering.  In order for auto-discovery to work, BGP
must have enough information to start a TCP session.  Is the implication
that any address within the subnet is fine for setting as the destination
address for your BGP session?

> L3DN can be run over TLs. 

Irrelevant.

The authentication data needs to mean something and context matters.

Is the information in the BGP Authentication Data field a TCP-MD5 shared
key?  Is it a TCP-AO key?  If it's TCP-AO, where's the send-id/receive-id
fields?  Is it the identity of an IPsec SPI?

Or, leaving aside sending session keys directly, is it the identifier for a
security keychain?  (See, for example, RFC 8177.)

See for example BFD's Authentication Section (RFC 5880) or your favorite IGP.

> TCP-MD5 and TCP-AO could we used as well as TLS. ] 

The terminology you might be looking for here is that the unprotected HELLO
packet leads to the possibility of a "downgrade attack".  It's why STARTTLS
mechanisms stopped being used.

The short form of the attack for L3ND is that if a session is permitted for
both raw TCP and also TLS, you suppress the HELLO and replay one only
carrying raw TCP.

Minimally, this should be called out in the Security Considerations.

The relevant point about TCP-MD5 and TCP-AO is that L3ND is vulnerable to
TCP reset attacks.  If you're looking for justification for comment,
consider using RFC 4272.

> [Sue: yes - GTSM does apply to all packets in TCP session.] 

Then the text shouldn't mention it solely in a SYN context.

> [Sue:  TLS would be nice for BGP as well. ] 

That's an opinion.

I've worked on applications with highly long-lived TLS sessions before.
Here's a few things that make living in TLS ugly for such things:

- If certificates are in use, they must be maintained.  While in some
  circumstances they may be treated as a feature, it also means that in the
  face of operational neglect they can expire you now have a certificate
  renewal fire drill.  Pick your favorite well known website outage from such
  issues as an example.  Now imagine it impacting your routing infrastructure
  that very likely requires OUT OF BAND provisioning a requirement for repair.
  + See also the recent large Facebook outage and how out of band is tricky
    to do.
- TCP has its own issues with hiccups in framing.  TLS introduces internal
  framing mechanisms.  The two play together in unpleasant ways in the face
  network issues to mess with protocol timing.
  + For L3ND, your timers are already short with 5 seconds.
  + If you though BGP KEEPALIVE marshaling was messy in a scaled
    implementation, now try to do it weaving through TLS framing.
- Popular TLS implementations aren't used to being used for weeks to months
  at a time.  The stacks aren't as clean as one might want for something
  like BGP.
  + L3ND plans on restarting, so this isn't horrible.
  + People don't like using Graceful Restart in many BGP deployments.

I suggest you poll for other developers who have lived in long-lived TLS
sesssion deployments.  They'll perhaps have other opinions.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 7 Mar 2022 23:20:51 -0500

Sue,

> "expanded ARP and ND" using MD5 security

I wouldn't make the analogy to ARP/ND.  Better comparisons would be to other
hello mechanisms such as PIM, etc.

MD5 is also a distraction.  Make it SHA-512 if it makes you happy.

> protocol that can be tuned to L3 

I believe "secure" is also an attempt at distraction.  FWIW, I'd invite Russ
to actually respond to these parts of the thread.

The key differentiators for our discussion are, in my opinion, the
following:
- Is the solution transport session based or not?
- What is the amount of state intended to be carried in the protocol?
- What security properties do you want on the contents of the protocol?

The security properties even more so than the rest of the properties have
strong impact on the likely solution.

Does BGP autoconfiguration need privacy?
- If you're passing around opaque keychain information, probably not.
- If you're passing around BGP session keys, absolutely.
- And remember that no session security is a common option already for BGP
  autconfiguration solutions.

Does BGP autoconfiguration need authentication?
- Optionally.  Some may not care about it and are happy to rely on BGP
  session security.
- Is authentication done by validation of a shared secret?  We have
  multiple IETF protocols that implement their authentication in this
  fashion.  Cryptographic strength is selectable.
- Is authentication done by certificate?  (D)TLS will immediately push you
  to needing a session layer and a deeper protocol.
  + If you're doing certificates, do you care about revocation? Certificate
    authorities?
  + What about expiration?

How much state are we pushing around?
- If it's an IPv4 and an IPv6 address per link, that's not a lot of BGP
  state.
- If it's dozens, this is a very different story.

Transport session?
- L3DL has its own transport session.
- Using TCP means you inherit the security considerations of TCP.

> It is the operators (and their vendor suppliers) for Case C that the IDR WG

The poll questions, per our discussion, are really B and C.

Which layers does the working group want to standardize BGP
autoconfiguration for?  L2?  L3?  L2 and L3 both?

For a given layer, which protocol or protocols is the proper fit?

We could end up with one to four solutions given the current set of
proposals on the table.  

We're simply trying to set the context for the choices.

> period of rapid changes in BGP sessions are needed. 

Your response is unclear.  Example, please?

> TCP-MD5 for authentication, but the security group has been discouraging

TLS requires TCP.  You get both sets of considerations.

> [Sue: I do not agree with this viewpoint.  

The downgrade attack was described in another message in the reply chain.

> "shout-in-the-dark protocols". 

Actually, it opens more attack space.  This was mentioned in other replies.

For hello style protocols, there's nothing to attack in that protocol.
Your options are to attack availability and find ways to keep the messages
from getting through.  The component sending the hello doesn't care, in the
absence of a session protocol, that you listen or try talking to it.

The minute you introduce a session protocol, you inherit the security
considerations of all session protocols.  This includes endpoint exhaustion,
ineterference with transport, man-in-the-middle, etc.

The benefits may outweigh the security considerations, but it doesn't
eliminate them.

> and allow yourself to roll from authority to self if the certificate

You're introducing speculation as to how the certificate infrastructure
should be operated.

A specification that's going to mandate the use of TLS as one of its options
needs to be incredibly clear as to how TLS inputs, including
certificates, are operated.

I suspect Russ can supply you with appropriate text.

> Perhaps we'll see this first in open-source code] 

Note that even ACME runs into issues flagged by KARP: When the routers
themselves are the components you're trying to manage, you have some very
challenging dependency issues.  If a certificate expires, how do you intend
to get it onto a device that is there to perhaps provide reachability to the
service that lets you manage them?

> SEND (secure ND + v4 variant of send). 

I think this is an orthogonal point.

> auto-configuration will not need it, but others will.   

Our Security ADs would be thrilled to see routing groups do better at
security.

Our Security ADs also share the common blindness of security professionals.
In routing, performance and operational considerations may matter more than
some security properties.

It's an interesting dialog and one I've had to engage in multiple times,
especially from a BFD standpoint.

> I think allowing extensibility + security is important.] 

One of your coauthors fought quite hard to skip extensibility as a property.

> [Sue: Sheer weight is a value judgment based on assumptions. 

That's a fair statement.

That said, I'm happy to count total pages of RFC needed to fully implement
our solutions. :-)

> Your assumptions are clear:  "Just ARP + ND for BGP without security". 

While our personal conversations have discussed the fact that there are
implementations that leverage ARP or ND for auto configuration, I don't
think you'll find that I've ever offered this as a discussion point here.

Please substantiate the above comment.

> My assumptions are: secure + extensible with configuration knobs to reduce

And here we see a related value based judgment.  TLS is a kind of security,
it's not a definition of security.

I'll be happy to see the question of extensibility re-opened since it was so
hard fought against by one of your coauthors.

> [Sue]: I disagree with this viewpoint based what I understand from security

For this point, Sue, I request you put Russ on the phone. :-)

If you're going to hang your hat on security, that's where we should spend
our time.

> If this is the major point, then we should ask Security-directorate for a

Excellent!  Analysis of cryptographic strength and cost for modern ciphers
isn't something the cryptography books on my shelves cover.  Call them up.

-- Jeff

---

From: "Susan Hares" <[email-masked]>
Date: Sat, 19 Mar 2022 08:03:39 -0400

Jeff:

Thank-you for letting me know you prefer the response in one long email. 
I'll respond to each of your comments below. 

Just to recall the high points: 

Point -1: 
  two types of protocols 
a) unreliable transport   
          multicast + Key-chain + timers  for every message 

b) reliable transport + optional security 
        transport (TCP) + key-chain 
     or  TLS + Self-Signed/CA-based 

The question is whether the timers cost you more than the 
transport.   

Point-2: 
Open Attributes are flexible hints of who is talking. 
Session-id != nonce 
Serial-number - tracks a set of attributes + encapsulations

Point-3: 
Encapsulations may be important to some
and irrelevant to others. 

Sue 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Monday, March 7, 2022 8:56 PM
To: Susan Hares
Cc: [email-masked]
Subject: Re: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

[General note: While I'd prefer to have a response for the original single
message in a single response, I'll be responding to the individual replies.
This should help those with threaded mail readers track the responses
better.]

Sue,

>ready to receive incoming sessions.

[Sue:  Yes, -01.txt correct errors or things that were missing]
[Sue]: However, when you have the Hello sequence, this can be restricted. 

[Jeff - Point 2] 

>try to reduce connection time.  Otherwise, you're looking at slowing down

[Sue]: The configuration time is  configurable. 
Your description is system dependent - so I suspect the people 
creating a particular router will tune it.  

[Jeff- Point-3]: 

Your text above says HELLO is stopped when TLS or TCP is established.  Those
are slightly different events.  TLS requires TCP, so you could stop HELLO
there.  TLS will go through its Handshake Protocol exchange. (RFC 4346,
§7.3)
Was that where HELLO should stop when TLS is enabled?

Similarly, if only a single incoming TCP connection is permitted until TLS
session is up (handshake has completed), it permits a similar denial of
service.

[Sue - Point-3]:  A particular implementation will configure TLS or TCP per
port. 
At that point, if TLS is configured then it will keep the Hello going until 
the TLS is up and running.    Adding a state machine to the L3DN, is 
a good idea.   

[Jeff - Point-4:] 

>In BGP parlance, the mutual exchange of OPEN and positive ACK would be the

[Sue: Point-4]:  The cost in keeping the Hello going until L3ND 
mutually exchanges is more messages on the wire. 
The trade-off at the TCP/TLS level was my choice. 
The WG can make a trade-off later. 
 
[Jeff: Point-5]
A few extra notes:

[Sue-Pont-5] 
Thank  you for mentioning this.  It should have been 1.2 or higher. 
A normative reference to TLS would be useful to add. 

[Jeff-Point-6] 

[sue-6]
A nonce is not a session-id.  It is a nonce. 
A nonce prevents "man-in-the-middle" attacks for the TCP 
variant of the protocol or some bug in the system. 
It is less likely in the TLS version. 

The purpose is: "It is needed to prevent a session closure due to a 
Repeated Open caused by a race or a dropped or delayed ACK" (-p. 11).  

[Jeff-Point-7]

 

>> protocol.  I'd personally recommend that a session restart simply replay
all

[Sue-7]: 
The encapsulation allow the L3DN players to exchange data across multiple
subnets. 
(e.g. page 16).  
I will cover this point in a unique post to the list. 
 
[Jeff-8] 

[...]
> draft-ymbk-idr-l3nd-01 that resolves this issues. 

[Jeff:  Point-8]

[Sue-8:]
  I am not clear why you state these three points above. 
  The text states:
"The Hello PDU solicits a unicast TLS/TCP session open request 
Of the same AFI from other devices on the link." (p. 9, -02) 

Only if there are two directions for hello, does the case in the  
Next step work.  "When a hello is received from a source
IP address with which there is no establish TLS/TCP L3ND, 
If the receiver has the higher of the two IP addresses, it should respond. "

I do not see where you are getting any of the rest of the text. 
Please provide examples and text from the -02.txt. 

[Jeff-point-9 for this email] 

My personal opinion is that fixed length is fine for the application.  The
Working Group will likely have opinions about the size.
 [Sue: agreed that size on attributes is ok]

[Jeff: point 9 this email] 

For clarity, I think there's some confusion in using the term "duplicate" in
this context.  It's not a duplicate on the same established session.
Instead, it's [an] Open that has a nonce that matches a previous transport
session.

[Sue- Point 9: ]
Jeff - A nonce is just a "nonce".  It is not a session id.  
It is a common term in many protocols to prevent a "man-in-middle" 
attack.  It is not a session-id.  

[Jeff- Point-10]
(long sequence on my example) 

>>That's not the scenario under consideration.  If the nonce changes, it's a

[Sue-10a] If that's the intent, I'd suggest either different verbiage or
make sure and
define the context for duplicate.

>at the start of the transport session once data can be exchanged.
[Sue-10b]
The open is sent with a "hello".
Nonce = identifier for duplicate Ack (man in middle attack)
Serial number = data base. 

>for a change in Attributes?

[Sue-10c]
It could be that a "policy" in the 
L3DN has logic that says - try another set of attributes. 

>I'm not sure what your intent is.

[Sue-10d] 
The idea is that if you keep lots of encapsulations. 
Or have attributes to catch something plugged in wrong. 
Then, the attribute or database is correct. 

>Why is Box-1 sending OPEN again?
[Sue-10e]
Different attribute list.  (Just as stated)

>> I think the above example may be a bit off.  Perhaps another try at it?

[Sue-10f] 
The example is not off.   
No, you are missing the point of the attributes. 

[Jeff-point-11] 
OPEN messages are not, as best I can tell, exchanged again once the OPEN
messages are mutually exchanged and ACKed.  
[Sue: 11a]
The machine running L3DN can go down and come up. 

> For clarity, consider labeling Serial Number as "last received serial
The serial number is a database-id. 

>> value representing the sender's state at the time of sending the 
[Sue-11b]: 
In the context of OPEN, you're informing the other side what the last state
you heard from them was.  However, you are correct, the original text was
correct. 

Text -02:  "The Serial Number is a monotonically increasing four octet value
Representing the sender's state at the time of sending the last PDU. " 

[Jeff- point-11 (old point-8) 

> Etypes, and Error codes.   The rest is extensible for each
implementations. 
End[New-point-11 /Old Point 8]

Sue: A state-machine is useful in the iteration after adoption. 

[Jeff New-Point-11] 

[Sue: (New:11/old: 9) A Serial number could be added to the ACK, but there
Is a trade-off here]

[Jeff-12]  

Sue-12:  TCP guarantees the right number of bytes. 
TCP does not guarantee the syntax of those bytes to be correct. 
The Ack is replying to the syntax of the message. 

-- Jeff

---

From: Randy Bush <[email-masked]>
Date: Sat, 19 Mar 2022 12:11:58 -0700

>      or  TLS + Self-Signed/CA-based

in a datacenter on the scale of O(10^5) devices, even a small failure
rate in stochastic protocols is not what one wants

> The question is whether the timers cost you more than the 

timers tune the probability

> Serial-number - tracks a set of attributes + encapsulations

in l3nd/l3dl,
  session-id == nonce
  serial number is kinda like a checkpoint from which you might resume

> and irrelevant to others. 

in l3nd/l3dl, perhaps 'encapsulation' is a misleading word.  think
collection of end point identifiers of the same afi/safi.  nothing is
actually being encapsulated.

randy

---

From: Jeffrey Haas <[email-masked]>
Date: Sat, 19 Mar 2022 15:16:45 -0400

Sue,

I won't have the energy to wrestle over disputes on the clarity of the state
machinery for l3nd until after IETF - and perhaps only after adoption calls
are done.  I believe I've made sufficient points for other WG participants
to start forming their own opinions.

That said, I did want to draw out one point you seem to find important:

> It is less likely in the TLS version. 

I'm quite clear what the cryptographic properties of nonces are intended to
be.  As currently present in the l3nd specification on a per-PDU basis, it's
not correctly being used - even this was a datagram protocol and not a
stream based protocol.

Also, stream based protocols are not appropriate places to try to leverage
per-PDU based nonces as protection vs. man-in-the-middle. There are no
framing guarantees that your PDUs will align with TCP frames for such
protection.  This is the reason you require either protection of the stream,
as TLS can do, or leverage transport protection mechanisms such as
TCP-AO/MD5.

I'd strongly suggest that if you feel nonces are relevant here that you
start a conversation with saag and tsv-art.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sat, 19 Mar 2022 15:22:22 -0400

Randy,

> rate in stochastic protocols is not what one wants

Were you intending your proposal to serve as auto-discovery for this number
of devices on a single link?

>   serial number is kinda like a checkpoint from which you might resume

Thank you for confirming that detail.

> actually being encapsulated.

I believe there was some level of similar tension over the terminology when
OSPFv3 was being worked on.  If we were talking about a single afi (I don't
think safi ends up being relevant here), we would have previously just
called this a "subnet" or "network".

A better term would help with document clarity, but I have no better
suggestions than subnet.

-- Jeff

---

From: Randy Bush <[email-masked]>
Date: Sat, 19 Mar 2022 12:28:31 -0700

> Were you intending your proposal to serve as auto-discovery for this
> number of devices on a single link?

no; but it probably could.  not that i would recommend it right off the
bat

> I believe there was some level of similar tension over the terminology
> when OSPFv3 was being worked on.  If we were talking about a single
> afi (I don't think safi ends up being relevant here), we would have
> previously just called this a "subnet" or "network".
> 

all the addresses in one encapsulation pdu need not be within a single
subnet; think loopbacks and over/underlay.

randy

---

From: Randy Bush <[email-masked]>
Date: Sat, 19 Mar 2022 12:38:00 -0700

> I'm quite clear what the cryptographic properties of nonces are
> intended to be.  As currently present in the l3nd specification on a
> per-PDU basis, it's not correctly being used

as i am weak on all these things, perhaps you could explain a bit
beyond "it is incorrect."

randy

---

From: Jeffrey Haas <[email-masked]>
Date: Sat, 19 Mar 2022 18:05:59 -0400

> beyond "it is incorrect."

I'd hoped my copy of Applied Cryptography would have a good citation for it,
but oddly it's not in the easy Index for citation.  So, here's one from
various NIST sources:

"A time-varying value that has, at most, an acceptably small chance of
repeating. For example, a nonce is a random value that is generated anew for
each use, a timestamp, a sequence number, or some combination of these." [1]

The Nonce field in L3ND (version -03) includes the following:
:    Should the TLS/TCP session terminate for any reason, the devices
:    SHOULD restart/resume HELLOs.  When the new TLS/TCP session is
:    started, if possible the OPEN PDU SHOULD try to resume the lost
:    logical session by using the same nonce and resuming from the last
:    Serial Number.

As you noted in your reply, this is functionally a session ID and the
text in the document's procedures uses it in that fashion.

For this to be a nonce, this wouldn't be the same between two sessions.

Additionally, if the intent was that PDUs were intended to be protected
versus replay, the nonce value would be part of the PDUs.  However, while
that can be helpful in cryptographic procedures for datagram protocols,
naked TCP doesn't have any framing gurantees for PDUs in the stream vs. its
own PDUs.  This makes naked TCP susceptible to man in the middle attacks
where the TCP sequence numbering is either respected and the underlying data
(if it includes no integrity mechanisms) subject to manipulation.  If it's
an active MitM, you can start playing with the sequence numbering actively.

TLS provides integrity to the stream and mitigates this part of an attack.

TCP transport protections such as TCP-AO/TCP-MD5 provide for integrity as
well.

TLS without TCP security won't help with attacks vs. TCP itself, such as
windowing attacks.  But that doesn't seem to be what you're securing.

My recommendation is to rename the nonce field to session ID.

-- Jeff

[1] https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-133r2.pdf

---

From: Randy Bush <[email-masked]>
Date: Sat, 19 Mar 2022 15:10:50 -0700

what's in my emacs buffer for the moment is

    The four octet Nonce identifies a session.  It enables detection of
    a duplicate OPEN PDU.  It SHOULD be either a random number or a high
    resolution timestamp.  It is needed to prevent session closure due
    to a repeated OPEN caused by a race or a dropped or delayed ACK.  It
    can be used to resume a dropped logical session.

> My recommendation is to rename the nonce field to session ID.

not a bad suggestion, though more editing than sounds fun on a weekend.

randy

---

From: Minto Jeyananth <[email-masked]>
Date: Wed, 23 Mar 2022 23:57:32 +0000

Susan,

 Few questions on point 1-b).

  1.  is there any mechanism to avoid UDP hello in multi-access link? The cost of the timer here is not same as draft-minto tighter periodic timer?
  2.  Both BGP and LDP running TCP keepalive apart from TCP keep-alive. Does l3nd add that too?
  3.  l3nd-ulpc exchanges BGP authentication information. does l3nd TCP keychain is configured or exchanged through UPD hello?
  4.  rfc6952 2.4.1.1 describes the attack on UDP multicast. l3nd will add keychain based authentication?
  5.  In multi-access link, l3nd required full mesh of session. Right?

Thanks
-minto

From: Idr <[email-masked]> on behalf of Susan Hares <[email-masked]>
Date: Saturday, March 19, 2022 at 5:04 AM
To: 'Jeffrey Haas' <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd
[External Email. Be cautious of content]

Jeff:

Point -1:
  two types of protocols
a) unreliable transport

Point-2:

Point-3:

Sue

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Monday, March 7, 2022 8:56 PM
To: Susan Hares
Cc: [email-masked]
Subject: Re: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

Sue,

[Jeff - Point 2]

[Jeff- Point-3]:

[Jeff - Point-4:]

[Jeff: Point-5]
A few extra notes:

[Sue-Pont-5]

[Jeff-Point-6]

[sue-6]

[Jeff-Point-7]

[Sue-7]:

[Jeff-8]

[...]

[Jeff:  Point-8]

[Sue-8:]

[Sue- Point 9: ]

[Jeff- Point-10]

[Sue-10b]

[Sue-10c]

[Sue-10d]

[Sue-10e]

[Sue-10f]
The example is not off.

[Jeff-point-11]

[Sue-11b]:

[Jeff New-Point-11]

[Jeff-12]

-- Jeff

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$>

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$>

Juniper Business Use Only

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 25 Mar 2022 04:20:48 -0400

Minto: 

Thank you for the excellent questions.   The response will be delayed until
next week. 

My IDR chair duties and other “non-IETF” duties have impacted my time to
give this the time it deserves. 

My co-chairs and I will respond to you next week. 

Cheers,  Sue 

From: Idr [mailto:[email-masked]] On Behalf Of Minto Jeyananth
Sent: Wednesday, March 23, 2022 7:58 PM
To: Susan Hares; 'Jeffrey Haas'
Cc: [email-masked]
Subject: Re: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

Susan,

1.	is there any mechanism to avoid UDP hello in multi-access link? The
cost of the timer here is not same as draft-minto tighter periodic timer? 
2.	Both BGP and LDP running TCP keepalive apart from TCP keep-alive.
Does l3nd add that too? 
3.	l3nd-ulpc exchanges BGP authentication information. does l3nd TCP
keychain is configured or exchanged through UPD hello? 
4.	rfc6952 2.4.1.1 describes the attack on UDP multicast. l3nd will add
keychain based authentication? 
5.	In multi-access link, l3nd required full mesh of session. Right? 

Thanks 

-minto 

Jeff:

Point -1:
  two types of protocols
a) unreliable transport

Point-2:

Point-3:

Sue

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Monday, March 7, 2022 8:56 PM
To: Susan Hares
Cc: [email-masked]
Subject: Re: [Idr] BGP autoconfiguration - draft-ymbk-idr-l3nd

Sue,

[Jeff - Point 2]

[Jeff- Point-3]:

[Jeff - Point-4:]

[Jeff: Point-5]
A few extra notes:

[Sue-Pont-5]

[Jeff-Point-6]

[sue-6]

[Jeff-Point-7]

[Sue-7]:

[Jeff-8]

[...]

[Jeff:  Point-8]

[Sue-8:]

[Sue- Point 9: ]

[Jeff- Point-10]

[Sue-10b]

[Sue-10c]

[Sue-10d]

[Sue-10e]

[Sue-10f]
The example is not off.

[Jeff-point-11]

[Sue-11b]:

[Jeff New-Point-11]

[Jeff-12]

-- Jeff

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NE
t6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$
<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NE
t6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$> 

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NE
t6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$
<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NE
t6yMaO-gk!TmhAR2N3V-_dp8Rgj1nEy5nCfQLccjMP5gTeFjwcGFy01QA7YkxJT4Rqmse0rw$>

---

From: tom petch <[email-masked]>
Date: Fri, 25 Mar 2022 09:39:19 +0000

From: Idr <[email-masked]> on behalf of Jeffrey Haas <[email-masked]>
Sent: 19 March 2022 22:05

<tp>
Looking at rats-yang-tpm-charra, which is IESG-approved, I see
          "A cryptographically generated random number which should
            not be predictable prior to its issuance from a random
            number generation function. The random number MUST be
            derived from an entropy source external to the Attester.
which I like for its stress on cryptography

Tom Petch

-- Jeff