Subject: IDR Agenda Items for Vienna

From: Yakov Rekhter <[email-masked]>
Date: Tue, 27 May 2003 09:10:43 -0700

Folks,
 
Its about time to start thinking about agenda items for the Vienna
IETF. Please forward any IDR agenda items you might have to me and Sue.

Sue & Yakov.

To: [email-masked]
Cc: [email-masked]
Subject: IDR Agenda Items for Vienna 
Date: Tue, 27 May 2003 09:10:43 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Folks,
 
Its about time to start thinking about agenda items for the Vienna
IETF. Please forward any IDR agenda items you might have to me and Sue.

Sue & Yakov.

To: Alex Zinin <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: AD-review comments on draft-ietf-idr-bgp4-20 
Date: Tue, 27 May 2003 08:16:29 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Alex,

> Folks,
> 
>  Please find below my AD-review comments. Hopefully they will help
>  improve the document. I tried to consult Andrew's list as much as
>  possible, but do feel free to point out if something has already been
>  discussed and agreed upon.
>  
>  Thanks go to Yakov for kicking me often enough ;)
> --
> Alex Zinin
> 
> Some nits:
> - run it by a spelling checker, please
> - disable hyphenation if possible
> - include boilerplates for IPR notice, Copyright notice

Sure.

> 
> General comment:
> 
>   in some places I highlighted the fact that required behavior is not
>   described using the 2119 language, so it is not clear if a MUST or
>   SHOULD or MAY is applicable. I am sure I've missed some more places
>   like this. I'd like to ask the editors to go through the doc and
>   check this.

Sure.

> > Status of this Memo
> > 
> > 
> ...
> >    The list of Internet-Draft Shadow Directories can be accessed at
> >    http://www.ietf.org/shadow.html.
> >
> > Specification of Requirements
> 
> Nit: move Abstract here. Move requirements after the Acks.

Ok.

> > Abstract
> 
> Should the Abstract say that this spec covers IPv4 only?

Sure.

> > 3. Summary of Operation
> ...
> >    This document uses the term `Autonomous System' (AS) throughout.  The
> >    classic definition of an Autonomous System is a set of routers under
> >    a single technical administration, using an interior gateway protocol
> >    (IGP) and common metrics to determine how to route packets within the
> >    AS, and using an inter-AS routing protocol to determine how to route
> >    packets to other ASs. Since this classic definition was developed, it
> >    has become common for a single AS to use several IGPs and sometimes
> >    several sets of metrics within an AS. The use of the term Autonomous
> >    System here stresses the fact that, even when multiple IGPs and met-
> >    rics are used, the administration of an AS appears to other ASs to
> >    have a single coherent interior routing plan and presents a consis-
> >    tent picture of what destinations are reachable through it.
> 
> Ed: Since 'AS' has been defined before, do we need to repeat the
> definition here?

The definition section before presents a *summary* of the definitions
used in the document. I think that the text reads fine as is, so
I would prefer not to change it.

> ...
> >    peer in the same AS is referred to as an internal peer. Internal BGP
> >    and external BGP are commonly abbreviated IBGP and EBGP.
> 
> Ed: These two have been defined before too

See my previous comment.

> ...
> > Care must be taken to
> >    ensure that the interior routers have all been updated with transit
> >    information before the BGP speakers announce to other ASs that tran-
> >    sit service is being provided.
> 
> What does the last sentence really mean from the implementation
> perspective? It used to mean the BGP/IGP synchronization check. Now
> that iBGP everywhere is assumed, how do we check this condition?

In the absence of any objections by June 10 I suggest to take this 
sentence out.

> >    This document specifies the base behavior of the BGP protocol. This
> >    behavior can and is modified by extention specifications.  When the
> Ed: "extension"

Sure.

> >    protocol is extended the new behavior is fully documented in the
> >    extention specifications.
> Ed: "extension"

Sure.

> 
> > 3.1 Routes: Advertisement and Storage
> > 
> >    For the purpose of this protocol, a route is defined as a unit of
> >    information that pairs a set of destinations with the attributes of a
> >    path to those destinations. The set of destinations are systems whose
> >    IP addresses are contained in one IP address prefix carried in the
> >    Network Layer Reachability Information (NLRI) field of an UPDATE mes-
> >    sage, and the path is the information reported in the path attributes
> >    field of the same UPDATE message.
> Ed: Repeated definition again

See above.

> ...
> >    If a BGP speaker chooses to advertise the route, it MAY add to or
> >    modify the path attributes of the route before advertising it to a
> >    peer.
> 
> The intent here is to say that it's ok to modify the attribute set of
> a previously received route when it's announced further. The way it
> reads though is that self-originated routes are also within the
> context and MAY sounds like you don't have to add attributes when
> announcing those.

I will replace "If a BGP speaker chooses to advertise the route" with
"If a BGP speaker chooses to advertise a previously received route".

> 
> ...
> 
> >    Changing attribute of a route is accomplished by advertising a
> >    replacement route. The replacement route carries new (changed)
> >    attributes and has the same NLRI as the original route.
> 
> "same NLRI" implies the same prefix, but not the NLRI field, which can
> be different (containing other routes), should the use of this term be
> normalized throughout the document?

I will replace "the same NLRI" with "the same address prefix".

> 
> > 4.2 OPEN Message Format
> > 
> >    After a TCP is established, the first message sent by each side is an
> 
> "TCP connection"

ok.

> > 5. Path Attributes
> ...
> >    If a path with recognized transitive optional attribute is accepted
> >    and passed along to other BGP peers and the Partial bit in the
> >    Attribute Flags octet is set to 1 by some previous AS, it is not 
> 
> 'MUST NOT' here?

Sure.

> > set
> >    back to 0 by the current AS. Unrecognized non-transitive optional
> >    attributes MUST be quietly ignored and not passed along to other BGP
> >    peers.
> ...
> >    The same attribute (attribute with the same type) can not appear more
> >    than once within the Path Attributes field of a particular UPDATE
> >    message.
> 
> What should an implementation do if this happens?

See section 6.3:

  If any attribute appears more than once in the UPDATE message, then
  the Error Subcode is set to Malformed Attribute List.

> >    The mandatory category refers to an attribute which MUST be present
> >    in both IBGP and EBGP exchanges if NLRI are contained in the UPDATE
> 
> Ed: "if the NLRI field is contained" instead?

No, as the NLRI field is always present in the UPDATE message
(although if NLRI is not present, then the NLRI field is empty).

> 
> > 5.1.2 AS_PATH
> ...
> >       b) When a given BGP speaker advertises the route to an external
> >       peer, then the advertising speaker updates the AS_PATH attribute
> >       as follows:
> > 
> >          1) if the first path segment of the AS_PATH is of type
> >          AS_SEQUENCE, the local system prepends its own AS number as the
> >          last element of the sequence (put it in the leftmost position).
> 
> 'Leftmost position'... isn't this still open for interpretation? How
> about wording this relative to the position of the octets in the
> protocol message?

I'll replace "the leftmost position" with "the leftmost position with
respect to the position of octets in the protocol message".

> >          If the act of prepending will cause an overflow in the AS_PATH
> >          segment, i.e. more than 255 ASs, it is legal to prepend a new
> >          segment of type AS_SEQUENCE and prepend its own AS number to
> >          this new segment.
> 
> What's the recommended behavior here?

"it is legal to prepend" really means "it SHOULD prepend". 
In the absence of any objections by June 10 I'll update the text.

> 
> 
> > 5.1.4 MULTI_EXIT_DISC
> > 
> > 
> >    The MULTI_EXIT_DISC is an optional non-transitive attribute which is
> >    intended to be used on external (inter-AS) links to discriminate
> >    among multiple exit or entry points to the same neighboring AS.  The
> >    value of the MULTI_EXIT_DISC attribute is a four octet unsigned num-
> >    ber which is called a metric. All other factors being equal, the exit
> >    point with lower metric SHOULD be preferred. If received over EBGP,
> >    the MULTI_EXIT_DISC attribute MAY be propagated over IBGP to other
> >    BGP speakers within the same AS. The MULTI_EXIT_DISC attribute
> 
> seems that a reference to 9.1.2.2 is due here, as using MED in local
> route calculation and not propagating it further is dangerous

Sure.

> >    received from a neighboring AS MUST NOT be propagated to other neigh-
> >    boring ASs.
> > 
> >    A BGP speaker MUST IMPLEMENT a mechanism based on local configuration
>                         ^^^^^^^^^lower-case

Sure.

> >    which allows the MULTI_EXIT_DISC attribute to be removed from a
> >    route. This MAY be done prior to determining the degree of preference
> 
> what's the recommended behavior here?

What the text is saying is that a BGP speaker optionally (MAY)
remove MED from a route. If the speaker does this, then this *has
to* happen prior to determining the degree of preference for the
route. So, what "This MAY" refers to is the fact that removing MED
is optional. To clarify I would replace "This MAY be done" with
"Removal of the MULTI_EXIT_DISC attribute MAY be done".

> >    of the route and performing route selection (decision process phases
> >    1 and 2).
> > 
> >    An implementation MAY also (based on local configuration) alter the
> >    value of the MULTI_EXIT_DISC attribute received over EBGP.  This MAY
> >    be done prior to determining the degree of preference of the route
> 
> what's the recommended behavior here?

The same as the previous comment.

> > 5.1.5 LOCAL_PREF
> ...
> > A BGP speaker SHALL calculate the degree of preference for
> >    each external route based on the locally configured policy, and
> 
> Should we be more honest here and say that the implementation must
> allow the admin to SET the degree of preference through the local
> policy to influence the best-path selection process, i.e., I don't
> think any implementation really *calculates* it.

Please see my answer to you comment on 9.1.1

> > 5.1.6 ATOMIC_AGGREGATE
> ...
> >    A BGP speaker that receives a route with the ATOMIC_AGGREGATE
> >    attribute MUST NOT make any NLRI of that route more specific (as
> >    defined in 9.1.4) when advertising this route to other BGP speakers.
> 
> Since deaggregation is not described in this document, do we need this
> para?

I would prefer to keep the current text, as to make sure that an
implementation wouldn't do deaggregation.

> >   A BGP speaker that receives a route with the ATOMIC_AGGREGATE
> >    attribute needs to be cognizant of the fact that the actual path to
> >    destinations, as specified in the NLRI of the route, while having the
> >    loop-free property, may not be the path specified in the AS_PATH
> >    attribute of the route.
> 
> What does this really mean from the implementation perspective?

This is mostly FYI. It has to do with the user of BGP...

> > 5.1.7 AGGREGATOR
> > 
> > 
> >    AGGREGATOR is an optional transitive attribute which MAY be included
> >    in updates which are formed by aggregation (see Section 9.2.2.2). A
> >    BGP speaker which performs route aggregation MAY add the AGGREGATOR
> 
> What's the recommended behavior here? Include or not, and under what
> circumstances?

The spec doesn't provide any recommendation on this, as it is optional.

> > 6. BGP Error Handling.
> ...
> >    The phrase "the BGP connection is closed" means that the TCP connec-
> >    tion has been closed, the associated Adj-RIB-In has been cleared, and
> >    that all resources for that BGP connection have been deallocated.
> >    Entries in the Loc-RIB associated with the remote peer are marked as
> >    invalid. The fact that the routes have become invalid is passed to
> >    other BGP peers before the routes are deleted from the system.
> 
> What does "the fact is passed" mean? Should we instead say that local
> route recalculation happens and peers are sent either updated best
> routes or withdrawals?

How about the following replacement for the last sentence:

   The local system recalculates its best routes for the destinations
   of the routes marked as invalid, and advertises to its peers either
   withdraws for the routes marked as invalid, or the new best routes
   before the invalid routes are deleted from the system.

> > 6.2 OPEN message error handling.
> ...
> >    If the Autonomous System field of the OPEN message is unacceptable,
> >    then the Error Subcode is set to Bad Peer AS. The determination of
> >    acceptable Autonomous System numbers is outside the scope of this
> >    protocol.
> 
> Shouldn't we say that configuration based detection should be
> supported, i.e., when remote-as is configured for the peer?

No.

> ...
> >   If the BGP Identifier field of the OPEN message is syntactically
> >    incorrect, then the Error Subcode is set to Bad BGP Identifier.  Syn-
> >    tactic correctness means that the BGP Identifier field represents a
> >    valid IP host address.
> 
> Is "valid IP host address" defined somewhere, btw?

Certainly not in this document. Perhaps for clarity I'll add
"unicast" in front of "IP host address".

> > 6.3 UPDATE message error handling.
> > 
> > 
> >    All errors detected while processing the UPDATE message are indicated
> >    by sending the NOTIFICATION message with Error Code UPDATE Message
> >    Error. The error subcode elaborates on the specific nature of the
> >    error.
> 
> "are indicated..." is this a MUST, SHOULD, or MAY?

MUST.

> ...
> >    If the ORIGIN attribute has an undefined value, then the Error Sub-
> >    code is set to Invalid Origin Attribute. The Data field contains the
> >    unrecognized attribute (type, length and value).
> 
> Curious: do we really have to drop a session on this condition? Given
> that the attribute was syntactically correct and the TLV was not
> broken, so the stream is still in sync and we can move on? Of course,
> if this is what current implementations do, we have no other choice.

In the current spec all the errors are fatal. Including errors
in the ORIGIN attribute.
  
> ...
> >    If the UPDATE message is received from an external peer, the local
> >    system MAY check whether the leftmost AS in the AS_PATH attribute is
> 
> Same comment about 'leftmost'... Maybe we should define this somewhere
> in the beginning of the spec?

I will replace "the leftmost AS" with "the leftmost AS with
respect to the position of octets in the protocol message".
  
> ...
> >    The NLRI field in the UPDATE message is checked for syntactic valid-
> >    ity. If the field is syntactically incorrect, then the Error Subcode
> >    is set to Invalid Network Field.
> 
> Should we give more data on what syntactic validity means in this case
> so people behave consistently?

As Curtis suggested a while ago:

     If the document is unclear to the well qualified reader (one
     possessing a thorough understanding of foundations of this work,
     including IP routing, TCP, TCP programming, and the referenced
     documents) then the document may need to be changed to improve
     clarity.

The case you mentioned above suggests that the reader is not
well qualified.

> > 6.7 Cease.
> ...
> > If the BGP speaker decides to terminate its BGP
> >    connection with a neighbor because the number of address prefixes
> >    received from the neighbor exceeds the locally configured upper
> >    bound, then the speaker MUST send to the neighbor a NOTIFICATION mes-
> >    sage with the Error Code Cease.
> 
> Should we also say that when the peer decides to discard incoming
> prefixes, this event should be logged locally?

In the absence of any objections by June 10 I'll add the following to 
the text:

    The speaker MAY also log this locally.

> > 9. UPDATE Message Handling
> > 
> > 
> >    An UPDATE message may be received only in the Established state.
>
> What if it is received in another state?

It is an error. To make this clear I'll add the following to the
text:

   Receiving an UPDATE message in any other state is an error.

> ...
> > 9.1 Decision Process
> > 
> > 
> >    The Decision Process selects routes for subsequent advertisement by
> >    applying the policies in the local Policy Information Base (PIB) to
> >    the routes stored in its Adj-RIBs-In. The output of the Decision Pro-
> >    cess is the set of routes that will be advertised to peers; the
> >    selected routes will be stored in the local speaker's Adj-RIB-Out
> RIB-Out or RIBs-out (plural)?

Plural.

> >    according to policy.
> > 
> >    The selection process is formalized by defining a function that takes
> >    the attribute of a given route as an argument and returns either (a)
> >    a non-negative integer denoting the degree of preference for the
> >    route, or (b) a value denoting that this route is ineligible to be
> >    installed in LocRib and will be excluded from the next phase of route
> 
> Loc-RIB

Ok.

> >    selection.
> ...
> >    The Decision Process operates on routes contained in the Adj-RIB-In,
> Adj-RIBs-In (plural) ?

Plural.

> >    and is responsible for:
> 
> > 9.1.1 Phase 1: Calculation of Degree of Preference
> ...
> >       If the route is learned from an external peer, then the local BGP
> >       speaker computes the degree of preference based on preconfigured
> >       policy information. If the return value indicates that the route
> >       is ineligible, the route MAY NOT serve as an input to the next
> >       phase of route selection; otherwise the return value is used as
> >       the LOCAL_PREF value in any IBGP readvertisement.
> 
> So, AFAIK, the major implementations do not follow this step
> (calculating the degree of preference, and then announcing). Instead,
> implementations allow setting the LOCAL_PREF value locally, which is
> taken into consideration during the best path selection, and is also
> reannounced further.

It is important to keep in mind that the whole section on the BGP
decision process does *not* mean that an implementation must implement
it precisely as it is described in the spec, as long as the implementation 
support the described functionality and its externally visible behavior 
is the same. With this in mind how about if I'll add the following:

   The BGP Decision Process in this document is conceptual and do
   not have to be implemented precisely as described here, as long
   as the implementations support the described functionality and
   their externally visible behavior is the same.

> Also "is used" is not specific enough. Is it SHOULD or MUST?

MUST.

> > 9.1.2 Phase 2: Route Selection
> ...
> >    If the AS_PATH attribute of a BGP route contains an AS loop, the BGP
> >    route should be excluded from the Phase 2 decision function.  AS loop
> >    detection is done by scanning the full AS path (as specified in the
> >    AS_PATH attribute), and checking that the autonomous system number of
> >    the local system does not appear in the AS path.  Operations of a BGP
> >    speaker that is configured to accept routes with its own autonomous
> >    system number in the AS path are outside the scope of this document.
> 
> If we're checking for an AS loop here (in Phase 2) as opposed to
> during the UPDATE message sanity checking, the route is already
> received and accepted in the peer's Adj-RIB-In. Those implementations
> I know don't even install such routes in the RIB...

This is the text that the WG agreed on (see e-mail from John Scudder on
Mon, 02 Dec 2002 10:54:45 EST). Also, see my response to your previous
comment.

> > 9.1.2.2 Breaking Ties (Phase 2)
> ...
> >       Similarly, neighborAS(n) is a function which returns the neighbor
> >       AS from which the route was received.  If the route is learned via
> >       IBGP, and the other IBGP speaker didn't originate the route, it is
> >       the neighbor AS from which the other IBGP speaker learned the
> >       route. If the route is learned via IBGP, and the other IBGP
> >       speaker originated the route, it is the local AS.
> 
> What if the route is locally originated?

Breaking ties has to do with the routes received from other BGP speakers,
not with the routes locally originated.

> > 9.1.4 Overlapping Routes
> ...
> >    When overlapping routes are present in the same Adj-RIB-In, the more
> >    specific route takes precedence, in order from more specific to least
> >    specific.
> > 
> Doesn't this happen at the packet forwarding stage?

Yes, it does. But only if both routes are present in the FIB.
I also think that this sentence isn't needed, so in the absence
of any objections by June 10 I propose to remove it.

> >    The set of destinations described by the overlap represents a portion
> >    of the less specific route that is feasible, but is not currently in
> >    use.  If a more specific route is later withdrawn, the set of desti-
> >    nations described by the overlap will still be reachable using the
> >    less specific route.
> > 
> >    If a BGP speaker receives overlapping routes, the Decision Process
> >    MUST consider both routes based on the configured acceptance policy.
> >    If both a less and a more specific route are accepted, then the Deci-
> >    sion Process MUST either install both the less and the more specific
>   
> Install where?

In Loc-RIB. I'll insert "in Loc-RIB" to make this clear.

> >    routes or it MUST aggregate the two routes and install the aggregated
> >    route, provided that both routes have the same value of the NEXT_HOP
> >    attribute.
> 
> anyone really does the latter?

Will find this from the implemenation report.

> >    If a BGP speaker chooses to aggregate, then it SHOULD either include
> >    all AS used to form the aggreagate in an AS_SET or add the
> >    ATOMIC_AGGREGATE attribute to the route.  This attribute is now pri-
> >    marily informational.  With the elimination of IP routing protocols
> >    that do not support classless routing and the elimination of router
> >    and host implementations that do not support classless routing, there
> >    is no longer a need to deaggregate.  Routes SHOULD NOT be de-aggre-
> >    gated.  A route that carries ATOMIC_AGGREGATE attribute in particular
> >    MUST NOT be de-aggregated. That is, the NLRI of this route can not be
> >    made more specific. Forwarding along such a route does not guarantee
> >    that IP packets will actually traverse only ASs listed in the AS_PATH
> >    attribute of the route.
> 
> Since we don't do deaggregation any more, should we remove the
> discussion about it completely and indicate in the "changes" section
> that deaggregation has been deprecated?

As I said before, I would prefer to keep the text on de-aggregation in.

> > 9.2 Update-Send Process
> ...
> >    When a BGP speaker receives an UPDATE message from an internal peer,
> >    the receiving BGP speaker SHALL NOT re-distribute the routing infor-
> >    mation contained in that UPDATE message to other internal peers,
> >    unless the speaker acts as a BGP Route Reflector [RFC2796].
> 
> Suggest to put "unless..." in brackets () to make it more apparent
> that this is not a normative ref.

Ok.

> > 9.2.1.1 Frequency of Route Advertisement
> >    Since fast convergence is needed within an autonomous system, either
> >    (a) the MinRouteAdvertisementInterval used for internal peers SHOULD
> >    be shorter than the MinRouteAdvertisementInterval used for external
> >    peers, or (b) the procedure describe in this section SHOULD NOT apply
> >    for routes sent to internal peers.
> 
> It sounded like MinRouteAdvertisementInterval was an architectural
> constant, but now it sounds like either this is a timer that can be
> assigned different settings or there are two constants:
> MinRouteAdvIntIBGP and MinRouteAdvIntEBGP.

There is a timer (MinRouteAdvertisementInterval) that can be assigned 
different settings.
  
> > 9.2.2.2 Aggregating Routing Information
> > 
> 
> Hmmm... I expected to see in this section some text talking about when
> and how an aggregate would be announced, i.e., when an aggregate
> prefix is configured, and more specific routes are present, the
> aggregate is announced, when no specifics are left--withdraw the
> aggregate. I haven't found anything on this topic...

That is outside the scope of the *protocol* spec. See rfc1519 for
more on this.

> > 9.3 Route Selection Criteria
> >
> >    Generally speaking, additional rules for comparing routes among sev-
> >    eral alternatives are outside the scope of this document. There are
> >    two exceptions:
> > 
> >       - If the local AS appears in the AS path of the new route being
> >       considered, then that new route can not be viewed as better than
> >       any other route (provided that the speaker is configured to accept
> >       such routes). If such a route were ever used, a routing loop could
> >       result.
> > 
> >       - In order to achieve successful distributed operation, only
> >       routes with a likelihood of stability can be chosen. Thus, an AS
> >       SHOULD avoid using unstable routes, and it SHOULD NOT make rapid
> >       spontaneous changes to its choice of route. Quantifying the terms
> >       "unstable" and "rapid" in the previous sentence will require expe-
> >       rience, but the principle is clear.
> 
> Where does this (the second one) fit within and how does this affect
> the route selection criteria?

Routes that flap often can be "penalize" (e.g., route dampening).
I'll add a pointer to the route dampening spec here.
  
> >    Care must be taken to ensure that BGP speakers in the same AS do not
> >    make inconsistent decisions.
> 
> How? 

By means outside of the protocol. How about if I'll just remove this
sentence ?

> What does this mean for the implementor?
>
> > 9.4 Originating BGP routes
> > 
> >    A BGP speaker may originate BGP routes by injecting routing informa-
> >    tion acquired by some other means (e.g. via an IGP) into BGP. A BGP
> >    speaker that originates BGP routes assigns the degree of preference
> > 
> 
> "assigns the degree of preference"... how do the implementations
> really do that?

E.g., via CLI. I'll add "(e.g., via CLI") after "assigns the degree
of preference".
  
> > 10 BGP Timers
> ...
> >    The suggested default value for the MinRouteAdvertisementInterval is
> >    30 seconds.
> 
> This was described as a parameter, not a timer. Further, it was
> earlier suggested that it should be shorter for iBGP than it is for
> eBGP. I'd expect the document to specify the recommended value for
> both.

This is for eBGP. For iBGP the suggested value is 5 secs (I'll add this
to the draft).

> > IANA Considerations
> ...
> >    All extensions to this protocol, including new message types and Path
> >    Attributes MUST only be made using the Standards Action process
> >    defined in [RFC2434].
> 
> This section should include the description of each registry that
> needs to be created (if needed) and maintained by IANA, as well as the
> allocation policy that is in the text already.

Sure.

Yakov.

Date: Fri, 23 May 2003 14:51:16 -0600
Subject: Re: EBGP - Setting Nexthop
From: Danny McPherson <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

On 5/23/03 2:19 PM, "Parag Deshpande" <[email-masked]> wrote:

> Thanks Danny,
> 
> Then what I get is that it really doesn't matter what the default
> behavior is since knobs (policies) can be used to modify NEXT_HOP
> as needed. (?)

Correct.  Both on the transmit and receive (e.g., enforce next-hop)
sides.

-danny

Date: Fri, 23 May 2003 15:19:30 -0500
From: "Parag Deshpande" <[email-masked]>
To: "Danny McPherson" <[email-masked]>, [email-masked]
Subject: Re: EBGP - Setting Nexthop
Sender: [email-masked]

Thanks Danny,

Then what I get is that it really doesn't matter what the default behavior
is since
knobs (policies) can be used to modify NEXT_HOP as needed. (?)

Parag

> > Hi,
> >
> > I have a doubt regarding setting of nexthop value in the following
scenario:
> >
> > Router R has 2 ebgp peers A and B, all on same subnet S1.
> > A - Sends a prefix to R with nexthop N1 where N1 = A.
> > R - Installs and then forwards the prefix to B with N1 = ?
> >
> > In this case should R set N1 = A or N1 = R (on S1).
>
> It's a matter of policy, really.  For instance, perhaps A and B don't peer
> directly and A doesn't want to accept packets directly from B, so setting
a
> third party NEXT_HOP may break things (e.g., Link Layer filtering is
> implemented or no Layer 2 connection exists directly between A and B, even
> though A, B & R share a common subnet) or violate some policy (In a
previous
> job we peered with a network at a multi-access exchange point -- purely
out
> of goodwill.  They began sending us lots of traffic and after some
> investigation we realized they were selling transit across the local
> exchange point via readvertising our routes to their transit customers and
> preserving the NEXT_HOP, such that their customers were sending traffic
> directly to us -- they never touched the outbound traffic!  Needless to
say,
> MAC-Layer filtering was deployed shortly thereafter).
>
> On the other hand, perhaps they're all in agreement that this is a fine
> thing in order to optimize the forwarding path AND connectivity exists
such
> that B can send traffic directly to A -- so it makes sense for R to
preserve
> the NEXT_HOP.
>
> > I saw a major vendor setting it to R. Is that a preffered practice?
> > If yes why?
>
> Again, it's all a matter of policy, and all the "major vendors" I'm
familiar
> with provide the knobs to set it pretty much however you prefer, though I
> have seen some variances in default behaviors.
>
> -danny
>
>
>

Date: Fri, 23 May 2003 13:03:27 -0600
Subject: Re: EBGP - Setting Nexthop
From: Danny McPherson <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

On 5/23/03 10:45 AM, "Parag Deshpande" <[email-masked]> wrote:

> Hi,
> 
> I have a doubt regarding setting of nexthop value in the following scenario:
> 
> Router R has 2 ebgp peers A and B, all on same subnet S1.
> A - Sends a prefix to R with nexthop N1 where N1 = A.
> R - Installs and then forwards the prefix to B with N1 = ?
> 
> In this case should R set N1 = A or N1 = R (on S1).

It's a matter of policy, really.  For instance, perhaps A and B don't peer
directly and A doesn't want to accept packets directly from B, so setting a
third party NEXT_HOP may break things (e.g., Link Layer filtering is
implemented or no Layer 2 connection exists directly between A and B, even
though A, B & R share a common subnet) or violate some policy (In a previous
job we peered with a network at a multi-access exchange point -- purely out
of goodwill.  They began sending us lots of traffic and after some
investigation we realized they were selling transit across the local
exchange point via readvertising our routes to their transit customers and
preserving the NEXT_HOP, such that their customers were sending traffic
directly to us -- they never touched the outbound traffic!  Needless to say,
MAC-Layer filtering was deployed shortly thereafter).

On the other hand, perhaps they're all in agreement that this is a fine
thing in order to optimize the forwarding path AND connectivity exists such
that B can send traffic directly to A -- so it makes sense for R to preserve
the NEXT_HOP.  

> I saw a major vendor setting it to R. Is that a preffered practice?
> If yes why?

Again, it's all a matter of policy, and all the "major vendors" I'm familiar
with provide the knobs to set it pretty much however you prefer, though I
have seen some variances in default behaviors.

-danny

Date: Fri, 23 May 2003 11:45:38 -0500
From: "Parag Deshpande" <[email-masked]>
To: [email-masked]
Subject: EBGP - Setting Nexthop
Sender: [email-masked]

Hi,

I have a doubt regarding setting of nexthop value in the following scenario:

Router R has 2 ebgp peers A and B, all on same subnet S1.
A - Sends a prefix to R with nexthop N1 where N1 = A.
R - Installs and then forwards the prefix to B with N1 = ?

In this case should R set N1 = A or N1 = R (on S1).

I saw a major vendor setting it to R. Is that a preffered practice?
If yes why?

Reference:
5.1.3 NEXT_HOP
      2) When sending a message to an external peer X, and the peer is
      one IP hop away from the speaker:
      ........
       - Otherwise, if the route being announced was learned from an
         external peer, the speaker can use in the NEXT_HOP attribute an
         IP address of any adjacent router (known from the received
         NEXT_HOP attribute) that the speaker itself uses for local
         route calculation, provided that peer X shares a common subnet
         with this address. This is a second form of "third party"
         NEXT_HOP attribute.
>> N1 = A
         - Otherwise, if the external peer to which the route is being
         advertised shares a common subnet with one of the interfaces of
         the announcing BGP speaker, the speaker MAY use the IP address
         associated with such an interface in the NEXT_HOP attribute.
         This is known as a "first party" NEXT_HOP attribute.
>> N1 = R
       .....

Thanks
Parag

To: IETF-Announce: ;
Cc: RFC Editor <[email-masked]>, Internet Architecture Board <[email-masked]>, [email-masked]
From: The IESG <[email-masked]>
Subject: Document Action: Security Requirements for Keys used with the  TCP MD5 Signature Option to Informational
Date: Wed, 21 May 2003 15:25:14 -0400
Sender: [email-masked]

The IESG has approved the Internet-Draft 'Security Requirements for 
Keys used with the TCP MD5 Signature Option' 
<draft-ietf-idr-md5-keys-00.txt> as an Informational RFC.  This 
document is the product of the Inter-Domain Routing Working Group.  The 
IESG contact persons are Bill Fenner and Alex Zinin.

RFC Editor Note:

Please change the title to "Key Management Considerations for the
TCP MD5 Signature Option".

Please change the following:

In section 3, the first bullet:
OLD:
      o Key lengths SHOULD be between 12 and 24 bytes, with larger keys
        having effectively zero cost when compared to shorter keys.

NEW:
      o Key lengths SHOULD be between 12 and 24 bytes, with larger keys
        having effectively zero additional computational cost when
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        compared to shorter keys.

In section 5, first paragraph:

OLD:
  this option may have lifetimes on the order of months.  It would seem
  prudent, then, to choose a *minimum* key length that guarantees that
  key-guessing runtimes are some reasonable [3-5??] multiple of the
  key-change interval under best-case (for the attacker) practical

NEW:
  this option may have lifetimes on the order of months.  It would seem
  prudent, then, to choose a minimum key length that guarantees that
                              ^^^^^^^ (remove emphasis)
  key-guessing runtimes are some small multiple of the key-change
                                  ^^^^^^^^^^^^^^
  interval under best-case (for the attacker) practical

In section 6, first paragraph:

OLD:
  that the reasonable upper-bound for software-based attack performance
  is 1.0e13 MD5 operations per second, then the *minimum* required key
  entropy is approximately 68 bits.  It is reasonable to round this

NEW:
  that the reasonable upper-bound for software-based attack performance
  is 1.0e13 MD5 operations per second, then the minimum required key
                                                ^^^^^^^ (remove emphasis)
  entropy is approximately 68 bits.  It is reasonable to round this

Date: Tue, 20 May 2003 14:55:57 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: [[email-masked]: Re: Comments on BGP Draft 20.....]
Sender: [email-masked]

Yakov,

----- Forwarded message from Russ White <[email-masked]> -----

Date: Tue, 20 May 2003 14:33:38 -0400 (EDT)
From: Russ White <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Subject: Re: Comments on BGP Draft 20.....
Reply-To: Russ White <[email-masked]>
X-Virus-Scanned: by AMaViS perl-11
X-OriginalArrivalTime: 20 May 2003 18:33:51.0941 (UTC) FILETIME=[5C876750:01C31EFE]

Yeah, this sounds better....

:-)

Russ

On Tue, 20 May 2003, Jeffrey Haas wrote:

> [off-list]
>
> Howzabout:
>    The primary function of a BGP speaking system is to exchange network
>    reachability information with other BGP systems. This network reacha-
>    bility information includes information on the list of Autonomous
>    Systems (ASs) that reachability information traverses.  This informa-
>    tion is sufficient to construct a graph of AS connectivity
> +  for this reachability
>    from which
>    routing loops may be pruned and some policy decisions at the AS level
>    may be enforced.
>
> --
> Jeff Haas
> NextHop Technologies
>

__________________________________
[email-masked] CCIE <>< Grace Alone

----- End forwarded message -----

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 20 May 2003 14:28:28 -0400
From: Jeffrey Haas <[email-masked]>
To: Yakov Rekhter <[email-masked]>
Cc: Russ White <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20.....
Sender: [email-masked]

On Tue, May 20, 2003 at 11:10:37AM -0700, Yakov Rekhter wrote:
> > Okay:
> > 
> > The Loc-RIB contains routes which are installed in the local routing table
> > and used for forwarding packets received, based on the destination address,
> > by the router.
> 
> In the absence of any objections within a week I'll put this in the text.

Except:
:   Whether or not the new BGP route replaces an existing
:   non-BGP route in the Routing Table depends on the policy configured
:   on the BGP speaker.

I think the existing text is fine.  The gotcha is one has to read ahead
a bit in the document to find the bit I just quoted.

> > It's value MUST NOT be changed by any other speaker.
> 
> 
> In the absence of any objections within a week I'll put this in the text.

Good grief.

I refer all parties concerned back to the thread titled "Re: issue 32.1",
specifically the consensus mail from Andrew with message-id:
<[email-masked]

Short summary:
1. You shouldn't change it.
2. People *do* change it, and do so for policy reasons.
3. You shouldn't change it, but since people do, we're going to tell
   you that you shouldn't and thus imply that you can if you really
   want to. :-)

My own preference was to *not* change it and MUST would be fine with me,
but consensus was previously reached.

> Yakov.

-- 
Jeff Haas 
NextHop Technologies

To: Russ White <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20..... 
Date: Tue, 20 May 2003 11:10:37 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Russ,

> > > This information is sufficeint to construct a graph of the AS connectivity
> > > from which routing loops may be pruned and some policy decisions at the AS
> > > level may be enforced.
> > >
> > > UPDATE Message Format:
> > >
> > > The information in the UPDATE message can be used to construct a graph
> > > describing the relationships of the various Autonomous Systems.
> > >
> > > In both cases this is true, I suppose, but in neither case does this 
> > > really describe what the AS Path is used for, right?
> >
> > Wrong. As the abstract states quite clearly that this information is used
> > to prune routing loops and make some policy decisions at the AS level.
> 
> Yes, but it's not used for inscribing a graph of interconnectivity between
> the AS' in the internetwork, is it? It can be used for that, I suppose, so
> the text is fine, but it's not used for that--I think that's what's
> confusing about it.
> 
> > > 3.2 Routing Information Bases
> > >
> > > b) Loc-RIB....
> > >
> > > I think it might be useful to state the contents of the Loc-RIB are
> > > actually installed in the local routing table, and thus used for forwardi
ng
> > > packets on this router. I don't see anyplace this connection is made
> > > explicit, it seems more like it's implicit throughout the doc.
> >
> > Please propose the text.
> 
> Okay:
> 
> The Loc-RIB contains routes which are installed in the local routing table
> and used for forwarding packets received, based on the destination address,
> by the router.

In the absence of any objections within a week I'll put this in the text.

> > > Network Layer Reachability Information
> > >
> > > "An UPDATE message can list multiple routes to be withdrawn...."
> > >
> > > Actually, we don't withdraw routes, we withdraw prefixes, right? The next
> > > paragraph shows this confusion, by talking about routes without attribute
s,
> > > but routes are prefixes combined with attributes, so.... They aren't
> > > routes, they're prefixes. You remove routes based on withdrawn prefixes, 
I
> > > think.
> >
> > We withdraw routes. The way BGP withdraws routes is by advertising
> > the NLRI field of these routes in the Withdrawn Routes field of
> > the UPDATE message. And that is precisely what the text said:
> >
> >    An UPDATE message can list multiple routes to be withdrawn from service.
> >    Each such route is identified by its destination (expressed as an IP
> >    prefix), which unambiguously identifies the route in the context of the
> >    BGP speaker - BGP speaker connection to which it has been previously
> >    advertised.
> 
> Hmmm... So, if you receive an update with no attributes, just prefixes in
> the withdrawn section, you won't consider that a withdraw, and remove the
> routes you have from the sending peer from the local tables?
> 
> A route without the attributes is a prefix. :-)
> 
> It depends on whether you are thinking of it in terms of what you're
> sending, or what you're causing on the receiver.
> 
> > > 5.1.1 ORIGIN
> > >
> > > "Its value SHOULD NOT be changed by any other speaker."
> > >
> > > I really think this should be "MUST NOT." I can't think of any reason it
> > > wouldn't be, except in the case of aggregation, and that case could be
> > > mentioned here as the only known exception (?).
> >
> > Please propose the text.
> 
> It's value MUST NOT be changed by any other speaker.

In the absence of any objections within a week I'll put this in the text.

Yakov.

Date: Tue, 20 May 2003 14:08:18 -0400 (EDT)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20.....
Sender: [email-masked]

> > Yes, but it's not used for inscribing a graph of interconnectivity between
> > the AS' in the internetwork, is it? It can be used for that, I suppose, so
> > the text is fine, but it's not used for that--I think that's what's
> > confusing about it.
>
> Perhaps to elaborate on Russ's point, the AS Path gives us the
> graph for this prefix.  Even with a collection of a bunch of routes,
> we're not guaranteed to have the Internet's AS graph.
>
> The text is a *little* vague in this context, but I can't think
> of better wording.

I agree--I've been trying to come up with a better wayh of working it, but
I can't think of one. I'd say it's too much of a nit to worry about it .

:-)

Russ

__________________________________
[email-masked] CCIE <>< Grace Alone

Date: Tue, 20 May 2003 14:05:12 -0400
From: Jeffrey Haas <[email-masked]>
To: Russ White <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20.....
Sender: [email-masked]

On Tue, May 20, 2003 at 01:44:16PM -0400, Russ White wrote:
> Yes, but it's not used for inscribing a graph of interconnectivity between
> the AS' in the internetwork, is it? It can be used for that, I suppose, so
> the text is fine, but it's not used for that--I think that's what's
> confusing about it.

Perhaps to elaborate on Russ's point, the AS Path gives us the 
graph for this prefix.  Even with a collection of a bunch of routes,
we're not guaranteed to have the Internet's AS graph.

The text is a *little* vague in this context, but I can't think
of better wording.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 20 May 2003 13:44:16 -0400 (EDT)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Yakov Rekhter <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20..... 
Sender: [email-masked]

> > This information is sufficeint to construct a graph of the AS connectivity
> > from which routing loops may be pruned and some policy decisions at the AS
> > level may be enforced.
> >
> > UPDATE Message Format:
> >
> > The information in the UPDATE message can be used to construct a graph
> > describing the relationships of the various Autonomous Systems.
> >
> > In both cases this is true, I suppose, but in neither case does this really
> > describe what the AS Path is used for, right?
>
> Wrong. As the abstract states quite clearly that this information is used
> to prune routing loops and make some policy decisions at the AS level.

Yes, but it's not used for inscribing a graph of interconnectivity between
the AS' in the internetwork, is it? It can be used for that, I suppose, so
the text is fine, but it's not used for that--I think that's what's
confusing about it.

> > 3.2 Routing Information Bases
> >
> > b) Loc-RIB....
> >
> > I think it might be useful to state the contents of the Loc-RIB are
> > actually installed in the local routing table, and thus used for forwarding
> > packets on this router. I don't see anyplace this connection is made
> > explicit, it seems more like it's implicit throughout the doc.
>
> Please propose the text.

Okay:

The Loc-RIB contains routes which are installed in the local routing table
and used for forwarding packets received, based on the destination address,
by the router.

> > Network Layer Reachability Information
> >
> > "An UPDATE message can list multiple routes to be withdrawn...."
> >
> > Actually, we don't withdraw routes, we withdraw prefixes, right? The next
> > paragraph shows this confusion, by talking about routes without attributes,
> > but routes are prefixes combined with attributes, so.... They aren't
> > routes, they're prefixes. You remove routes based on withdrawn prefixes, I
> > think.
>
> We withdraw routes. The way BGP withdraws routes is by advertising
> the NLRI field of these routes in the Withdrawn Routes field of
> the UPDATE message. And that is precisely what the text said:
>
>    An UPDATE message can list multiple routes to be withdrawn from service.
>    Each such route is identified by its destination (expressed as an IP
>    prefix), which unambiguously identifies the route in the context of the
>    BGP speaker - BGP speaker connection to which it has been previously
>    advertised.

Hmmm... So, if you receive an update with no attributes, just prefixes in
the withdrawn section, you won't consider that a withdraw, and remove the
routes you have from the sending peer from the local tables?

A route without the attributes is a prefix. :-)

It depends on whether you are thinking of it in terms of what you're
sending, or what you're causing on the receiver.

> > 5.1.1 ORIGIN
> >
> > "Its value SHOULD NOT be changed by any other speaker."
> >
> > I really think this should be "MUST NOT." I can't think of any reason it
> > wouldn't be, except in the case of aggregation, and that case could be
> > mentioned here as the only known exception (?).
>
> Please propose the text.

It's value MUST NOT be changed by any other speaker.

:-)

Russ

__________________________________
[email-masked] CCIE <>< Grace Alone

To: Russ White <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Comments on BGP Draft 20..... 
Date: Tue, 20 May 2003 09:35:07 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Russ,

> 
> Some of these are going to echo Alex's comments, but that's okay, I think.
> Mostly just nits....
> 
> :-)

Thanks for the comments. My response is in-line...

> 
> Russ
> 
> __________________________________
> [email-masked] CCIE <>< Grace Alone
> 
> -----
> 
> Abstract:
> 
> This information is sufficeint to construct a graph of the AS connectivity
> from which routing loops may be pruned and some policy decisions at the AS
> level may be enforced.
> 
> UPDATE Message Format:
> 
> The information in the UPDATE message can be used to construct a graph
> describing the relationships of the various Autonomous Systems.
> 
> In both cases this is true, I suppose, but in neither case does this really
> describe what the AS Path is used for, right? 

Wrong. As the abstract states quite clearly that this information is used
to prune routing loops and make some policy decisions at the AS level.

> I would think we'd want to
> describe it less in terms of a "graph of the connectivity in the
> internetwork," and more in terms of "a graph of the path through Autonomous
> Systems ued to reach the destination advertised." It could be confusing,
> since there isn't anyplace where we discuss building a graph of
> inconnectivity between the Autonomous Systems....
> 
> -----
> 
> Forwarding Paradigm:
> 
> This document uses the term "Autonomous System" (AS)  throughout....
> 
> This entire paragraph is a repeat--I'd leave it just in the definitions.

The definition section suppose to have a *summary* of the definitions
used in the spec.

> -----
> 
> Forwarding Paradigms:
> 
> The initial data flow....
> 
> This paragraph has two different thoughts in it, one about incremental
> updates, and the other about keeping data that you've received. It seems
> like just putting a return after "as the routing tables change."

The two are related, as the reason for keeing updates you've received
is because the exchange of information is based on incremental updates.

> -----
> 
> Forwarding Paradigms:
> 
> The paragraph starting "KEEPALIVE messages" should, I think, be moved up
> above the section on route exchange. I don't know why, it just seems less
> like it's jumping all over the place that way.

Disagree.

> -----
> 
> 3.1 Routes: Advertisement and Storage
> 
> It almost seems like the section about The initial data flow should maybe
> be put entirely under this section someplace (?).
> 
> The first paragraph in this section is really a definition of a route vs a
> prefix, and should probably be in the definitions.

see above.

> The paragraph "Changing attribute of a route...." needs a "the," or
> attribute needs an "s."

Ok.

> -----
> 
> 3.2 Routing Information Bases
> 
> b) Loc-RIB....
> 
> I think it might be useful to state the contents of the Loc-RIB are
> actually installed in the local routing table, and thus used for forwarding
> packets on this router. I don't see anyplace this connection is made
> explicit, it seems more like it's implicit throughout the doc.

Please propose the text.

> -----
> 
> Page 18, a) LOCAL_PREF
> 
> "....to inform other peers...." should be "....to inform its other
> peers...."

Sure.

> -----
> 
> Network Layer Reachability Information
> 
> "This varibale length field contains a list of IP address prefixes."
> 
> I think we can kill "address" here.

Sure.

> 
> a) Length
> 
> "The Length field inidicates...." The sentence can start with
> "Indicates..."

I prefer to keep the current text.

> 
> b) Prefix
> 
> "The Prefix field indicates...." The sentence can start with
> "Indicates...."

Ditto.

> 
> -----
> 
> Network Layer Reachability Information
> 
> "An UPDATE message can list multiple routes to be withdrawn...."
> 
> Actually, we don't withdraw routes, we withdraw prefixes, right? The next
> paragraph shows this confusion, by talking about routes without attributes,
> but routes are prefixes combined with attributes, so.... They aren't
> routes, they're prefixes. You remove routes based on withdrawn prefixes, I
> think.

We withdraw routes. The way BGP withdraws routes is by advertising
the NLRI field of these routes in the Withdrawn Routes field of
the UPDATE message. And that is precisely what the text said:

   An UPDATE message can list multiple routes to be withdrawn from service.
   Each such route is identified by its destination (expressed as an IP
   prefix), which unambiguously identifies the route in the context of the
   BGP speaker - BGP speaker connection to which it has been previously
   advertised.

> 
> ------
> 
> 5. Path Attributes
> 
> "Well-known attributes MUST be recognized by all BGP implementations."
> 
> This sentence, as strange as it may sound, implies it's the attributes
> fault if the BGP implementation doesn't recogonize it, that it's up to the
> attribute definers to, in some way, make certain that BGP implementations
> will recognize it. I think it should probably be worded the other way
> 'round:
> 
> "BGP implementations MUST recognize all well-known attributes."

Sure.

> -----
> 
> 5. Path Attributes
> 
> "All well-known attributes MUST be passed along (after proper updating, if
> necessary) to other BGP peers."
> 
> This just seems a little rough. Maybe this:
> 
> "Once a BGP peer has updated any well-known attributes, it MUST pass these
> attributes in any updates it transmits to its peers."

Sure.

> 
> -----
> 
> 5.1.1 ORIGIN
> 
> "Its value SHOULD NOT be changed by any other speaker."
> 
> I really think this should be "MUST NOT." I can't think of any reason it
> wouldn't be, except in the case of aggregation, and that case could be
> mentioned here as the only known exception (?).

Please propose the text.

Yakov.

Date: Mon, 19 May 2003 09:29:21 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
To: Mareline Sheldon <[email-masked]>
Cc: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Mareline,
The design implicitly takes care of the scenario you explain.  Though I
confess that I have not been really clear on this in this version of the
draft.

To advertise two ECMP routes with different attributes we will use two
UPDATEs where each will be sent with a blank ECMP_NEXT_HOP attribute (nos.
of next-hops will be kept as zero). The receiver upon receiving such
UPDATEs will know that since the ECMP_NEXT_HOP attribute is present in the
UPDATE, it needs to be added in addition to what it has already.

I guess the following text needs to be added in the draft.

The receiver SHOULD not remove any previous route and add the route
received with an ECMP_NEXT_HOP attribute rather than replace the previous
routes.

When advertising more than one ECMP hop with identical attributes the
sender SHOULD send a single update with multiple hops listed in the
ECMP_NEXT_HOP attribute.

When advertising more than one ECMP hop which do not have identical
attributes multiple BGP updates MUST be sent with the ECMP_NEXT_HOP
attribute included to suppress route replacement.

But a more important question is that whether we need this kind of
mechanism in BGP or not. We already have multiple drafts proposed which
strive to achieve similar goals using different techniques and mechanisms.
I guess one motivation being, to allow inter-operatibility between
different vendors to allow advertisement of multiple BGP paths of same
preference.

Once we're through with the above discussion, we can sit down and look into
the nitty-gritties of each proposal.

Regards,
Manav

----- Original Message ----- 
From: "Mareline Sheldon" <[email-masked]>
To: "Manav Bhatia" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, May 18, 2003 9:54 AM
Subject: Re: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt

> Manav,
> Can we using this draft advertise two ECMP routes of equal preference but
say, with different
> AS Paths. As far as i could gather you use one additional attribute to
describe equal cost
> routes using the same path attributes. This way you can definitely
advertise routes with all
> the same attributes. But what happens say when one of my path attributes
are different. Eg. AS
> PATH 112 123 in one attribute and AS PATH 564 232 in the other?
>
> Can this be done here?
>
> Regards,
> Mareline S.
>
> --- Manav Bhatia <[email-masked]> wrote:
> > Hi,
> > Please look into this new Internet draft which is available from the
> > on-line Internet-Drafts directories.
> >
> > I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> > To: IETF-Announce: ;
> > Subject: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> > From: [email-masked]
> > Date: Thu, 15 May 2003 07:19:32 -0400
> > Reply-to: [email-masked]
> > Sender: [email-masked]
> >
> > A New Internet-Draft is available
> >
> > Title  : Advertising Equal Cost Multi-Path (ECMP) routes in BGP
> > Author(s) : M. Bhatia
> > Filename : draft-bhatia-ecmp-routes-in-bgp-00.txt
> > Pages  : 7
> > Date  : 2003-5-14
> >
> > This document describes an extensible mechanism that will allow a BGP
> > [BGP4] speaker to advertise equal cost multi-path (ECMP) routes for a
> > destination to its peers without changing the semantics of the UPDATE
> > message.
> >
> > A new BGP attribute is introduced that will be used to advertise the
> > multiple next hops for the feasible and the un-feasible ECMP BGP routes
to
> > the remote peers.
> >
> > The mechanisms described in this document are applicable to all
routers,
> > both those with the ability to inject multiple routing  entries in
their
> > forwarding table and those without (although the latter need not
implement
> > some extensions described in this document).
> >
> > A URL for this Internet-Draft is:
> >
http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-00.txt
> >
> >
> > To remove yourself from the IETF Announcement list, send a message to
> > ietf-announce-request with the word unsubscribe in the body of the
message.
> >
> > Internet-Drafts are also available by anonymous FTP. Login with the
> > username "anonymous" and a password of your e-mail address. After
logging
> > in, type "cd internet-drafts" and then
> >  "get draft-bhatia-ecmp-routes-in-bgp-00.txt".
> >
> >
> >
>
>
> __________________________________
> Do you Yahoo!?
> The New Yahoo! Search - Faster. Easier. Bingo.
> http://search.yahoo.com
>

Date: Sat, 17 May 2003 21:24:24 -0700 (PDT)
From: Mareline Sheldon <[email-masked]>
Subject: Re: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
To: Manav Bhatia <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

Manav,
Can we using this draft advertise two ECMP routes of equal preference but say, with different
AS Paths. As far as i could gather you use one additional attribute to describe equal cost
routes using the same path attributes. This way you can definitely advertise routes with all
the same attributes. But what happens say when one of my path attributes are different. Eg. AS
PATH 112 123 in one attribute and AS PATH 564 232 in the other?

Can this be done here?

Regards,
Mareline S.

--- Manav Bhatia <[email-masked]> wrote:
> Hi,
> Please look into this new Internet draft which is available from the
> on-line Internet-Drafts directories.
> 
> I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> To: IETF-Announce: ;
> Subject: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> From: [email-masked]
> Date: Thu, 15 May 2003 07:19:32 -0400
> Reply-to: [email-masked]
> Sender: [email-masked]
> 
> A New Internet-Draft is available
> 
> Title  : Advertising Equal Cost Multi-Path (ECMP) routes in BGP
> Author(s) : M. Bhatia
> Filename : draft-bhatia-ecmp-routes-in-bgp-00.txt
> Pages  : 7
> Date  : 2003-5-14
> 
> This document describes an extensible mechanism that will allow a BGP
> [BGP4] speaker to advertise equal cost multi-path (ECMP) routes for a
> destination to its peers without changing the semantics of the UPDATE
> message.
> 
> A new BGP attribute is introduced that will be used to advertise the
> multiple next hops for the feasible and the un-feasible ECMP BGP routes to
> the remote peers.
> 
> The mechanisms described in this document are applicable to all routers,
> both those with the ability to inject multiple routing  entries in their
> forwarding table and those without (although the latter need not implement
> some extensions described in this document).
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-00.txt
> 
> 
> To remove yourself from the IETF Announcement list, send a message to
> ietf-announce-request with the word unsubscribe in the body of the message.
> 
> Internet-Drafts are also available by anonymous FTP. Login with the
> username "anonymous" and a password of your e-mail address. After logging
> in, type "cd internet-drafts" and then
>  "get draft-bhatia-ecmp-routes-in-bgp-00.txt".
> 
> 
> 

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com

To: Mireille Shammas <[email-masked]>
Cc: Manav Bhatia <[email-masked]>, [email-masked]
Reply-To: [email-masked]
Subject: Re: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt 
Date: Thu, 15 May 2003 12:50:40 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Mireille Shammas writes:
> Hi Manav,
> In this draft you don't mention anything about ECMP in a BGP/MPLS VPN network
> and more precisely ECMP between PE and CE where EBGP is used. Note that IBGP 
> is
> used to relay the MPLS label information between PEs. I think this will total
> ly
> depend on how you choose the labels on a local PE to advertise to a remote PE
> to carry the VPN traffic back to the CE. Sample scenario below:
> 
> |      |-------ebgp session--------|
> |                                                        |       |
> | CE|-------ebgp session--------| PE1| =======IBGP/MPLS===========|PE2| .....
> ..
> 
> |      |-------ebgp session--------|
> |                                                        |       |
> 
>  I am mainly interested in what will happen on PE1, and how to balance
> VPN/labelled traffic coming from PE1 towards CE .
> Thanks
> Mireille

The MPLS LSP ends at PE1.  PE2 doesn't care what happens at the other
end of the LSP.  PE1 is free to do the load split exactly as it does
now without telling PE2 that there are more than one next-hop.

The more interesting (difficult) cases are where PE1 has EBGP peers
that advertise routes with different but equal preference attributes
and it doesn't matter at all whether MPLS is in use.

The really difficult (and common) case is where equal cost routes are
advertised by two border routers into the IBGP mesh and a router
internal to the mesh splits among the two routers.  This router does
not advertise anything to BGP about this route and therefore cannot
advertise it as a multipath.

Curtis

Date: Thu, 15 May 2003 10:38:44 -0400
From: Mireille Shammas <[email-masked]>
Organization: Alcatel CID
To: Manav Bhatia <[email-masked]>
Cc: [email-masked]
Subject: Re: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
Sender: [email-masked]

Hi Manav,
In this draft you don't mention anything about ECMP in a BGP/MPLS VPN network
and more precisely ECMP between PE and CE where EBGP is used. Note that IBGP is
used to relay the MPLS label information between PEs. I think this will totally
depend on how you choose the labels on a local PE to advertise to a remote PE
to carry the VPN traffic back to the CE. Sample scenario below:

|      |-------ebgp session--------|
|                                                        |       |
| CE|-------ebgp session--------| PE1| =======IBGP/MPLS===========|PE2| .......

|      |-------ebgp session--------|
|                                                        |       |

 I am mainly interested in what will happen on PE1, and how to balance
VPN/labelled traffic coming from PE1 towards CE .
Thanks
Mireille

Manav Bhatia wrote:

> Hi,
> Please look into this new Internet draft which is available from the
> on-line Internet-Drafts directories.
>
> I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> To: IETF-Announce: ;
> Subject: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
> From: [email-masked]
> Date: Thu, 15 May 2003 07:19:32 -0400
> Reply-to: [email-masked]
> Sender: [email-masked]
>
> A New Internet-Draft is available
>
> Title  : Advertising Equal Cost Multi-Path (ECMP) routes in BGP
> Author(s) : M. Bhatia
> Filename : draft-bhatia-ecmp-routes-in-bgp-00.txt
> Pages  : 7
> Date  : 2003-5-14
>
> This document describes an extensible mechanism that will allow a BGP
> [BGP4] speaker to advertise equal cost multi-path (ECMP) routes for a
> destination to its peers without changing the semantics of the UPDATE
> message.
>
> A new BGP attribute is introduced that will be used to advertise the
> multiple next hops for the feasible and the un-feasible ECMP BGP routes to
> the remote peers.
>
> The mechanisms described in this document are applicable to all routers,
> both those with the ability to inject multiple routing  entries in their
> forwarding table and those without (although the latter need not implement
> some extensions described in this document).
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-00.txt
>
> To remove yourself from the IETF Announcement list, send a message to
> ietf-announce-request with the word unsubscribe in the body of the message.
>
> Internet-Drafts are also available by anonymous FTP. Login with the
> username "anonymous" and a password of your e-mail address. After logging
> in, type "cd internet-drafts" and then
>  "get draft-bhatia-ecmp-routes-in-bgp-00.txt".

Date: Thu, 15 May 2003 18:38:48 +0530
From: Manav Bhatia <[email-masked]>
Subject: FW: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
To: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Hi,
Please look into this new Internet draft which is available from the
on-line Internet-Drafts directories.

I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
To: IETF-Announce: ;
Subject: I-D ACTION:draft-bhatia-ecmp-routes-in-bgp-00.txt
From: [email-masked]
Date: Thu, 15 May 2003 07:19:32 -0400
Reply-to: [email-masked]
Sender: [email-masked]

A New Internet-Draft is available

Title  : Advertising Equal Cost Multi-Path (ECMP) routes in BGP
Author(s) : M. Bhatia
Filename : draft-bhatia-ecmp-routes-in-bgp-00.txt
Pages  : 7
Date  : 2003-5-14

This document describes an extensible mechanism that will allow a BGP
[BGP4] speaker to advertise equal cost multi-path (ECMP) routes for a
destination to its peers without changing the semantics of the UPDATE
message.

A new BGP attribute is introduced that will be used to advertise the
multiple next hops for the feasible and the un-feasible ECMP BGP routes to
the remote peers.

The mechanisms described in this document are applicable to all routers,
both those with the ability to inject multiple routing  entries in their
forwarding table and those without (although the latter need not implement
some extensions described in this document).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-00.txt

To remove yourself from the IETF Announcement list, send a message to
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the
username "anonymous" and a password of your e-mail address. After logging
in, type "cd internet-drafts" and then
 "get draft-bhatia-ecmp-routes-in-bgp-00.txt".

To: David Meyer <[email-masked]>
Cc: Curtis Villamizar <[email-masked]>, [email-masked]
Reply-To: [email-masked]
Subject: Re: draft-ietf-idr-bgp-analysis-03.txt 
Date: Wed, 14 May 2003 18:06:08 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, David Meyer writes:
> 
> 	Curtis,
> 	
> 	These are great comments. One point
> 
> >> ------------
> >> 
> >> The following statement is incorrect:
> >> 
> >>    Finally, since the dynamic properties of the network cannot be
> >>    quantitatively bounded, stability must be addressed via heuristics
> >>    such as BGP Route Flap Damping [RFC2439]. Due to the nature of BGP,
> >>    such damping should be viewed as a matter local to an autonomous
> >>    system matter (see also Appendix F.2 of [BGP4]).
> >> 
> >> The amount of change is inherently bounded in BGP (as I described
> >> above).  BGP Route Flap Damping was initially proposed for two
> >> reasons, 1) to protect a specific commercial implementation that was
> >> not sufficiently robust, 2) to improve convergence of stable routes.
> >> BGP Route Flap Damping is not necessary to bound the amount of change
> >> in BPG routing.
> 
> 	Yes, but route flap dampening is just a heuristic that we
> 	use because the dynamics can't be bounded; that's what I
> 	was after. Do you disagree with this?
> 
> 	Dave

I made a comment on that later in my earlier note but I'll provide
more detail.

There were two initial motivations for BGP Route Flap Damping.  One
was a certain BGP implementation wasn't robust when we first started
but was much more robust by the time the RFC came out.  Initially if
you pushed hard enough it would fall over.  The second reason was even
after implementations out there were all quite robust, convergence for
stable routes was a lot slower than we'd like if there were a lot of
unstable routes.  [A third reason was a certain router would drop
packets when it had a route cache implementation but we'd all rather
forget about that design error.]  Anyway these reasons motivated the
idea and kept it going.

BGP is inherently work conserving, meaning that above some amount of
incoming change for a given number of prefixes, the amount of work and
the amount of outgoing change was either bound by the number of
flapping prefixes or constant above some number of prefixes (saturated
CPU).  This amount of work and outgoing (advertised out) churn is
bounded but still quite high.  BGP Route Flap Damping was deployed in
recognition of the fact that a very small percentage of the prefixes
representing an even smaller part of the reachable address space where
contributing most of the churn and the vast majority of prefixes were
quite stable.  That was the practical reason that providers were
willing to turn on BGP Route Flap Damping and have to deal with the
occasional headache of clearing the history for prefixes that they'd
rather not get damped (or where the problem was know to have been
fixed - like when some other NOC called).

[ As a side note, BGP Route Flap Damping was never implemented
correctly.  It should take the AS path into consideration and only
damp an unstable path if another existed that was quite stable, not
damp the prefix.  It was known that per prefix damping would cause
problems so in the spec, the AS path as part of the key field was not
optional.  If it had been implemented correctly we would not have the
problems with BGP Route Flap Damping that we have experienced. ]

My argument amounts to 1) route churn IS bounded, however 2) BGP Route
Flap Damping exists because a) the bounds is uncomfortably high, b)
there used to be some broken implementations in use that were not
sufficiently robust, c) a small percentage of prefixes contributed
most of the churn, d) getting rid of that small percentage was
percieved as good for the Internet at large (the majority of stable
prefixes and reachable destinations).

Curtis

Date: Wed, 14 May 2003 10:15:28 -0700
From: Keyur Patel <[email-masked]>
To: [email-masked]
Cc: [email-masked], David Meyer <[email-masked]>
Subject: Re: draft-ietf-idr-bgp-analysis-03.txt
Sender: [email-masked]

Curtis:
    Thanks for your comments. We will incorporate them in next revision.
-Keyur

Curtis Villamizar wrote:

>David, Keyur,
>
>I have some suggestions for improvement to this draft ( BGP-4 Protocol
>Analysis <draft-ietf-idr-bgp-analysis-03.txt>).  See comments inline
>below.  Feel free to take what you consider valid and worth changing.
>
>Curtis
>
>
>In "1. Introduction" you should mention that BGP4 was the first to
>support CIDR and due to their lack of support for CIDR versions 1-3
>are considered obsolete and unusable in today's Internet.
>
>------------
>
>Somewhere in key features it should be mentioned that BGP makes the
>assumption that packets are routed from source towards destination
>independent of the source.  A good place for this would be near the
>statement "BGP does not make any assumptions about intra-autonomous
>system routing protocols deployed within the various autonomous
>systems".  Or refer to the statement in the beginning of
>"7. Applicability".
>
>------------
>
>In the following paragraph, why don't we just say that this algorithm
>is referred to as a "Path Vector" algorithm.
>
>   BGP uses an algorithm that is neither a pure distance vector
>   algorithm or a pure link state algorithm. It is instead a modified
>   distance vector algorithm that uses path information to avoid
>   traditional distance vector problems. Each route within BGP pairs
>   destination with path information to that destination. Path
>   information (also known as AS_PATH information) is stored within the
>   AS_PATH attribute in BGP. This allows BGP to reconstruct large
>   portions of overall topology whenever required.
>
>------------
>
>Alex probably made you put in some FSM stuff (don't read too much into
>the wording here).  I don't think it belongs in this document.  That
>is clearly something for the protocol spec.
>
>------------
>
>In the section "4. BGP Persistent Peer Oscillations" or in a nearby
>section (preferable) it should be mentioned that BGP is work
>conserving.  Here is some suggested text:
>
>   A robust BGP implementation is work conserving.  This means that if
>   the number of prefixes is bound, arbitrarily high levels of route
>   change can be tolerated with bounded impact on route convergence
>   for occasionaly changes in generally stable routes.
>
>   A BGP implementation under high load conditions should empty as
>   much inbound routing updates from its input streams, processing
>   only the most recent route if the route for a given NLRI changes
>   multiple times.  TCP also provides blocking on the writes on the
>   sender side.  A BGP implementation under load should expect blocks
>   on write calls and send only the most recent routes when sockets
>   unblock rather than sending entire history.  
>
>   A robust implemention of BGP should have the following
>   characteristics:
>
>      1.  It is able to operate in almost arbitrarily high levels of
>	  route flap without loosing peerings (failing to send
>	  keepalives) or loosing other protocol adjacencies as a
>	  result of BGP load.
>
>      2.  Instability of a subset of routes should not affect the
>          route advertisements or forwarding associated with the set
>          of stable routes.
>
>      3.  High levels of instability and peers of different CPU speed
>          or load resulting in faster or slower processing of routes
>          should not cause instability and should have a bounded
>          impact on the convergence time for generally stable routes.
>
>   Numerous robust BGP implementations exist.  Producing a robust
>   implementation is not a trivial matter but clearly acheivable.
>
>------------
>
>I find the following paragraph problematic without further
>explanation.
>
>   It is important to note that BGP does not require all the routers
>   within an autonomous system to participate in the BGP protocol. In
>   particular, only the border routers that provide connectivity between
>   the local autonomous system and their adjacent autonomous systems
>   need participate in BGP. The ability to constrain the set of BGP
>   speakers is one way to address scaling issues.
>
>Either you need to default to the borders and exit at any border or
>you need some mechanism to tunnel between border routers for a pure
>transport network.  I favor removing the above paragraph.  Tunnelling
>to remove BGP is out of scope.  Default routing to reach any arbitrary
>border need not be mentioned.  Things which actually do improve
>scaling within an AS are RR and confeds.
>
>------------
>
>Section "5.1 Link bandwidth and CPU utilization" may still be overly
>simplistic and as a result may be incorrect.  See comments below.
>
>
>------------
>
>In terms of bandwidth, the number of unique AS paths in practice is a
>small number compared to the number of NLRI.  Since many NLRI are
>packed in a single update with the AS path included only once, in
>practice the number of NLRI completely dominates the amount of
>bandwidth consumed.
>
>The MR = 4 * (N + (M * A)) may be inaccurate for the reasons I gave in
>the prior paragraph.  The M*A may drasticly understate the impact of
>the unique AS paths.  Instead of defining A as the number of AS, A
>could be defined as the number of unique AS paths with M*A then being
>average AS path length times number of unique AS paths.  
>
>Also why is both memory and bandwidth represented as MR?  Wouldn't BW
>be a better variable name for bandwidth?
>
>The O(C * M) thing in the next paragraph is also invalid above some
>value of C but for different reasons.  Above some value of C, either
>the sender will begin pacing it sending of updates on its own
>(suppressing multiple changes over very short periods) or the receiver
>will be unable to keep up with the rate of change and force
>suppression of multiple changes over very short periods by causing the
>BGP socket to block on the sender.
>
>------------
>
>The following statement is incorrect:
>
>   Finally, since the dynamic properties of the network cannot be
>   quantitatively bounded, stability must be addressed via heuristics
>   such as BGP Route Flap Damping [RFC2439]. Due to the nature of BGP,
>   such damping should be viewed as a matter local to an autonomous
>   system matter (see also Appendix F.2 of [BGP4]).
>
>The amount of change is inherently bounded in BGP (as I described
>above).  BGP Route Flap Damping was initially proposed for two
>reasons, 1) to protect a specific commercial implementation that was
>not sufficiently robust, 2) to improve convergence of stable routes.
>BGP Route Flap Damping is not necessary to bound the amount of change
>in BPG routing.
>
>------------
>
>We can drop the following comparison to a historic protocol:
>
>   It may also be instructive to compare bandwidth and CPU requirements
>   of BGP with the Exterior Gateway Protocol (EGP). While with BGP the
>   complete information is exchanged only at the connection
>   establishment time, with EGP the complete information is exchanged
>   periodically (usually every 3 minutes). Note that both for BGP and
>   for EGP the amount of information exchanged is roughly on the order
>   of the number of networks reachable via a peer that sends the
>   information. Therefore, even if one assumes extreme  instabilities of
>   BGP, its worst case behavior will be the same as the steady state
>   behavior of its predecessor, EGP.
>
>   Operational experience with BGP showed that the incremental update
>   approach employed by BGP provides qualitative improvement in both
>   bandwidth and CPU utilization when compared with complete periodic
>   updates used by EGP (see also presentation by Dennis Ferguson at the
>   Twentieth IETF, March 11-15, 1991, St. Louis).
>
>We should drop other references to EGP.
>
>------------
>
>In "5.1.2. Memory requirements", the MR = O((N + M * A) * K) the same
>comment applies regarding the M*A term.  A should be unique AS paths,
>not number of AS and is not multiplied by K.  In practice the K term
>is small because it is the number of peers sending full routing, which
>is generally much less than the worst case number of peers.  Large
>providers who carry full routing typically send each other only their
>customer routes to avoid providing free transit to each other.  This
>reduces the impact of K.
>
>------------
>
>We can drop:
>
>   It is interesting to note that prior to the introduction of BGP in
>   the NSFNET Backbone, memory requirements on the NSFNET Backbone
>   routers running EGP were on the order of O(N *K).
>
>------------
>
>In the MR = ((N*4) + (M*A)*2) * K, we can make this quite accurate by
>defining N as the average number of routes advertised by each peer and
>A as the number of unique AS paths, moving (M*A)*2) outside of the *K
>and changing N*4 to N*R and (M*A)*2) to (M*A)*P) where R is the number
>of bytes required to store a route and P is the number of bytes needed
>to store one AS in an AS path.  If K is small, then some overhead such
>as the patricia trie storage figures into R and if K is large, the
>data structures may be linked lists off the patricia trie.  Claiming
>that a route can be stored in 4 bytes is rather naive.
>
>The N*R*K term in practice dominates over the (M*A)*P term.  If we
>conservatively estimate a route as taking 16 bytes (large K allowing
>patricia trie overhead to be ignored, indices or pointers to unique AS
>Path or other attributes, etc).  If we wanted to include a term to
>make things accurate for small K we could add U*X where U is the
>number of unique NRLI and X is the overhead per unique NLRI (and I ran
>out of useful single letters).  Typically X is greater than R.
>
>------------
>
>   Interestingly, in his review of the BGP protocol for the BGP review
>   committee in March of 1990, Paul Tsuchiya noted that "BGP does not
>   scale well.
>
>Paul was wrong.  It does scale well and that's why it is being used.
>BGP is a solution that scales as well as the problem allows and no
>better.
>
>------------
>
>In "10. Security Considerations" we can reference the separate
>security analysis document.  Or maybe not.
>
>
>  
>

To: "Joris Dobbelsteen" <[email-masked]>
Cc: "'IDR WG (E-mail)'" <[email-masked]>
Reply-To: [email-masked]
Subject: Re: Issue 19) Security Considerations 
Date: Wed, 14 May 2003 11:08:04 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, "Joris Dobbelsteen" 
writes:
> Curtis, it was not my intension to upset you in any way. Still it looks
> like I successed doing that....... Any way, we do need some feedback
> from practical deployments.

I intended to respond to this message after clearing a few things up,
but it got deferred for much longer than it should have.  No offense
taken.  None intended either.

> My personal consideration would be not to put it in a chapter 4.2 but
> get it throughout the entire draft.

That would be fine.

> Packet filters provide the good protection for IBGP sessions. These are
> harder/impractical for EBGP(?).
> Internal attacks and hacks on the network channels are not considered
> practical.

The reason that packet filters are hard to get deployed on EBGP is the
attack can come from your peer and you have no control over your
peer's router, except to request that filters be put in place.

If an internal router is compromised there are many attacks possible
in addition to BGP.  The same applies to an internal compromise to
other critical infrastructure machines such as NMS.

> IPSec is not recommended because at least three reasons:
> TCP-MD5 is already widely implemented and deployed.
> Complication in network management, because packet filters need to be
> adapted for IPSec traffic (port 500 - IKE).
> IPSec has higher processing demands that TCP-MD5 does, lowing the
> barrier for a successful DoS attack. Hardware IPSec implementations are
> considered impractical, due to the cost or performance.
> 
> This brings up your statement that IPSec covers the ports. This is not
> true, when encryption (ESP) is not used. The use of intigrity (AH)
> will not hide the ports, it works sorta TCP-MD5, if you want...
> I also did never consider using ESP, since it was not desired and it
> would be the way to 'promote' DoS attacks.
> Unfortunally I don't have any insight on IPSec hardware, maybe some else
> can give some insight on this.
> 
> I suppose this closes the IPSec/TCP-MD5 issue, leaving TCP-MD5 to the 
> current best option.

Use of AH is an option.  It offers little benefit over TCP-MD5.  It
would be better if there was a way for ESP to expose the ports.

> Still this leaves EBGP open for consideration.
> 
> The use of BGP TTL Security Hack (BTSH) might be a very simple and
> effective way of protecting the external routers from attacks,
> especially DoS attacks. Malicious data can be prevented using TCP-MD5.
> The disadvantage is that it is not deployed widely, but only requires
> routers to send there traffic with an initial TTL of 255. I don't
> know if this is currently possible to archieve: whether current
> implementations can be set to send with an initial TTL of 255. It is
> not needed for both end-points to support BTSH, although desired.
> <draft-gill-btsh-01.txt>

It is simple.  It is easy.  It works.  This TTL check can be done in
hardware in many (most?) routers.

Another mitigation of attack is to build routers such that the BGP TCP
(and optionally the whole Adj-In and much of the AdjOut handling) is
done on the line card, minimally the TCP-MD5 authentication.  Some
routers also include internal hardware queues that can be used for SFQ
handling of traffic to the line card from outside.  If an attack
occurs on an EBGP peer, the attack only affects peers served by that
line card and if SFQ is available may only affect that peer.  If so, a
peers that do not implement (or enable) filtering or BTSH affects only
their own connectivity.

> I couldn't find any information about dynamic 4-tuple filtering
> (google turns up BGP dynamic capabilities), unfortunally.

Vijay Gill gave a presentation at NANOG.  Once established a 4-tuple
filter (src/dst addr+port) is installed for a BGP peering giving that
traffic priority over other 4-tuples.  A DoS attack on the intigrity
check then impacts the ability to get to the established state but
does not impact established connections.

> So what else can be done to prevent a EBGP session from attacks,
> especially DoS attacks? Data insertion/manipulations can be guarded
> against using TCP-MD5 (or similar).

Traffic based DoS which overwhelms the intigrity check is the hardest
problem we currently face.  It is solved for IBGP by adequate
filtering and keeping the routers or other infrastructure machines
from being compromised.  BTSH solves it for EBGP unless the peer is
compromised.  Limiting the impact of a full bandwidth attack from a
peer is something that a limited set of routers may be capable of.

> - Joris

Sorry for the delay.  I deferred a response to this and then it got
lost/buried in my inbox.

Curtis

> >-----Original Message-----
> >From: [email-masked] [mailto:[email-masked]]On Behalf Of
> >Curtis Villamizar
> >Sent: Thursday, 3 April 2003 16:36
> >To: Joris Dobbelsteen
> >Cc: 'IDR WG (E-mail)'
> >Subject: Re: Issue 19) Security Considerations 
> >
> >In message <[email-masked]>, 
> >"Joris Dobbelsteen" 
> >writes:
> >> [snip]
> >
> >Joris,
> >
> >Quite frankly I'm outraged at such comments.  Only by looking at
> >theoretical security issues and ignoring reality (not that the two
> >don't highly but imperfectly overlap) can you come to the conclusion
> >that IPSEC is needed and in its current form is viable as a security
> >solution for BGP.
> >
> >I think its about time we injected some reality into
> >draft-murphy-bgp-vuln-02.txt.
> >
> >I've added a practical considerations section.  I stuck it in as 4.2.
> >
> >Comments are welcome, particularly comments from people actually
> >running BGP networks or building BGP routers used by ISPs.
> >
> >I did not mention advanced filtering works-in-progress or proposals
> >such as BTSH or dynamic 4-tuple EBGP filtering since these are not yet
> >implemented or deployed afaik.  [aside: I strongly believe that BTSH
> >will prove to be a viable to protect EBGP and a preferable replacement
> >for current filtering which some older TTM (time-to-market) line cards
> >still in use are unable to support.]
> >
> >I should also note that the filtering best practices are far from
> >universally deployed and in some cases are difficult to fully deploy
> >due to residual use of TTM line cards unable to support filtering.
> >
> >Note that IPSEC with port numbers exposed would be a viable security
> >solution.  It would still be a greater computational burden than
> >TCP-MD5 and still might be less preferred by ISPs for that reason for
> >some architectures.  This change to IPSEC would at least yield two
> >viable options and might encourage implementation and deployment of
> >IPSEC as a security solution for BGP.
> >
> >Curtis
> >
> >
> >--- draft-murphy-bgp-vuln-02.txt	Wed Mar  5 21:00:00 2003
> >+++ draft-murphy-bgp-vuln-02.txt++	Thu Apr  3 09:18:12 2003
> >@@ -149,6 +149,7 @@
> > 3.2.2.2 Timer events 
> >..............................................   16
> > 4 Security Considerations 
> >.........................................   16
> > 4.1 Residual Risk 
> >.................................................   16
> >+4.2 Practical Considerations 
> >......................................   16
> > 5 References 
> >......................................................   17
> > 6 Author's Address 
> >................................................   18
> > 
> >@@ -901,6 +902,79 @@
> > Filtering is in use near some customer attachment points, but is not
> > effective near the Internet center.  The other mechanisms are still
> > controversial and are not yet in common use.
> >+
> >+4.2 Practical Considerations
> >+
> >+The primary usage of BGP is as a means to provide reachability
> >+information to Autonomous Systems (AS) and to distribute external
> >+reachability internally within an AS.  BGP is the routing protocol
> >+used to distribute global routing information in the Internet.  BGP is
> >+therefore used by all major Internet Service Providers (ISP) and many
> >+smaller providers and other organizations.  
> >+
> >+The role which BGP plays in the Internet puts BGP implementations in
> >+unique conditions and places unique security requirements on BGP.  BGP
> >+is operated over interprovider interfaces in which traffic levels push
> >+the state of the art in specialized packet forwarding hardware and
> >+exceed the performace capabilities of hardware implementation of
> >+decryption by many decimal orders of magnitude.  
> >+
> >+ISP networks must be and are under tight control.  The only viable
> >+means to protect the network elements from Denial of Service (DoS)
> >+attacks under such conditions are packet based filtering techniques
> >+based on relatively simple inspections of packets.
> >+
> >+To protect Internal BGP (IBGP) sessions, filters are applied at all
> >+borders to an ISP network which remove all traffic destined for
> >+addresses of network elements internal addresses (typically contained
> >+within a single prefix) and the BGP port number (179).  Packets from
> >+within an ISP are not forwarded from an internal interface to the BGP
> >+speaker's address on which External BGP (EBGP) sessions are supported,
> >+or to a peer's EBGP address if the BGP port number is found.  With
> >+appropriate consideration in router design, in the event of failure of
> >+a BGP peer to provide the equivalent filtering the risk of compromise
> >+can be limited to the peering session on which filtering is not
> >+performed by the peer or the interface or line card on which the
> >+peering is supported.  There is substantial motivation and little
> >+effort for ISPs to maintain such filters.
> >+
> >+Being composed entirely of specialized network equipment, under strict
> >+control of the ISP, the ISP network is not subject to attacks from
> >+within than enterprise networks are with more generalized computing
> >+systems and staff less carefully trained in the area of secure
> >+procedures.  ...
> 
> For me personally, the above sentence is a little hard to understand.
> You mean that ???
> The Internal BGP routers (or specialized network equipment) that is under
> strict control of the ISP is not subject to attacks, other than those that
> are common in enterprise networks. These networks have staff that is
> less carefully trained in the area of security procedures.
> 
> >+ ...........  Monitoring of traffic from within requires either
> >+compromise of relatively physically secure and carefully administered
> >+network elements or monitoring physical media.  Injection of traffic
> >+requires either compromise of network elements or intercept and
> >+replacement of traffic on physical media.
> >+
> >+The difficulty of compromise of network elements and of undetected
> >+tapping into physical media carrying extremely high volumes of traffic
> >+is much greater than the difficulty of injecting sufficient traffic
> >+from outside a network to effect a DoS attack.  As a result, the
> >+ability to packet filter on the basis of port numbers far exceeds the
> >+need to cryptographic strength in encapsulation.
> >+
> >+These practical considerations yield the situation in which TCP-MD5,
> >+though cryptographic weak, far better serves ISP security needs than
> >+the cryptographicly much stronger IPSEC which makes packet filtering
> >+infeasible.
> >+
> >+Use of BGP in smaller networks yields similar requirements.  The
> >+capability of a single workstation with high speed interface to
> >+generate false traffic far exceeds the capability of software based
> >+decryption or appropriately priced cryptographic hardware.  From a
> >+practical standpoint, these networks are also better served by
> >+appropriate administrative care, filtering, and TCP-MD5 than by IPSEC.
> >+
> >+This situation is likely to persist unless either cryptographic
> >+hardware becomes many orders of magnitude faster and cheaper or IPSEC
> >+supports an ability to leave IP port numbers exposed.  This
> >+requirement has been made known to the IPSEC WG.
> >+
> 
> See above, using intigrity only (AH), leaves TCP (not IP) port numbers
> readable for everyone. This security is rather an IPSec configuration
> option.
> 
> >+Until such time as IPSEC is modified, there is little choice but to
> >+mandate TCP-MD5 implementation and recommend TCP-MD5 usage for BGP and
> >+discourage IPSEC usage for BGP.
> > 
> > 5.  References
> > 
> >
> 

To: [email-masked]
Cc: [email-masked]
Reply-To: [email-masked]
Subject: draft-ietf-idr-bgp-analysis-03.txt
Date: Wed, 14 May 2003 10:08:40 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

David, Keyur,

I have some suggestions for improvement to this draft ( BGP-4 Protocol
Analysis <draft-ietf-idr-bgp-analysis-03.txt>).  See comments inline
below.  Feel free to take what you consider valid and worth changing.

Curtis

In "1. Introduction" you should mention that BGP4 was the first to
support CIDR and due to their lack of support for CIDR versions 1-3
are considered obsolete and unusable in today's Internet.

------------

Somewhere in key features it should be mentioned that BGP makes the
assumption that packets are routed from source towards destination
independent of the source.  A good place for this would be near the
statement "BGP does not make any assumptions about intra-autonomous
system routing protocols deployed within the various autonomous
systems".  Or refer to the statement in the beginning of
"7. Applicability".

------------

In the following paragraph, why don't we just say that this algorithm
is referred to as a "Path Vector" algorithm.

   BGP uses an algorithm that is neither a pure distance vector
   algorithm or a pure link state algorithm. It is instead a modified
   distance vector algorithm that uses path information to avoid
   traditional distance vector problems. Each route within BGP pairs
   destination with path information to that destination. Path
   information (also known as AS_PATH information) is stored within the
   AS_PATH attribute in BGP. This allows BGP to reconstruct large
   portions of overall topology whenever required.

------------

Alex probably made you put in some FSM stuff (don't read too much into
the wording here).  I don't think it belongs in this document.  That
is clearly something for the protocol spec.

------------

In the section "4. BGP Persistent Peer Oscillations" or in a nearby
section (preferable) it should be mentioned that BGP is work
conserving.  Here is some suggested text:

   A robust BGP implementation is work conserving.  This means that if
   the number of prefixes is bound, arbitrarily high levels of route
   change can be tolerated with bounded impact on route convergence
   for occasionaly changes in generally stable routes.

   A BGP implementation under high load conditions should empty as
   much inbound routing updates from its input streams, processing
   only the most recent route if the route for a given NLRI changes
   multiple times.  TCP also provides blocking on the writes on the
   sender side.  A BGP implementation under load should expect blocks
   on write calls and send only the most recent routes when sockets
   unblock rather than sending entire history.  

   A robust implemention of BGP should have the following
   characteristics:

      1.  It is able to operate in almost arbitrarily high levels of
	  route flap without loosing peerings (failing to send
	  keepalives) or loosing other protocol adjacencies as a
	  result of BGP load.

      2.  Instability of a subset of routes should not affect the
          route advertisements or forwarding associated with the set
          of stable routes.

      3.  High levels of instability and peers of different CPU speed
          or load resulting in faster or slower processing of routes
          should not cause instability and should have a bounded
          impact on the convergence time for generally stable routes.

   Numerous robust BGP implementations exist.  Producing a robust
   implementation is not a trivial matter but clearly acheivable.

------------

I find the following paragraph problematic without further
explanation.

   It is important to note that BGP does not require all the routers
   within an autonomous system to participate in the BGP protocol. In
   particular, only the border routers that provide connectivity between
   the local autonomous system and their adjacent autonomous systems
   need participate in BGP. The ability to constrain the set of BGP
   speakers is one way to address scaling issues.

Either you need to default to the borders and exit at any border or
you need some mechanism to tunnel between border routers for a pure
transport network.  I favor removing the above paragraph.  Tunnelling
to remove BGP is out of scope.  Default routing to reach any arbitrary
border need not be mentioned.  Things which actually do improve
scaling within an AS are RR and confeds.

------------

Section "5.1 Link bandwidth and CPU utilization" may still be overly
simplistic and as a result may be incorrect.  See comments below.

------------

In terms of bandwidth, the number of unique AS paths in practice is a
small number compared to the number of NLRI.  Since many NLRI are
packed in a single update with the AS path included only once, in
practice the number of NLRI completely dominates the amount of
bandwidth consumed.

The MR = 4 * (N + (M * A)) may be inaccurate for the reasons I gave in
the prior paragraph.  The M*A may drasticly understate the impact of
the unique AS paths.  Instead of defining A as the number of AS, A
could be defined as the number of unique AS paths with M*A then being
average AS path length times number of unique AS paths.  

Also why is both memory and bandwidth represented as MR?  Wouldn't BW
be a better variable name for bandwidth?

The O(C * M) thing in the next paragraph is also invalid above some
value of C but for different reasons.  Above some value of C, either
the sender will begin pacing it sending of updates on its own
(suppressing multiple changes over very short periods) or the receiver
will be unable to keep up with the rate of change and force
suppression of multiple changes over very short periods by causing the
BGP socket to block on the sender.

------------

The following statement is incorrect:

   Finally, since the dynamic properties of the network cannot be
   quantitatively bounded, stability must be addressed via heuristics
   such as BGP Route Flap Damping [RFC2439]. Due to the nature of BGP,
   such damping should be viewed as a matter local to an autonomous
   system matter (see also Appendix F.2 of [BGP4]).

The amount of change is inherently bounded in BGP (as I described
above).  BGP Route Flap Damping was initially proposed for two
reasons, 1) to protect a specific commercial implementation that was
not sufficiently robust, 2) to improve convergence of stable routes.
BGP Route Flap Damping is not necessary to bound the amount of change
in BPG routing.

------------

We can drop the following comparison to a historic protocol:

   It may also be instructive to compare bandwidth and CPU requirements
   of BGP with the Exterior Gateway Protocol (EGP). While with BGP the
   complete information is exchanged only at the connection
   establishment time, with EGP the complete information is exchanged
   periodically (usually every 3 minutes). Note that both for BGP and
   for EGP the amount of information exchanged is roughly on the order
   of the number of networks reachable via a peer that sends the
   information. Therefore, even if one assumes extreme  instabilities of
   BGP, its worst case behavior will be the same as the steady state
   behavior of its predecessor, EGP.

   Operational experience with BGP showed that the incremental update
   approach employed by BGP provides qualitative improvement in both
   bandwidth and CPU utilization when compared with complete periodic
   updates used by EGP (see also presentation by Dennis Ferguson at the
   Twentieth IETF, March 11-15, 1991, St. Louis).

We should drop other references to EGP.

------------

In "5.1.2. Memory requirements", the MR = O((N + M * A) * K) the same
comment applies regarding the M*A term.  A should be unique AS paths,
not number of AS and is not multiplied by K.  In practice the K term
is small because it is the number of peers sending full routing, which
is generally much less than the worst case number of peers.  Large
providers who carry full routing typically send each other only their
customer routes to avoid providing free transit to each other.  This
reduces the impact of K.

------------

We can drop:

   It is interesting to note that prior to the introduction of BGP in
   the NSFNET Backbone, memory requirements on the NSFNET Backbone
   routers running EGP were on the order of O(N *K).

------------

In the MR = ((N*4) + (M*A)*2) * K, we can make this quite accurate by
defining N as the average number of routes advertised by each peer and
A as the number of unique AS paths, moving (M*A)*2) outside of the *K
and changing N*4 to N*R and (M*A)*2) to (M*A)*P) where R is the number
of bytes required to store a route and P is the number of bytes needed
to store one AS in an AS path.  If K is small, then some overhead such
as the patricia trie storage figures into R and if K is large, the
data structures may be linked lists off the patricia trie.  Claiming
that a route can be stored in 4 bytes is rather naive.

The N*R*K term in practice dominates over the (M*A)*P term.  If we
conservatively estimate a route as taking 16 bytes (large K allowing
patricia trie overhead to be ignored, indices or pointers to unique AS
Path or other attributes, etc).  If we wanted to include a term to
make things accurate for small K we could add U*X where U is the
number of unique NRLI and X is the overhead per unique NLRI (and I ran
out of useful single letters).  Typically X is greater than R.

------------

   Interestingly, in his review of the BGP protocol for the BGP review
   committee in March of 1990, Paul Tsuchiya noted that "BGP does not
   scale well.

Paul was wrong.  It does scale well and that's why it is being used.
BGP is a solution that scales as well as the problem allows and no
better.

------------

In "10. Security Considerations" we can reference the separate
security analysis document.  Or maybe not.

To: IETF-Announce: ;
Cc: [email-masked]
From: [email-masked]
Reply-To: [email-masked]
Subject: I-D ACTION:draft-ietf-idr-bgp-analysis-03.txt
Date: Wed, 14 May 2003 07:20:42 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Inter-Domain Routing Working Group of the IETF.

	Title		: BGP-4 Protocol Analysis
	Author(s)	: D. Meyer, K. Patel
	Filename	: draft-ietf-idr-bgp-analysis-03.txt
	Pages		: 19
	Date		: 2003-5-13
	
The purpose of this report is to document how the requirements for
advancing a routing protocol from Draft Standard to full Standard
have been satisfied by Border Gateway Protocol version 4 (BGP-4).
This report satisfies the requirement for'the second report', as
described in Section 6.0 of RFC 1264 [RFC1264].  In order to fulfill
the requirement, this report augments RFC 1774 [RFC1774] and
summarizes the key features of BGP protocol, and analyzes the
protocol with respect to scaling and performance.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp-analysis-03.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-idr-bgp-analysis-03.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-idr-bgp-analysis-03.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-idr-bgp-analysis-03.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-idr-bgp-analysis-03.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Tue, 13 May 2003 09:15:20 +0200
To: Jeffrey Haas <[email-masked]>
From: "John G. Scudder" <[email-masked]>
Subject: Re: On BGP and VPLS
Cc: Eric Rosen <[email-masked]>, [email-masked]
Sender: [email-masked]

At 2:25 PM -0400 5/12/03, Jeffrey Haas wrote:
>If you flood more than one type
>of reachability, you get to make hard choices such as which reachability
>do you want to converge faster and if you start exceeding the resource
>bounds of your router, which information do you toss?

OK, but this is orthogonal to whether you carry said information in 
BGP or some other protocol, since they're all sharing the same 
resources.

By the way, I think this use of the term "flooding" is rather 
unfortunate since it already has a well understood meaning in the 
routing protocol context, and it suggests link-state.

--John

Date: Mon, 12 May 2003 14:25:46 -0400
From: Jeffrey Haas <[email-masked]>
To: Eric Rosen <[email-masked]>
Cc: [email-masked]
Subject: Re: On BGP and VPLS
Sender: [email-masked]

Aside from noting that sctp would be a fine way to do the multiple
streams, I think all of my issues have relatively little to do with
technical issues and are really implementation/operational ones.

If its reachability you're flooding, and the information changes
on a hop-by-hop basis, BGP is fine.  If you flood more than one type
of reachability, you get to make hard choices such as which reachability
do you want to converge faster and if you start exceeding the resource
bounds of your router, which information do you toss?

The more that you cram into one package, the more rope you give people.
Feedback from several network operators often makes me think we've already
given them too many chain-lengths of rope. :-)

On Mon, May 12, 2003 at 02:08:41PM -0400, Eric Rosen wrote:
> I think a vendor would be unlikely  to "start from the spec".  A more likely
> implementation strategy  would be to allow  BGP connections on  both the old
> port and the  new port.  Capability advertisement or  ORF or something would
> be  used  to  choose  which  kind  of  info  gets  forwarded  on  which  BGP
> connections.  
> 
> Of course,  one day  someone would notice  that this might  require multiple
> parallel TCP connections  where a single one might do just  as well.  So the
> suggestion would probably be made that,  as an optimization, one could use a
> single port,  but encode the  different kinds of  data as NLRI  of different
> address families.
> 
> So saying "use a different port" doesn't really make the problem go away. 

-- 
Jeff Haas 
NextHop Technologies

To: Jeffrey Haas <[email-masked]>
Cc: Pedro Roque Marques <[email-masked]>, [email-masked]
Subject: Re: On BGP and VPLS 
Reply-To: [email-masked]
Date: Mon, 12 May 2003 14:08:41 -0400
From: Eric Rosen <[email-masked]>
Sender: [email-masked]

Jeff> But  it'd be  nice for  everyone that  wants to  "co-opt  just another
Jeff> little piece of BGP because it works" that maybe you should start from
Jeff> the spec, get your own TCP port and flood the stuff in parallel.

I think a vendor would be unlikely  to "start from the spec".  A more likely
implementation strategy  would be to allow  BGP connections on  both the old
port and the  new port.  Capability advertisement or  ORF or something would
be  used  to  choose  which  kind  of  info  gets  forwarded  on  which  BGP
connections.  

Of course,  one day  someone would notice  that this might  require multiple
parallel TCP connections  where a single one might do just  as well.  So the
suggestion would probably be made that,  as an optimization, one could use a
single port,  but encode the  different kinds of  data as NLRI  of different
address families.

So saying "use a different port" doesn't really make the problem go away. 

Date: Mon, 12 May 2003 09:56:37 -0400
From: Jeffrey Haas <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked]
Subject: Re: On BGP and VPLS
Sender: [email-masked]

On Wed, May 07, 2003 at 02:15:39PM -0700, Pedro Roque Marques wrote:
> My argument is that the flooding algorithm in the BGP spec is
> applicable to other NLRI-types.

You could flood it using NNTP too.  It doesn't mean its the One True
Answer.  I think that is most of Alex's point.

> And that the document can be, and has
> been sucesfully used to do so. i.e. it is still a coherent document
> when you interpret it in the context of a different NLRI-type.

There comes a time when you're just distributing too much stuff
in one protocol and take too many chances at destabilizing what
sort of works well.

I'm not saying that VPLS is like this, having never read the specs.
But it'd be nice for everyone that wants to "co-opt just another little
piece of BGP because it works" that maybe you should start from
the spec, get your own TCP port and flood the stuff in parallel.

Same kind of basket, just different basket.  Hopefully fewer broken
eggs.

Now I know what the DNS folk feel like.

>   Pedro.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 9 May 2003 10:13:14 -0400 (Eastern Daylight Time)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Comments on BGP Draft 20.....
Sender: [email-masked]

Some of these are going to echo Alex's comments, but that's okay, I think.
Mostly just nits....

:-)

Russ

__________________________________
[email-masked] CCIE <>< Grace Alone

-----

Abstract:

This information is sufficeint to construct a graph of the AS connectivity
from which routing loops may be pruned and some policy decisions at the AS
level may be enforced.

UPDATE Message Format:

The information in the UPDATE message can be used to construct a graph
describing the relationships of the various Autonomous Systems.

In both cases this is true, I suppose, but in neither case does this really
describe what the AS Path is used for, right? I would think we'd want to
describe it less in terms of a "graph of the connectivity in the
internetwork," and more in terms of "a graph of the path through Autonomous
Systems ued to reach the destination advertised." It could be confusing,
since there isn't anyplace where we discuss building a graph of
inconnectivity between the Autonomous Systems....

-----

Forwarding Paradigm:

This document uses the term "Autonomous System" (AS)  throughout....

This entire paragraph is a repeat--I'd leave it just in the definitions.

-----

Forwarding Paradigms:

The initial data flow....

This paragraph has two different thoughts in it, one about incremental
updates, and the other about keeping data that you've received. It seems
like just putting a return after "as the routing tables change."

-----

Forwarding Paradigms:

The paragraph starting "KEEPALIVE messages" should, I think, be moved up
above the section on route exchange. I don't know why, it just seems less
like it's jumping all over the place that way.

-----

3.1 Routes: Advertisement and Storage

It almost seems like the section about The initial data flow should maybe
be put entirely under this section someplace (?).

The first paragraph in this section is really a definition of a route vs a
prefix, and should probably be in the definitions.

The paragraph "Changing attribute of a route...." needs a "the," or
attribute needs an "s."

-----

3.2 Routing Information Bases

b) Loc-RIB....

I think it might be useful to state the contents of the Loc-RIB are
actually installed in the local routing table, and thus used for forwarding
packets on this router. I don't see anyplace this connection is made
explicit, it seems more like it's implicit throughout the doc.

-----

Page 18, a) LOCAL_PREF

"....to inform other peers...." should be "....to inform its other
peers...."

-----

Network Layer Reachability Information

"This varibale length field contains a list of IP address prefixes."

I think we can kill "address" here.

a) Length

"The Length field inidicates...." The sentence can start with
"Indicates..."

b) Prefix

"The Prefix field indicates...." The sentence can start with
"Indicates...."

-----

Network Layer Reachability Information

"An UPDATE message can list multiple routes to be withdrawn...."

Actually, we don't withdraw routes, we withdraw prefixes, right? The next
paragraph shows this confusion, by talking about routes without attributes,
but routes are prefixes combined with attributes, so.... They aren't
routes, they're prefixes. You remove routes based on withdrawn prefixes, I
think.

------

5. Path Attributes

"Well-known attributes MUST be recognized by all BGP implementations."

This sentence, as strange as it may sound, implies it's the attributes
fault if the BGP implementation doesn't recogonize it, that it's up to the
attribute definers to, in some way, make certain that BGP implementations
will recognize it. I think it should probably be worded the other way
'round:

"BGP implementations MUST recognize all well-known attributes."

-----

5. Path Attributes

"All well-known attributes MUST be passed along (after proper updating, if
necessary) to other BGP peers."

This just seems a little rough. Maybe this:

"Once a BGP peer has updated any well-known attributes, it MUST pass these
attributes in any updates it transmits to its peers."

-----

5.1.1 ORIGIN

"Its value SHOULD NOT be changed by any other speaker."

I really think this should be "MUST NOT." I can't think of any reason it
wouldn't be, except in the case of aggregation, and that case could be
mentioned here as the only known exception (?).

To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: On BGP and VPLS 
Date: Wed, 07 May 2003 17:45:48 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Pedro Roque Marqu
es writes:
> Curtis Villamizar writes:
> 
> > Pedro,
> 
> > This is the BGP4 base spec.  The interpretation of the BGP
> > advertisements are IP prefix aka NLRI for which routes are
> > advertised and not general keys mapping to general records.
> 
> > If some extension of BGP4 such as VPN or PW makes some other use of
> > BGP flooding then that's fine but need not be reflected in the base
> > document.
> 
> > The changes that you are only vaguely specifying (s/route/record/g
> > s/IP prefix/key/g) doesn't at all pertain to the use of BGP as
> > defined in the base spec, hasn't been interpreted as being in
> > conflict with any existing document, and I don't think this is a
> > productive discussion during last call of a very key document.
> 
> Curtis,
> In no way i was suggesting that we change the base spec.
> 
> My argument is that the flooding algorithm in the BGP spec is
> applicable to other NLRI-types. And that the document can be, and has
> been sucesfully used to do so. i.e. it is still a coherent document
> when you interpret it in the context of a different NLRI-type.
> 
> regards,
>   Pedro.

So you are not suggesting a change at all to BGP4.  If so you don't
need to involve the IDR WG in a semantic discussion.

If the issue is whether to use BGP4 for distribution of VPLS
information and the objection were along the lines of scaling, or some
other technical matter then that was not at all clear.  

Unless I missed something, IDR was added to the Cc mid discussion.  If
so, maybe you should tell us what draft you are discussing and be
clear about what the issue is.

Curtis

To: Pedro Roque Marques <[email-masked]>
Cc: Alex Zinin <[email-masked]>, [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: On BGP and VPLS 
Date: Wed, 07 May 2003 17:20:25 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Pedro Roque Marqu
es writes:
> Pedro Roque Marques writes:
> 
> >>> The way i see it there is an high likely-hood of this turning into
> >>> an "Yes, it is" "No, it isn't" discussion. And I'd really like to
> >>> avoid that.
> 
> >> Agreed.
> 
> Following up on my own e-mail... i don't think it is in any way
> productive to continue the discussion torwards this path.
> 
> Lets try to turn the discussion around to the positive side:
> 
> Ignore VPLS for now.
> 
> Lets define a problem:
> 
> 1. A database consisting on entries (key, attr) needs to be propagated
> accross routers of the same domain and accross different
> administrative domains.
> 
> 2. An given key may be originated by more than one of the
> participating systems.
> 
> 3. A key advertised via a given member of a domain depends on
> reachability to that advertiser.
> 
> Task at hand is to find a solution to the problem above.
> 
>   Pedro.

Pedro,

You are welcome to consider BGP for this key distribution even if the
BGP spec does not match the terminology you are looking for.  The
terminology would not be the deciding factor.  If there are technical
problems with whatever you propose, that is a separate matter.

Now please drop IDR from the Cc and go about doing the ppvpn work,
whether it is VPLS or selecting a mechanism for your hypothetical
key/attr distribution.

Curtis

Date: Wed, 7 May 2003 14:15:39 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: On BGP and VPLS 
Sender: [email-masked]

Curtis Villamizar writes:

> Pedro,

> This is the BGP4 base spec.  The interpretation of the BGP
> advertisements are IP prefix aka NLRI for which routes are
> advertised and not general keys mapping to general records.

> If some extension of BGP4 such as VPN or PW makes some other use of
> BGP flooding then that's fine but need not be reflected in the base
> document.

> The changes that you are only vaguely specifying (s/route/record/g
> s/IP prefix/key/g) doesn't at all pertain to the use of BGP as
> defined in the base spec, hasn't been interpreted as being in
> conflict with any existing document, and I don't think this is a
> productive discussion during last call of a very key document.

Curtis,
In no way i was suggesting that we change the base spec.

My argument is that the flooding algorithm in the BGP spec is
applicable to other NLRI-types. And that the document can be, and has
been sucesfully used to do so. i.e. it is still a coherent document
when you interpret it in the context of a different NLRI-type.

regards,
  Pedro.

To: Pedro Roque Marques <[email-masked]>
Cc: Alex Zinin <[email-masked]>, [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: On BGP and VPLS 
Date: Wed, 07 May 2003 17:06:04 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

Pedro,

This is the BGP4 base spec.  The interpretation of the BGP
advertisements are IP prefix aka NLRI for which routes are advertised
and not general keys mapping to general records.

If some extension of BGP4 such as VPN or PW makes some other use of
BGP flooding then that's fine but need not be reflected in the base
document.

The changes that you are only vaguely specifying (s/route/record/g
s/IP prefix/key/g) doesn't at all pertain to the use of BGP as defined
in the base spec, hasn't been interpreted as being in conflict with
any existing document, and I don't think this is a productive
discussion during last call of a very key document.

Curtis

Date: Wed, 7 May 2003 13:58:58 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: On BGP and VPLS
Sender: [email-masked]

Pedro Roque Marques writes:

>>> The way i see it there is an high likely-hood of this turning into
>>> an "Yes, it is" "No, it isn't" discussion. And I'd really like to
>>> avoid that.

>> Agreed.

Following up on my own e-mail... i don't think it is in any way
productive to continue the discussion torwards this path.

Lets try to turn the discussion around to the positive side:

Ignore VPLS for now.

Lets define a problem:

1. A database consisting on entries (key, attr) needs to be propagated
accross routers of the same domain and accross different
administrative domains.

2. An given key may be originated by more than one of the
participating systems.

3. A key advertised via a given member of a domain depends on
reachability to that advertiser.

Task at hand is to find a solution to the problem above.

  Pedro.

Date: Wed, 7 May 2003 13:30:24 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: On BGP and VPLS
Sender: [email-masked]

Alex Zinin writes:

> Pedro, Sorry for the delay. I found your message in my IDR folder
> just recently. I'll limit my involvement to this message, I think I
> will have said enough for my opinion to be heard :)

Alex,

>> The way i see it there is an high likely-hood of this turning into
>> an "Yes, it is" "No, it isn't" discussion. And I'd really like to
>> avoid that.

> Agreed.

It seems to me that we immediatly got into this dead-lock point. Allow
me to make a further attempt to clarify my point of view.

>> This point seems to be predicated in the statement that "BGP uses
>> the NLRI field to carry IP reachability"...

> Yes, plus that "BGP is an IP routing protocol."

That is one possible application of BGP. It does not invalidate that
possibility of other applications.

Let me rephrase my point:

It is possible to extract from the BGP specification a common set of
functionality that is independent of 'IP routing'. Such common set of
functionality consists on the ability to flood databases of
information accross multiple domains, in a distributed fashion.

Since this problem reoccurs often in networking, several other
applications have made use of this commonality to avoid reinventing
the wheel.

The 'common functionality' in question is not so much what is
documented in the BGP specification but the algorithms required to
implement it.

> This is where we disagree in the first place.

> There's a fine line between a database distribution flooding
> algorithm and a path-vector routing algorithm. BGP is clearly not
> the former.

"Yes, it is" :-)
Give me an argument and i'll try a more productive response.

>> As an exercise, if we take the existing spec and do:
>> s/route/record/g s/IP prefix/key/g

>> Do we still have a document that makes sense.. ?

> I'm not sure :)

>> Except for the vague bits about aggregation, about which BGP itself
>> does little about, i would contend that the result would be pretty
>> much the same.

> You are forgetting the parts about Loc-RIB, routing and forwarding
> tables, next-hop, etc. In any case, such a beast would seize to be
> an IP routing protocol, but would still perform best path selection
> to an opaque key in the Internet. If you need a database
> distribution mechanism as you described above, you don't need the
> path vector behavior, nor per-peer state for each key, nor
> next-hops. I.e., you don't need what BGP does.

You repeat this argument throughout your e-mail. And i'm completly
missing your point:

The path-vector algorithm is essential to the flooding of the BGP
database information. Without it flooding would eternally loop.

path-vector as in:
 o as-path and cluster-list loop detection
 o iBGP doesn't advertise iBGP

This is what makes BGP flooding work. None of this information is used
for Loc-rib purposes.

>>>  2. Distribution of information

[...]

>> P routers do not carry 2547 routing information.

> Unless the SPs want to use the existing RR infrastructure.

All the SPs i've worked with explicitly do not want to do this.

>> Not really... i can advertise the same key from multiple sources in
>> L2VPNs also. All policy mechanisms do work... igp distance, etc. It
>> is just the semantics once the path is selected that are different.
>> As an example think working and protect PE for a given emulated
>> circuit (or lan).

> I think you might have missed my point here. Though a BGP speaker
> will receive the same key from multiple sources, and will select the
> best path to it, in the VPLS case, it is not interested in the
> _best_ path, it is interested in only receiving the key, regardless
> of where it comes from.

That is factually incorrect. I gave you an example in the original
e-mail to illustrate the point. The same 'key' maybe advertised from
more than one source. BGP needs to select the best-path in any such
occurence.

> So RR and eBGP are the examples where a given BGP speaker would
> receive extra copies of information. As I replied to Mark:

>     Well, in the IP/BGP case, it is not necessarily the same info,
> as path attributes are likely to be different and the BGP speaker is
> interested in selecting the best among them while preserving others
> as the back-up. In the VPLS case, we don't need the best, just one
> copy is sufficient.

That is incorrect.

PE1 advertises 'key' preference 100, label 10000 - this is working
circuit.
PE2 advertises 'key' preference 200, label 10001 - this is protect
circuit.

It works just like IP routing. All the preference mechanisms that you
use in IP routing can be used here. IGP failure to PE1 for instance,
causes a remote system to automatically reroute.

> This does not change the fact that information that BGP as an IP
> routing protocol distributes is aggregateable.

That doesn't mean that BGP does aggregate anything either. BGP doesn't
specify any algorithm for aggregation itself. What BGP does address is
the interaction between possible aggregation and its loop detection
mechanism.

> Also note that route
> aggregation rules are part of the routing protocol specification and
> definitely depend on the protocol behavior (true for BGP, OSPF, and
> ISIS), so it is not a completely distinct notion, though some
> implementations decouple the two.

BGP does not specify any rules for what routes should be aggregated or
into what they should be aggregated. It cannot since these are
operational decisions.

The statement above is false as far as i can understand it.

>>>  5. Coupling of VPLS and BGP SW

>>>     a) Lesser BGP code stability--bugs in the VPLS part of the
>>> code

>> You have no basis to conclude that.

> I do :)

Please present a justification then.
We are back to "no, it isn't" / "yes, it is" mode.

> The fact is that pieces of code in routing protocol implementations
> are not only statically related via the function call tree, but also
> dynamically and indirectly... but I will stop right here, because
> we'll inevitably get into implementation specifics...

This is what i call FUD...
"Uncertainty" not being backed by any argument.

The way i see it this is a central piece of your (IESG) original
"concern" statement.

>>>     b) Potential dynamic effects--since with a BGP-based approach,

>> I'm sorry but this is just FUD.

> I hope people don't think about potential interference between large
> distributed systems as FUD.

You are wording it just that way: "potential interference between large
distributed systems".

No facts, no specific points. Just vague allegations.

If the above is "fair game" then i want to ask the IESG to use the
same "potential interference" criterium to, say, IPv6...

IPv6 not only involves changes to BGP to support this NLRI but changes
to all hosts and pretty much all protocols and applications. It can
"dynamically and indirectly" cause interference too.

Perhaps IPv6 is a treat to the internet stability ? Since i don't
intend to run IPv6 in the forseable future it standardization of IPv6
is going cause 'interference' with my workstation software, which
i rather not deal with.

>    I tend to look at this more broadly--putting VPLS functionality
> in BGP increases the chances of interference.  Some consider this a
> strictly implementation-specific issue.  I think that whether or not
> VPLS-specific functionality is sufficiently decoupled from base BGP
> is an implementation aspect; while increased risk of interference is
> an architectural one.

I believe that this completly confuses architecture with
implementation.

This is not an architectural consideration. This is a vague aspersion
on the competency of BGP implementors.

Your argument taken to the extreme would result in forbidding all and
any standardization of new software mechanisms.

  Pedro.

Date: Wed, 7 May 2003 11:28:15 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: On BGP and VPLS
Sender: [email-masked]

Pedro,

Sorry for the delay. I found your message in my IDR folder just
recently. I'll limit my involvement to this message, I think I will
have said enough for my opinion to be heard :)

> This point seems to be predicated in the statement that "BGP uses the
> NLRI field to carry IP reachability"...

Yes, plus that "BGP is an IP routing protocol."

> It opens up a sort of philosophical discussion on BGP. This is of
> course a highly subjective topic which is hard to quantify or to prove
> by logical terms.

> Allow me to present my personal view.

> BGP is a particular implementation of an algorithm that performs non
> looping database flooding distribution. That algorithm consists mostly
> on the path vector (used both in ebgp and route reflection) plus route
> advertisement rules. This is the publicly specified part of the beast.

This is where we disagree in the first place.

There's a fine line between a database distribution flooding algorithm
and a path-vector routing algorithm. BGP is clearly not the former.

> However that ends up being about 10% of the database exchange
> algorithm. Each implementation uses distinct algorithms to do the real
> heavy lifting: the advertisement of database updates to its peers,
> given that each peer is allowed to flow control and that the ammount
> of information to be distributed is typically non-trivial compared to
> the resources of the system.

> None of the functions above actually do depend on the format of your
> database records. As long as there is a primary key associated with
> each record. Modern implementations, given that they are required to
> handle 3/4 different types of records w/ different keys (ipv4, ipv6,
> 2547, 2547-for-ipv6, etc) will tend to treat these keys just as
> database systems do: as a bit string without any semantics associated
> w/ it.

> Note also that the number of distinct tables exchanged in a 2547
> implementation may be in the thousands. So segregation of which record
> belongs to which table is necessarily a solved problem in practice.

> There is one part of BGP that however interacts w/ the semantics of
> the particular database being exchanged: route selection from the
> Loc-RIB.

> The Loc-RIB is by definition where BGP interacts w/ remaining users of
> the database and it includes rules that are system specific.

> As an exercise, if we take the existing spec and do:
> s/route/record/g
> s/IP prefix/key/g

> Do we still have a document that makes sense.. ?

I'm not sure :)

> Except for the vague bits about aggregation, about which BGP itself
> does little about, i would contend that the result would be pretty
> much the same.

You are forgetting the parts about Loc-RIB, routing and forwarding
tables, next-hop, etc. In any case, such a beast would seize to be an
IP routing protocol, but would still perform best path selection to an
opaque key in the Internet. If you need a database distribution
mechanism as you described above, you don't need the path vector
behavior, nor per-peer state for each key, nor next-hops. I.e., you
don't need what BGP does.

> 2547 which you cite is a particular good example, imho. A 2547 NLRI
> ends up being used to create IP reachability information, but while it
> is a safi 128 record, it is not IP reachability and it is not treated
> as such.

2547 augments _IP prefixes_ with route distinguishers only to make
sure that the prefix is unique when used as the key in the RIB. The
rest of the reachability/prefix semantics are preserved.

>>  2. Distribution of information

> That is not the case w/ 2547. PE routers typically have interest in
> only a subset of the routing information. They tend to do inbound
> filtering in current network deployements but one can also do outbound
> filtering in the RRs via either extended-community ORF or subsequent
> improvements to ORF (draft-marques-ppvpn-rt-contrain).

Note that I purposely compared the approach in the document with
the base BGP, not 2547.

> P routers do not carry 2547 routing information.

Unless the SPs want to use the existing RR infrastructure.

> RR in VPN deployments are typically not in the topology. My
> understanding of the P-router term is that it is a transit node that
> does not have VPN information.

Agree, they don't have to. The text should have been "More than that,
route reflectors (in some cases implemented on P routers) end up..."

> Not really... i can advertise the same key from multiple sources in
> L2VPNs also. All policy mechanisms do work... igp distance, etc. It is
> just the semantics once the path is selected that are different.
> As an example think working and protect PE for a given emulated
> circuit (or lan).

I think you might have missed my point here. Though a BGP speaker will
receive the same key from multiple sources, and will select the best
path to it, in the VPLS case, it is not interested in the _best_ path,
it is interested in only receiving the key, regardless of where it
comes from.

> I don't know which model you have in mind but in a typical VPN
> deployment scenario (l3 or l2/vpls/etc) a PE has 2 peering sessions to
> a RR outside of the topology. The second copy of the information is
> there for redudancy...

> If a full mesh where used, only 1 copy would be present.

So RR and eBGP are the examples where a given BGP speaker would
receive extra copies of information. As I replied to Mark:

    Well, in the IP/BGP case, it is not necessarily the same info,
    as path attributes are likely to be different and the BGP
    speaker is interested in selecting the best among them while
    preserving others as the back-up. In the VPLS case, we don't
    need the best, just one copy is sufficient.

    This is not necessarily an issue per se, rather an interesting
    observation exposing the transport nature of this particular
    proposed application of BGP. We have similar properties (more
    than one copy received) in the flooding algorithm in IGPs, but
    we admit that flooding is a transport component very specific
    to IGPs (where every node needs all info, btw), and we don't
    keep track of where we receive PDUs from as we do in BGP.

>>  3. Aggregation of information for large-scale operation
...
> To give you an example, in JunOS aggregation is implemented as a
> separate routing protocol... if i'm not mistaken the model is lifted
> from 'gated'. Clearly the idea that aggregation may be a distinct
> component from BGP has been around for a while.

This does not change the fact that information that BGP as an IP
routing protocol distributes is aggregateable. Also note that route
aggregation rules are part of the routing protocol specification and
definitely depend on the protocol behavior (true for BGP, OSPF, and
ISIS), so it is not a completely distinct notion, though some
implementations decouple the two.

> VPLS doesn't really need aggregation although it does use an IGP :-)
> PE to PE connectivity is performed indepently from the 'forwarding
> distinguisher' advertisement (i.e. the inner label). Any or multiple
> routing and singaling protocols may be used for this
> functionality. Only the information exterior to the SP network
> (service attachements) is carried through BGP.

This part should go into the thread on "Info Summarization" where one
of the questions is how we can limit the amount of state/information
that a given participating node will have to maintain.

I'd like to again highlight the difference between aggregateable
semantics of NLRI contents in BGP when used as IP routing protocols,
and non-aggregateable semantics of it in the proposal, which means
that mechanisms different from those existing in current BGP practices
would need to be used to limit the amount of maintained info.

>>  The above gives me a very uncomfortable feeling that the proposal
>> is stretching BGP to perform functions it was not designed for.

> Any succesful protocol will be used for means other than it was
> designed for. That is usually a sign that the designers got something
> right.

I was being mild.

>>  4. Backwards compatibility and SW upgrade requirements

> That is not an issue as we've seen above. The deployment model is
> different from what you assume.

As before, unless the SPs want to use the existing RR infrastructure.

>>  5. Coupling of VPLS and BGP SW

>>     a) Lesser BGP code stability--bugs in the VPLS part of the code

> You have no basis to conclude that.

I do :)

> Any modern BGP implementation worth its salt consists of
> AF-independent code + AF-specific code. The fact is that you can
> implement VPLS without touching the AF-independent code.

The fact is that pieces of code in routing protocol implementations
are not only statically related via the function call tree, but also
dynamically and indirectly... but I will stop right here, because
we'll inevitably get into implementation specifics...

>>     b) Potential dynamic effects--since with a BGP-based approach,

> I'm sorry but this is just FUD.

I hope people don't think about potential interference between large
distributed systems as FUD.

> All router implementations do have some level of resource sharing
> between completly unrelated features. In some of them, all
> functionality shares all resources.

Agreed, though I was talking about tighter coupling when Inet BGP and
VPLS BGP are in the same process/thread (which is very likely the
case). As I told in my answer to Mark:

   I tend to look at this more broadly--putting VPLS
   functionality in BGP increases the chances of interference.
   Some consider this a strictly implementation-specific issue.
   I think that whether or not VPLS-specific functionality is
   sufficiently decoupled from base BGP is an implementation
   aspect; while increased risk of interference is an architectural
   one.

Again, I'll stop here too.
   
>>  My recommendation would be for the WG to consider these points.

> The way i see it there is an high likely-hood of this turning into an
> "Yes, it is" "No, it isn't" discussion. And I'd really like to avoid
> that.

Agreed.

> A question to you and to the WG(s) in general:

> - What are the main concerns that you have w/ the generic database
> exchange view of BGP (Lets call it the "Basically General Purpose"
> theory).

The fact that BGP is not a generic database exchange protocol,
and I don't think it should be positioned as such.

> - Can we have a reasonable discussion about the best engineering
> approach to provide database exchange services for
> routing-related-applications without getting into a religious argument
> about "2547 is evil" ? i.e. can we try to separate how highly each one
> of us rates the actual application from this discussion ?

I think we'll have to agree on the definition of "database exchange
services" and "routing-related-applications", but generally, yes,
sure.

> - I believe one of the preconditions for a resonable discussion is to
> realise that implementors are the most interested people in not
> introducing regressions to shipping code. They actually get to fix it
> after being screamed at for a considerable lenght of time.
> I'd really like to get past the "you can't implement a feature i don't
> want because your are going to break the code" kind of discussion.

I think there is a generally good understanding of this. I don't think
this is something that is sufficient for the IETF to base technical
conclusions on.

> - Are we going to have a similiar discussion about LDP ? LDP is not
> any less relevant for network stability nor a protocol which is any
> simpler than BGP (if anything the level of complexity is higher given
> that LDP has all the db exchange problem of BGP + a non trivial
> ammount of issues of its own).

I have absolutely no problem with this.

Thanks for your comments.

Alex

Date: Mon, 5 May 2003 16:38:15 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: AD-review comments on draft-ietf-idr-bgp4-20
Sender: [email-masked]

Folks,

 Please find below my AD-review comments. Hopefully they will help
 improve the document. I tried to consult Andrew's list as much as
 possible, but do feel free to point out if something has already been
 discussed and agreed upon.
 
 Thanks go to Yakov for kicking me often enough ;)

--
Alex Zinin

Some nits:
- run it by a spelling checker, please
- disable hyphenation if possible
- include boilerplates for IPR notice, Copyright notice

General comment:

  in some places I highlighted the fact that required behavior is not
  described using the 2119 language, so it is not clear if a MUST or
  SHOULD or MAY is applicable. I am sure I've missed some more places
  like this. I'd like to ask the editors to go through the doc and
  check this.

>                   A Border Gateway Protocol 4 (BGP-4)
>                       <draft-ietf-idr-bgp4-20.txt>
> 
> 
> Status of this Memo
> 
> 
...
>    The list of Internet-Draft Shadow Directories can be accessed at
>    http://www.ietf.org/shadow.html.
>
> Specification of Requirements

Nit: move Abstract here. Move requirements after the Acks.

> Abstract

Should the Abstract say that this spec covers IPv4 only?

> 3. Summary of Operation
...
>    This document uses the term `Autonomous System' (AS) throughout.  The
>    classic definition of an Autonomous System is a set of routers under
>    a single technical administration, using an interior gateway protocol
>    (IGP) and common metrics to determine how to route packets within the
>    AS, and using an inter-AS routing protocol to determine how to route
>    packets to other ASs. Since this classic definition was developed, it
>    has become common for a single AS to use several IGPs and sometimes
>    several sets of metrics within an AS. The use of the term Autonomous
>    System here stresses the fact that, even when multiple IGPs and met-
>    rics are used, the administration of an AS appears to other ASs to
>    have a single coherent interior routing plan and presents a consis-
>    tent picture of what destinations are reachable through it.

Ed: Since 'AS' has been defined before, do we need to repeat the
definition here?

...
>    peer in the same AS is referred to as an internal peer. Internal BGP
>    and external BGP are commonly abbreviated IBGP and EBGP.

Ed: These two have been defined before too

...
> Care must be taken to
>    ensure that the interior routers have all been updated with transit
>    information before the BGP speakers announce to other ASs that tran-
>    sit service is being provided.

What does the last sentence really mean from the implementation
perspective? It used to mean the BGP/IGP synchronization check. Now
that iBGP everywhere is assumed, how do we check this condition?

>    This document specifies the base behavior of the BGP protocol. This
>    behavior can and is modified by extention specifications.  When the
Ed: "extension"

>    protocol is extended the new behavior is fully documented in the
>    extention specifications.
Ed: "extension"

> 3.1 Routes: Advertisement and Storage
> 
>    For the purpose of this protocol, a route is defined as a unit of
>    information that pairs a set of destinations with the attributes of a
>    path to those destinations. The set of destinations are systems whose
>    IP addresses are contained in one IP address prefix carried in the
>    Network Layer Reachability Information (NLRI) field of an UPDATE mes-
>    sage, and the path is the information reported in the path attributes
>    field of the same UPDATE message.
Ed: Repeated definition again
...
>    If a BGP speaker chooses to advertise the route, it MAY add to or
>    modify the path attributes of the route before advertising it to a
>    peer.

The intent here is to say that it's ok to modify the attribute set of
a previously received route when it's announced further. The way it
reads though is that self-originated routes are also within the
context and MAY sounds like you don't have to add attributes when
announcing those.

...

>    Changing attribute of a route is accomplished by advertising a
>    replacement route. The replacement route carries new (changed)
>    attributes and has the same NLRI as the original route.

"same NLRI" implies the same prefix, but not the NLRI field, which can
be different (containing other routes), should the use of this term be
normalized throughout the document?

> 4.2 OPEN Message Format
> 
>    After a TCP is established, the first message sent by each side is an

"TCP connection"

> 5. Path Attributes
...
>    If a path with recognized transitive optional attribute is accepted
>    and passed along to other BGP peers and the Partial bit in the
>    Attribute Flags octet is set to 1 by some previous AS, it is not 

'MUST NOT' here?

> set
>    back to 0 by the current AS. Unrecognized non-transitive optional
>    attributes MUST be quietly ignored and not passed along to other BGP
>    peers.
...
>    The same attribute (attribute with the same type) can not appear more
>    than once within the Path Attributes field of a particular UPDATE
>    message.

What should an implementation do if this happens?

>    The mandatory category refers to an attribute which MUST be present
>    in both IBGP and EBGP exchanges if NLRI are contained in the UPDATE

Ed: "if the NLRI field is contained" instead?

> 5.1.2 AS_PATH
...
>       b) When a given BGP speaker advertises the route to an external
>       peer, then the advertising speaker updates the AS_PATH attribute
>       as follows:
> 
>          1) if the first path segment of the AS_PATH is of type
>          AS_SEQUENCE, the local system prepends its own AS number as the
>          last element of the sequence (put it in the leftmost position).

'Leftmost position'... isn't this still open for interpretation? How
about wording this relative to the position of the octets in the
protocol message?

>          If the act of prepending will cause an overflow in the AS_PATH
>          segment, i.e. more than 255 ASs, it is legal

What's the recommended behavior here?

>          to prepend a new
>          segment of type AS_SEQUENCE and prepend its own AS number to
>          this new segment.

> 5.1.4 MULTI_EXIT_DISC
> 
> 
>    The MULTI_EXIT_DISC is an optional non-transitive attribute which is
>    intended to be used on external (inter-AS) links to discriminate
>    among multiple exit or entry points to the same neighboring AS.  The
>    value of the MULTI_EXIT_DISC attribute is a four octet unsigned num-
>    ber which is called a metric. All other factors being equal, the exit
>    point with lower metric SHOULD be preferred. If received over EBGP,
>    the MULTI_EXIT_DISC attribute MAY be propagated over IBGP to other
>    BGP speakers within the same AS. The MULTI_EXIT_DISC attribute

seems that a reference to 9.1.2.2 is due here, as using MED in local
route calculation and not propagating it further is dangerous

>    received from a neighboring AS MUST NOT be propagated to other neigh-
>    boring ASs.
> 
>    A BGP speaker MUST IMPLEMENT a mechanism based on local configuration
                        ^^^^^^^^^lower-case
                        
>    which allows the MULTI_EXIT_DISC attribute to be removed from a
>    route. This MAY be done prior to determining the degree of preference

what's the recommended behavior here?

>    of the route and performing route selection (decision process phases
>    1 and 2).
> 
>    An implementation MAY also (based on local configuration) alter the
>    value of the MULTI_EXIT_DISC attribute received over EBGP.  This MAY
>    be done prior to determining the degree of preference of the route

what's the recommended behavior here?

> 5.1.5 LOCAL_PREF
...
> A BGP speaker SHALL calculate the degree of preference for
>    each external route based on the locally configured policy, and

Should we be more honest here and say that the implementation must
allow the admin to SET the degree of preference through the local
policy to influence the best-path selection process, i.e., I don't
think any implementation really *calculates* it.

> 5.1.6 ATOMIC_AGGREGATE
...
>    A BGP speaker that receives a route with the ATOMIC_AGGREGATE
>    attribute MUST NOT make any NLRI of that route more specific (as
>    defined in 9.1.4) when advertising this route to other BGP speakers.

Since deaggregation is not described in this document, do we need this
para?

>   A BGP speaker that receives a route with the ATOMIC_AGGREGATE
>    attribute needs to be cognizant of the fact that the actual path to
>    destinations, as specified in the NLRI of the route, while having the
>    loop-free property, may not be the path specified in the AS_PATH
>    attribute of the route.

What does this really mean from the implementation perspective?

> 5.1.7 AGGREGATOR
> 
> 
>    AGGREGATOR is an optional transitive attribute which MAY be included
>    in updates which are formed by aggregation (see Section 9.2.2.2). A
>    BGP speaker which performs route aggregation MAY add the AGGREGATOR

What's the recommended behavior here? Include or not, and under what
circumstances?

> 6. BGP Error Handling.
...
>    The phrase "the BGP connection is closed" means that the TCP connec-
>    tion has been closed, the associated Adj-RIB-In has been cleared, and
>    that all resources for that BGP connection have been deallocated.
>    Entries in the Loc-RIB associated with the remote peer are marked as
>    invalid. The fact that the routes have become invalid is passed to
>    other BGP peers before the routes are deleted from the system.

What does "the fact is passed" mean? Should we instead say that local
route recalculation happens and peers are sent either updated best
routes or withdrawals?

> 6.2 OPEN message error handling.
...
>    If the Autonomous System field of the OPEN message is unacceptable,
>    then the Error Subcode is set to Bad Peer AS. The determination of
>    acceptable Autonomous System numbers is outside the scope of this
>    protocol.

Shouldn't we say that configuration based detection should be
supported, i.e., when remote-as is configured for the peer?

...
>   If the BGP Identifier field of the OPEN message is syntactically
>    incorrect, then the Error Subcode is set to Bad BGP Identifier.  Syn-
>    tactic correctness means that the BGP Identifier field represents a
>    valid IP host address.

Is "valid IP host address" defined somewhere, btw?

> 6.3 UPDATE message error handling.
> 
> 
>    All errors detected while processing the UPDATE message are indicated
>    by sending the NOTIFICATION message with Error Code UPDATE Message
>    Error. The error subcode elaborates on the specific nature of the
>    error.

"are indicated..." is this a MUST, SHOULD, or MAY?
...
>    If the ORIGIN attribute has an undefined value, then the Error Sub-
>    code is set to Invalid Origin Attribute. The Data field contains the
>    unrecognized attribute (type, length and value).

Curious: do we really have to drop a session on this condition? Given
that the attribute was syntactically correct and the TLV was not
broken, so the stream is still in sync and we can move on? Of course,
if this is what current implementations do, we have no other choice.

...
>    If the UPDATE message is received from an external peer, the local
>    system MAY check whether the leftmost AS in the AS_PATH attribute is

Same comment about 'leftmost'... Maybe we should define this somewhere
in the beginning of the spec?

...
>    The NLRI field in the UPDATE message is checked for syntactic valid-
>    ity. If the field is syntactically incorrect, then the Error Subcode
>    is set to Invalid Network Field.

Should we give more data on what syntactic validity means in this case
so people behave consistently?

> 6.7 Cease.
...
> If the BGP speaker decides to terminate its BGP
>    connection with a neighbor because the number of address prefixes
>    received from the neighbor exceeds the locally configured upper
>    bound, then the speaker MUST send to the neighbor a NOTIFICATION mes-
>    sage with the Error Code Cease.

Should we also say that when the peer decides to discard incoming
prefixes, this event should be logged locally?

> 8. BGP Finite State machine

General comment: I would _really_ appreciate more people looking at this
section.

>    The optional Session attributes are listed below. These optional
>    attributes may be supported either per connection or per local sys-
>    tem:
> 
>         1) Delay Open flag

Where's the description of this flag and how/when is it set? Same for
others below. Should we have a brief description for each attribute?

>         2) Open Delay Timer
>         3) Perform automatic start flag
>         4) Perform automatic stop flag
>         5) Passive TCP establishment flag
>         6) Perform BGP peer oscillation damping flag
>            (which will be denoted as stop_peer_flap in text)
>         7) Idle Hold timer
>         8) Perform Collision detect in Established flag
>         9) Accept connections from un-configured peers
>        10) Track TCP state flag
>        11) Send NOTIFICATION without an OPEN flag
> 
Suggestion: to make reading of the FSM description below easier, we
could "merge" the multiword flag names and normalize them, e.g.
'perform automatic start flag' to 'PerformAutoStart flag'. 'Passive
TCP establishment flag' to 'PassiveTCPEstablishment flag',
'stop_peer_flap' to 'StopPeerFlag'.

> 8.1.1 Administrative Events
> 
> 
>    Please note that only Event 1 (manual start) and Event 2 (manual
>    stop) are mandatory administrative events. All other administrative
>    events are optional. The optional attributes do not have to be sup-
>    ported. However, if these attributes are supported, the state of the
>    flags should be as indicated.

'flags should be as indicated' does not give a clear understanding of
what they are used for. Should the events be sanity-checked by
checking those attributes? what's the recommended behavior when the
flags are in a different state?

>        Event3: Automatic start
> 
>               Definition: Local system automatically starts the
>                           BGP connection.
> 

When is this event generated by the system? Under what conditions?
>               Status:     Optional depending on local system.
> 
>               Optional
>               attributes: 1) Perform automatic start flag SHOULD be set
>                              if this event occurs.
>                           2) if the passive Passive TCP establishment flag

passive Passive?

>        Event5: Automatic start with passive TCP flag
> 
>               Definition: Local system automatically starts the
>                           BGP connection with the passive flag
>                           enabled.  The passive flag indicates
> 

Same question about generation conditions
..
>        Event23: Open collision dump
> 
>               Definition: An event generated administratively
>                           when a connection collision has been
>                           detected while processing an incoming
>                           OPEN message and this connection has been
>                           selected to disconnected. See Section
'to be disconnected'
>                           6.8 for more information on collision
>                           detection.
> 
>                           Event23 is an administrative based only
'based on'?
>                           implementation specific policy. This
>                           Event may occur if the FSM is implemented
>                           as two linked state machines.
> 
> 
>               Status:     Optional, depending on local system
> 
>               Optional
>               Attributes: If the state machine is to process this
>                           attribute in Established state,
>                            1) Peform Collision detect in Established
'Perform'
>                                flag SHOULD be set.

...

>        Event25: NotifMsg
> 
>               Definition: An event is generated when a
>                           NOTIFICATION messages is received and
message
>                           the error code is anything but
>                           "version error".
> 
>               Status:     Mandatory

> 8.2.1 FSM Definition
> 
> 
>    BGP MUST maintain a separate FSM for each configured peer, Each BGP
>    peer paired in a potential connection unless configured to remain in
>    the idle state, or configured to remain passive, will attempt to  to
to to

>    connect to the other.  For the purpose of this discussion, the active
>    or connect side of the TCP connection (the side of a TCP connection
'active or connecting'?

>    sending the first TCP SYN packet) is called outgoing.  The passive or
>    listening side (the sender of the first SYN ACK) is called an incom-
>    ing connection (see Section 8.2.1.1 on the terms active and passive
>    below).
> 
>    A BGP implementation MUST connect to and listen on TCP port 179 for
>    incoming connections in addition to trying to connect to peers.  For
>    each incoming connection, a state machine MUST be instantiated.

Is this true for implementations that resolve connection collision
through one FSM with two transport connections?

> 8.2.1.1 Terms "active" and "passive"
> 
> 
>    The terms active and passive have been in our vocabulary for almost a
>    decade and have proven useful.  

Ed: The style here is quite different from the rest of the document
(i.e., personalization), plus time values tend become outdated with
time :)

> 8.2.1.2 FSM and collision detection
> 
> 
>    There is one FSM per BGP connection.  Prior to determining what peer
>    a connection is associated with there may be two connections for a
>    given peer.  There SHOULD be no more than one connection per peer.

Is above "SHOULD" normative? I.e., should be "should" instead?

>   The collision detection identifies the case where there is more than
>    one connection per peer and provides guidance for which connection to
>    get rid of.  When this occurs, the corresponding FSM for the connec-
>    tion that is closed SHOULD be disposed of.
> 

BTW, I think the specification would really benefit from a section
that describes processing of incoming transport connections.

> 8.2.2 Finite State Machine
> 
> 
>       Idle state:
> 
>          Initially BGP is in the Idle state.

Not BGP, but the peer FSM, right?

> 
>          In this state BGP refuses all incoming BGP connections.  No

all incoming connections from that peer?

> 
>          resources are allocated to the peer. In response to a
>          manual start event(Event1) or an automatic start
>          event(Event3), the local system:
>             - initializes all BGP resources,
all BGP resources or only those needed for the peer?
also, what does 'initialize' mean here?

>             - sets ConnectRetryCnt (the connect retry counter) to zero

Seems we have inconsistency in FSM parameter naming here.

>         In response to a manual start event with the passive TCP connection
>         flag (Event 4) or automatic start with the passive TCP connection
>         flag (Event 5), the local system:
>             - initializes all BGP resources,
>             - sets ConnectRetryCnt (the connect retry counter) to zero,
>             - starts the Connect Retry timer with initial value,
>             - listens for a connection that may be initiated by
>               the remote peer, and
>             - changes its state to Active.

Ditto comments here

>         The method of preventing persistent peer oscillation is
>         outside the scope of this document.

So we have these events, but we don't define how to handle them?

>         Any other events [Events 9-12, 15-28] received in the Idle state
>         does not cause change in the state of the local system.

'do not cause changes'  ?

>         In response to a manual stop event [Event2], the local system:
>            - drops the TCP connection,
>            - releases all BGP resources,
>            - sets ConnectRetryCnt (the connect retry count) to zero
>            - sets the Connect Retry timer to zero, and

sets timer to zero? 'Stops the timer' instead?

>            - changes its state to Idle.
...

>         If the BGP port receives a valid TCP connection indication
BGP port?
>         [Event 14], the TCP connection is processed and
>         the connection remains in the Connect state.
> 
>         If the TCP connection receives an invalid indication [Event 15]:

TCP connection receives?

>         the local system rejects the TCP connection and the connection
>         remains in the Connect state.
> 
>         If the TCP connection succeeds [Event 16 or Event 17],
>         the local system checks the Delay Open flag prior to
>         processing. If the Delay Open flag is set, the local system:
>              - sets the Connect Retry timer to zero,
"stops" instead?

>              - set the Open Delay timer to the initial value, and

sets

>              - stays in the Connect state.
>         If the Delay Open flag is not set, the local system:
>              - sets the Connect Retry timer to zero,
stops

>              - completes BGP initialization

What does the above really mean?

...
>         the Open Delay Timer. If the Open Delay timer is running,
>         the local system:
>             - restarts the connect retry time with initial value,
>             - stops the Open Delay timer and resets value to zero,
>             - continues to listen for a connection that may be
>               initiated by the remote BGP peer, and
>             - changes its state to Active.
>         If the open Delay timer is not running, the local system:
>            - sets the Connect Retry timer to zero,
>            - drops the TCP connection,
>            - releases all BGP resources, and
all BGP resources?

>            - changes its state to Idle.
> 
>         If an OPEN message is received with the Open Delay timer is
>         running [Event 20], the local system:
>            - sets the Connect Retry timer to zero,
>            - completes the BGP initialization,
What does it mean?

>            - stops and clears the Open Delay timer (sets the value to zero),
>            - sends an OPEN message,
>            - sends a KEEPALIVE message,
>            - If the hold timer value is non-zero,
>                    - start the keepalive timer to inital value,
"starts"... start to initial value?

>                    - reset the hold timer to the negotiated value,
Resets

>              else if hold timer value is zero,
>                    - reset the keepalive timer, and

resets

>                    - reset the hold timer value to zero
resets

>            - and changes its state to OpenConfirm.
> 

OK, I'll stop reviewing the FSM text here and will skip to the next
section. Given the number of English grammar mistakes, it is clear to
me that either it has not been sufficiently reviewed or even read by
someone carefully enough or the comments have not been incorporated.
Please address.

...
> 9. UPDATE Message Handling
> 
> 
>    An UPDATE message may be received only in the Established state.
What if it is received in another state?

...
> 9.1 Decision Process
> 
> 
>    The Decision Process selects routes for subsequent advertisement by
>    applying the policies in the local Policy Information Base (PIB) to
>    the routes stored in its Adj-RIBs-In. The output of the Decision Pro-
>    cess is the set of routes that will be advertised to peers; the
>    selected routes will be stored in the local speaker's Adj-RIB-Out
RIB-Out or RIBs-out (plural)?

>    according to policy.
> 
>    The selection process is formalized by defining a function that takes
>    the attribute of a given route as an argument and returns either (a)
>    a non-negative integer denoting the degree of preference for the
>    route, or (b) a value denoting that this route is ineligible to be
>    installed in LocRib and will be excluded from the next phase of route

Loc-RIB
>    selection.
...
>    The Decision Process operates on routes contained in the Adj-RIB-In,
Adj-RIBs-In (plural) ?
>    and is responsible for:

> 9.1.1 Phase 1: Calculation of Degree of Preference
...
>       If the route is learned from an external peer, then the local BGP
>       speaker computes the degree of preference based on preconfigured
>       policy information. If the return value indicates that the route
>       is ineligible, the route MAY NOT serve as an input to the next
>       phase of route selection; otherwise the return value is used as
>       the LOCAL_PREF value in any IBGP readvertisement.

So, AFAIK, the major implementations do not follow this step
(calculating the degree of preference, and then announcing). Instead,
implementations allow setting the LOCAL_PREF value locally, which is
taken into consideration during the best path selection, and is also
reannounced further.

Also "is used" is not specific enough. Is it SHOULD or MUST?

> 9.1.2 Phase 2: Route Selection
...
>    If the AS_PATH attribute of a BGP route contains an AS loop, the BGP
>    route should be excluded from the Phase 2 decision function.  AS loop
>    detection is done by scanning the full AS path (as specified in the
>    AS_PATH attribute), and checking that the autonomous system number of
>    the local system does not appear in the AS path.  Operations of a BGP
>    speaker that is configured to accept routes with its own autonomous
>    system number in the AS path are outside the scope of this document.

If we're checking for an AS loop here (in Phase 2) as opposed to
during the UPDATE message sanity checking, the route is already
received and accepted in the peer's Adj-RIB-In. Those implementations
I know don't even install such routes in the RIB...

> 9.1.2.2 Breaking Ties (Phase 2)
...
>       Similarly, neighborAS(n) is a function which returns the neighbor
>       AS from which the route was received.  If the route is learned via
>       IBGP, and the other IBGP speaker didn't originate the route, it is
>       the neighbor AS from which the other IBGP speaker learned the
>       route. If the route is learned via IBGP, and the other IBGP
>       speaker originated the route, it is the local AS.

What if the route is locally originated?

> 9.1.4 Overlapping Routes
...
>    When overlapping routes are present in the same Adj-RIB-In, the more
>    specific route takes precedence, in order from more specific to least
>    specific.
> 
Doesn't this happen at the packet forwarding stage?

> 
>    The set of destinations described by the overlap represents a portion
>    of the less specific route that is feasible, but is not currently in
>    use.  If a more specific route is later withdrawn, the set of desti-
>    nations described by the overlap will still be reachable using the
>    less specific route.
> 
>    If a BGP speaker receives overlapping routes, the Decision Process
>    MUST consider both routes based on the configured acceptance policy.
>    If both a less and a more specific route are accepted, then the Deci-
>    sion Process MUST either install both the less and the more specific
Install where?

>    routes or it MUST aggregate the two routes and install the aggregated
>    route, provided that both routes have the same value of the NEXT_HOP
>    attribute.

anyone really does the latter?

>    If a BGP speaker chooses to aggregate, then it SHOULD either include
>    all AS used to form the aggreagate in an AS_SET or add the
>    ATOMIC_AGGREGATE attribute to the route.  This attribute is now pri-
>    marily informational.  With the elimination of IP routing protocols
>    that do not support classless routing and the elimination of router
>    and host implementations that do not support classless routing, there
>    is no longer a need to deaggregate.  Routes SHOULD NOT be de-aggre-
>    gated.  A route that carries ATOMIC_AGGREGATE attribute in particular
>    MUST NOT be de-aggregated. That is, the NLRI of this route can not be
>    made more specific. Forwarding along such a route does not guarantee
>    that IP packets will actually traverse only ASs listed in the AS_PATH
>    attribute of the route.

Since we don't do deaggregation any more, should we remove the
discussion about it completely and indicate in the "changes" section
that deaggregation has been deprecated?

> 9.2 Update-Send Process
...
>    When a BGP speaker receives an UPDATE message from an internal peer,
>    the receiving BGP speaker SHALL NOT re-distribute the routing infor-
>    mation contained in that UPDATE message to other internal peers,
>    unless the speaker acts as a BGP Route Reflector [RFC2796].

Suggest to put "unless..." in brackets () to make it more apparent
that this is not a normative ref.

> 9.2.1.1 Frequency of Route Advertisement
>    Since fast convergence is needed within an autonomous system, either
>    (a) the MinRouteAdvertisementInterval used for internal peers SHOULD
>    be shorter than the MinRouteAdvertisementInterval used for external
>    peers, or (b) the procedure describe in this section SHOULD NOT apply
>    for routes sent to internal peers.

It sounded like MinRouteAdvertisementInterval was an architectural
constant, but now it sounds like either this is a timer that can be
assigned different settings or there are two constants:
MinRouteAdvIntIBGP and MinRouteAdvIntEBGP.

> 9.2.2.2 Aggregating Routing Information
> 

Hmmm... I expected to see in this section some text talking about when
and how an aggregate would be announced, i.e., when an aggregate
prefix is configured, and more specific routes are present, the
aggregate is announced, when no specifics are left--withdraw the
aggregate. I haven't found anything on this topic...

> 9.3 Route Selection Criteria
>
>    Generally speaking, additional rules for comparing routes among sev-
>    eral alternatives are outside the scope of this document. There are
>    two exceptions:
> 
>       - If the local AS appears in the AS path of the new route being
>       considered, then that new route can not be viewed as better than
>       any other route (provided that the speaker is configured to accept
>       such routes). If such a route were ever used, a routing loop could
>       result.
> 
>       - In order to achieve successful distributed operation, only
>       routes with a likelihood of stability can be chosen. Thus, an AS
>       SHOULD avoid using unstable routes, and it SHOULD NOT make rapid
>       spontaneous changes to its choice of route. Quantifying the terms
>       "unstable" and "rapid" in the previous sentence will require expe-
>       rience, but the principle is clear.
> 

Where does this (the second one) fit within and how does this affect
the route selection criteria?

>    Care must be taken to ensure that BGP speakers in the same AS do not
>    make inconsistent decisions.

How? What does this mean for the implementor?

> 9.4 Originating BGP routes
> 
>    A BGP speaker may originate BGP routes by injecting routing informa-
>    tion acquired by some other means (e.g. via an IGP) into BGP. A BGP
>    speaker that originates BGP routes assigns the degree of preference
> 

"assigns the degree of preference"... how do the implementations
really do that?

> 10 BGP Timers
...
>    The suggested default value for the MinRouteAdvertisementInterval is
>    30 seconds.

This was described as a parameter, not a timer. Further, it was
earlier suggested that it should be shorter for iBGP than it is for
eBGP. I'd expect the document to specify the recommended value for
both.

> IANA Considerations
...
>    All extensions to this protocol, including new message types and Path
>    Attributes MUST only be made using the Standards Action process
>    defined in [RFC2434].

This section should include the description of each registry that
needs to be created (if needed) and maintained by IANA, as well as the
allocation policy that is in the text already.

<EOM>

To: Jeffrey Haas <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, [email-masked]
Reply-To: [email-masked]
Subject: Re: Issue 19) Security Considerations 
Date: Mon, 05 May 2003 16:09:20 -0400
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Jeffrey Haas writes:
> Yakov,
> 
> On Mon, May 05, 2003 at 09:59:14AM -0700, Yakov Rekhter wrote:
> > I don't recall seeing any objections to adding this to the document.
> 
> It was more that I hadn't heard anything from anyone one way or the
> other. 
> 
> > Yakov.
> 
> -- 
> Jeff Haas 
> NextHop Technologies

Neither did I, in case there was any question about whether something
was being worked out off list.

Curtis

Date: Mon, 5 May 2003 14:45:13 -0400
From: Jeffrey Haas <[email-masked]>
To: Yakov Rekhter <[email-masked]>
Cc: [email-masked]
Subject: Re: Issue 19) Security Considerations
Sender: [email-masked]

Yakov,

On Mon, May 05, 2003 at 09:59:14AM -0700, Yakov Rekhter wrote:
> I don't recall seeing any objections to adding this to the document.

It was more that I hadn't heard anything from anyone one way or the
other. 

> Yakov.

-- 
Jeff Haas 
NextHop Technologies

To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: Re: Issue 19) Security Considerations 
Date: Mon, 05 May 2003 09:59:14 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Jeff,

> Curtis wrote in a previous message:
> 
> On Thu, Apr 03, 2003 at 09:36:06AM -0500, Curtis Villamizar wrote:
> > --- draft-murphy-bgp-vuln-02.txt	Wed Mar  5 21:00:00 2003
> > +++ draft-murphy-bgp-vuln-02.txt++	Thu Apr  3 09:18:12 2003
> > @@ -149,6 +149,7 @@
> >  3.2.2.2 Timer events ..............................................   16
> >  4 Security Considerations .........................................   16
> >  4.1 Residual Risk .................................................   16
> > +4.2 Practical Considerations ......................................   16
> >  5 References ......................................................   17
> >  6 Author's Address ................................................   18
> >  
> > @@ -901,6 +902,79 @@
> >  Filtering is in use near some customer attachment points, but is not
> >  effective near the Internet center.  The other mechanisms are still
> >  controversial and are not yet in common use.
> > +
> > +4.2 Practical Considerations
> [...]
> 
> This looks like it has good merit.  Shouldn't we add this to the document?
> (Well, not "we" since Sandy is authoring it, but it seems like a good idea.)

I don't recall seeing any objections to adding this to the document.

Yakov.

To: Shankar Vemulapalli <[email-masked]>
Cc: [email-masked]
Subject: Re: draft-ietf-idr-bgp4-20.txt 
Date: Mon, 05 May 2003 06:22:13 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Shankar,

> Hi -
> 
> Not sure if this is already pointed out -
> 
> In draft-ietf-idr-bgp4-20.txt - page  14
>          [RFC2842] defines the Capabilities Optional Parameter
> and on page 85 -
>    [RFC2842] R. Chandra, J. Scudder, "Capabilities Advertisement with
>    BGP-4", RFC2842.
> 
> should be changed to newer RFC  - RFC3392 - to reflect the latest info.

Agreed.

Yakov.

Date: Sat, 3 May 2003 15:06:46 -0700 (PDT)
From: Shankar Vemulapalli <[email-masked]>
To: [email-masked]
Subject: draft-ietf-idr-bgp4-20.txt
Sender: [email-masked]

Hi -

Not sure if this is already pointed out -

In draft-ietf-idr-bgp4-20.txt - page  14
         [RFC2842] defines the Capabilities Optional Parameter
and on page 85 -
   [RFC2842] R. Chandra, J. Scudder, "Capabilities Advertisement with
   BGP-4", RFC2842.

should be changed to newer RFC  - RFC3392 - to reflect the latest info.

Thanks,

/Shankar

Date: Sat, 3 May 2003 01:05:20 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: On BGP and VPLS
Sender: [email-masked]

[crossposted to idr WG mailing list]

Alex Zinin writes:

>  More specifically, below I tried to put together a list of concerns
> I have about the approach described in draft-kompella-ppvpn-vpls,
> that I would like the WG to consider.

>  1. Use of the NLRI field

>    As an IP routing protocol, BGP uses the NLRI field to carry IP
> reachability information in the form of IP prefixes. Prefixes within
> the NLRI field are used for two main purposes in BGP: a) as the
> destination/mask pair in the routes installed by BGP in the routing
> table, and b) as a handle to an entry in the BGP RIBs.

>    The document in subject changes the semantics of the NLRI field
> quite substantially even when compared to 2547. First, all of its IP
> prefix-related properties are lost. There is no more IP routing, or
> any addressing information in it. Second, the notion of TLVs is
> introduced inside this field, which a) is not needed in BGP as an IP
> routing protocol, and b) because of its variable length property
> changes the nature of the NLRI contents, i.e., it's being a stable
> handle in the BGP database. To solve these problems the
> implementations would need to use only a part of the contents of the
> NLRI field as the handle used to index within the RIBs, and store
> the rest as attributes.

Alex,
This point seems to be predicated in the statement that "BGP uses the
NLRI field to carry IP reachability"...

It opens up a sort of philosophical discussion on BGP. This is of
course a highly subjective topic which is hard to quantify or to prove
by logical terms.

Allow me to present my personal view.

BGP is a particular implementation of an algorithm that performs non
looping database flooding distribution. That algorithm consists mostly
on the path vector (used both in ebgp and route reflection) plus route
advertisement rules. This is the publicly specified part of the beast.

However that ends up being about 10% of the database exchange
algorithm. Each implementation uses distinct algorithms to do the real
heavy lifting: the advertisement of database updates to its peers,
given that each peer is allowed to flow control and that the ammount
of information to be distributed is typically non-trivial compared to
the resources of the system.

None of the functions above actually do depend on the format of your
database records. As long as there is a primary key associated with
each record. Modern implementations, given that they are required to
handle 3/4 different types of records w/ different keys (ipv4, ipv6,
2547, 2547-for-ipv6, etc) will tend to treat these keys just as
database systems do: as a bit string without any semantics associated
w/ it.

Note also that the number of distinct tables exchanged in a 2547
implementation may be in the thousands. So segregation of which record
belongs to which table is necessarily a solved problem in practice.

There is one part of BGP that however interacts w/ the semantics of
the particular database being exchanged: route selection from the
Loc-RIB.

The Loc-RIB is by definition where BGP interacts w/ remaining users of
the database and it includes rules that are system specific.

As an exercise, if we take the existing spec and do:
s/route/record/g
s/IP prefix/key/g

Do we still have a document that makes sense.. ?

Except for the vague bits about aggregation, about which BGP itself
does little about, i would contend that the result would be pretty
much the same.

2547 which you cite is a particular good example, imho. A 2547 NLRI
ends up being used to create IP reachability information, but while it
is a safi 128 record, it is not IP reachability and it is not treated
as such.

>  2. Distribution of information

>    When used as an IP routing protocol, BGP distributes routes among
> all participating routers. Each router (PE or P using VPN
> terminology) is interested in _all_ routes received from its peers;
> it selects the best path for each prefix if multiple are available
> and installs it in it's routing table; the best paths are propagated
> further to other peers.

That is not the case w/ 2547. PE routers typically have interest in
only a subset of the routing information. They tend to do inbound
filtering in current network deployements but one can also do outbound
filtering in the RRs via either extended-community ORF or subsequent
improvements to ORF (draft-marques-ppvpn-rt-contrain).

P routers do not carry 2547 routing information.

>    The way BGP is used in the document results in a situation where
> information relevant only to a subset of routers (e.g. PW-specific,
> or VPLS-specific info) is sent to all PEs participating in the BGP
> domain. More than that, P routers, usually used as route reflectors
> in IP routing, end up storing all information while they are not
> using any of it locally.

RR in VPN deployments are typically not in the topology. My
understanding of the P-router term is that it is a transit node that
does not have VPN information.

>    Note also, that best path selection that is normally performed by
> BGP when it receives information about the same prefix from multiple
> peers, is not needed in the VPLS case, and (even if implementations
> decided to apply the same algo as in regular BGP) would just be an
> artifact.

Not really... i can advertise the same key from multiple sources in
L2VPNs also. All policy mechanisms do work... igp distance, etc. It is
just the semantics once the path is selected that are different.

As an example think working and protect PE for a given emulated
circuit (or lan).

>    The above exposes the difference between the routing nature of
> BGP when used for IP (where reachability info is distributed and the
> path properties are as important as the info itself), and its purely
> transport application in the proposal (where only the fact of
> information delivery is important.)

>    Interestingly enough, from the transport perspective, BGP, though
> reduces the number of sessions a given PE has to maintain (and thus
> the sender's complexity), introduces additional overhead from the
> receiver's perspective--if a PE router has multiple BGP sessions
> (which is normally the case), it will receive the same information
> more than once, while clearly a single copy is enough.

I don't know which model you have in mind but in a typical VPN
deployment scenario (l3 or l2/vpls/etc) a PE has 2 peering sessions to
a RR outside of the topology. The second copy of the information is
there for redudancy...

If a full mesh where used, only 1 copy would be present.

>  3. Aggregation of information for large-scale operation

>    When distributing information among a large number of systems, it
> is important to be able to aggregate information as it travels
> further ahead to ensure scalability of the system. In routing this
> is achieved by summarizing a set of prefixes and announcing them as
> a less specific prefix. For example, AS'es in the Internet do not
> exchange granular IP prefixes visible inside IGPs, but instead send
> each other aggregate prefixes via BGP.

>    It is not clear to me how, given the format of the NLRI field,
> VPLS information can be aggregated using the proposal in the
> document.

To give you an example, in JunOS aggregation is implemented as a
separate routing protocol... if i'm not mistaken the model is lifted
from 'gated'. Clearly the idea that aggregation may be a distinct
component from BGP has been around for a while.

VPLS doesn't really need aggregation although it does use an IGP :-)
PE to PE connectivity is performed indepently from the 'forwarding
distinguisher' advertisement (i.e. the inner label). Any or multiple
routing and singaling protocols may be used for this
functionality. Only the information exterior to the SP network
(service attachements) is carried through BGP.

>  The above gives me a very uncomfortable feeling that the proposal
> is stretching BGP to perform functions it was not designed for.

Any succesful protocol will be used for means other than it was
designed for. That is usually a sign that the designers got something
right.

Let me give you an example: BGP is currently used to block spam
propaggating networks/hosts. What this an original goal of the design
? Hardly. When used to block spam BGP does not advertise any valid
forwarding information for instance. And i'm sure it is a question of
time until we add port information to the record keys.

>  Below are some additional points that should be taken into
> consideration.
   
>  4. Backwards compatibility and SW upgrade requirements

>    Because the proposal suggests using a new AFI/SAFI combination,
> PE routers will not be able to announce VPLS information using the
> existing BGP infrastructure. All BGP speakers in a SP's network,
> including the P routers, will have to be upgraded with new SW,
> though information needs to be exchanged only among the PEs.

That is not an issue as we've seen above. The deployment model is
different from what you assume.

>  5. Coupling of VPLS and BGP SW

>    Putting VPLS-related functions in BGP leads to two unwanted
> consequences:

>     a) Lesser BGP code stability--bugs in the VPLS part of the code
> will likely affect parts of BGP used for Internet routing, thus
> increasing the chances of BGP failures in SP networks.  The same
> argument works in the opposite direction.

You have no basis to conclude that.

Any modern BGP implementation worth its salt consists of
AF-independent code + AF-specific code. The fact is that you can
implement VPLS without touching the AF-independent code.

Any line of code change that you make to an implementation as the
potential to introduce bugs... 

>     b) Potential dynamic effects--since with a BGP-based approach,
> VPLS- and routing-related processes are likely to share the same
> internal router resources (such as timers, threads, locks/mutex'es,
> queues, memory pools), dynamics of the VPLS system are likely to
> influence the dynamics of the routing- related functions and vice
> versa. The larger the overlap between the two systems, the higher
> are the chances of such interference.

I'm sorry but this is just FUD.
All router implementations do have some level of resource sharing
between completly unrelated features. In some of them, all
functionality shares all resources.

Don't want BGP sharing timers w/ X.25-over-TCP... disable one of them.

>  My recommendation would be for the WG to consider these points.

The way i see it there is an high likely-hood of this turning into an
"Yes, it is" "No, it isn't" discussion. And I'd really like to avoid
that.

A question to you and to the WG(s) in general:

- What are the main concerns that you have w/ the generic database
exchange view of BGP (Lets call it the "Basically General Purpose"
theory).

- Can we have a reasonable discussion about the best engineering
approach to provide database exchange services for
routing-related-applications without getting into a religious argument
about "2547 is evil" ? i.e. can we try to separate how highly each one
of us rates the actual application from this discussion ?

- I believe one of the preconditions for a resonable discussion is to
realise that implementors are the most interested people in not
introducing regressions to shipping code. They actually get to fix it
after being screamed at for a considerable lenght of time.
I'd really like to get past the "you can't implement a feature i don't
want because your are going to break the code" kind of discussion.

- Are we going to have a similiar discussion about LDP ? LDP is not
any less relevant for network stability nor a protocol which is any
simpler than BGP (if anything the level of complexity is higher given
that LDP has all the db exchange problem of BGP + a non trivial
ammount of issues of its own).

regards,
  Pedro.