Subject: RP overloading: main question wrt code impact and resource sharing

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 17:22:48 -0700

Inspired by Robert's posting, I think we can formalize the concerns
about impact on the running code and resources sharing as follows.
(Note that there are other issues under consideration too.)

What essentially we are talking about is the most comfortable location
of the demux point among the apps. We have two major scenarios:

The BGP-based model: in this case, there's a single demux point for
routing and non-routing apps, and it is at the AFI/SAFI
parsing/encoding stage. All apps are explicitly exposed to BGP.

The non-BGP model: in this case, BGP is not exposed to non-routing
apps and the demux point for them, as well as for BGP as a whole, is
at the TCP level.

It seems that a more comfortable feeling about the non-BGP model
(again, only from the code impact and resource sharing POV) is based
on the following assumptions:

  Code impact:

   1. The code below the demux point (i.e. TCP, IP, kernel) that
      affects all apps is not changed when a new app is added.

   2. The code in existing apps does not have to be changed when a
      new app is introduced, i.e., the apps are sufficiently decoupled

   3. The code in separate apps may be developed, tested, released,
      debugged and packaged independently from other apps.

  Resource sharing:

   1. Separate apps do not share a single transport connection, which
      implies that:

         a) they don't compete for socket buffers, and hence
            don't block or starve each other at the TCP level
            
         b) they don't compete for possible protocol-level transport-
            related buffers and queues, and hence don't starve or
            block each other at the protocol send/receive level

         c) they don't compete for a possible per-connection
            processing time budget, hence don't starve each other
            at the intra-process scheduling level

         d) they don't compete for resources within the network
           (e.g., BW), when the protocol session spans multiple hops
            
         e) they likely do not compete for resources in other
            parts of the local system that may be protocol-aware
            (such as line cards putting all BGP segments in a separate
            rate-limited queue)

   2. Separate apps are likely to not share a single scheduling context,
      which implies that they will not compete for the same per-process
      time budget.

   3. Separate apps are likely to have their memory segments protected
      from each other in the OS'es supporting this. Hence, the risk of
      non-routing apps corrupting routing data is decreased.

   4. Separate apps are likely to have their own pools of resources,
      such as memory heaps, so the risk of memory starvation is
      decreased.

   5. In OS'es supporting protected memory, the above results in fate
      separation.

   Shout if I forgot something...
      
If we take the BGP model and set the goal to achieving the same fuzzy
feeling, here's the type of SW engineering discussion we would have
(please feel free to comment and correct):

>   Code impact:
> 
>    1. The code below the demux point (i.e. TCP, IP, kernel) that
>       affects all apps is not changed when a new app is added.

possible with proper abstraction and AFI/SAFI API registries.

>    2. The code in existing apps does not have to be changed when a
>       new app is introduced, i.e., the apps are sufficiently decoupled

should be the case for AFI/SAFI's already

>    3. The code in separate apps may be developed, tested, released,
>       debugged and packaged independently from other apps.

theoretically possible, hardly expected in reality...

>   Resource sharing:
> 
>    1. Separate apps do not share a single transport connection, which
>       implies that:

we can have a separate session per AFI/SAFI

>          a) they don't compete for socket buffers, and hence
>             don't block or starve each other at the TCP level

true with the above

>          b) they don't compete for possible protocol-level transport-
>             related buffers and queues, and hence don't starve or
>             block each other at the protocol send/receive level

ditto

>          c) they don't compete for a possible per-connection
>             processing time budget, hence don't starve each other
>             at the intra-process scheduling level

ditto

>          d) they don't compete for resources within the network
>            (e.g., BW), when the protocol session spans multiple hops

ditto

>          e) they likely do not compete for resources in other
>             parts of the local system that may be protocol-aware
>             (such as line cards putting all BGP segments in a separate
>             rate-limited queue)

this one may not be true, since rate limiter still thinks it's BGP

>    2. Separate apps are likely to not share a single scheduling context,
>       which implies that they will not compete for the same per-process
>       time budget.

achievable with proper decoupling, and APIs.
according to Robert folks are working on this.

>    3. Separate apps are likely to have their memory segments protected
>       from each other in the OS'es supporting this. Hence, the risk of
>       non-routing apps corrupting routing data is decreased.

possible, though IPC is likely to be a performance hit
don't know if anyone is looking at this.
given the well-known SW architectures of some vendors,
we're far from there...

>    4. Separate apps are likely to have their own pools of resources,
>       such as memory heaps, so the risk of memory starvation is
>       decreased.

doable with per AFI/SAFI resource pools.

>    5. In OS'es supporting protected memory, the above results in fate
>       separation.

hard without protected memory

So, from the code impact and resource sharing perspective
_exclusively_ (other aspects aside), it is the question of whether we
feel more comfortable with explicit, "manual" code and resource
separation that the BGP model requires, or with implicit, natural one
that the non-BGP model encourages, though does not guarantee.

Notice that we're now deep into SW engineering details. When
commenting, please be careful to make sure we don't degrade into
vendor bashing.

Regards

Alex

_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion