Subject: Re: [Anima-signaling] GRASP issue 51: Flooded objectives

From: Toerless Eckert <[email-masked]>
Date: Mon, 1 Aug 2016 01:55:29 -0700

Inline.

On Mon, Aug 01, 2016 at 08:47:22AM +1200, Brian E Carpenter wrote:
> Hi,
> 
> Prior to raising this on the WG list, I'd like feedback from the design team.
> 
> Following side discussion with the bootstrap and ACP teams in Berlin, there
> is a new issue about flooded objectives (not really about the Flood message
> itself). It has two aspects:
> 
> 51.1 Should flooded objectives have a time-to-live before they are deleted from
> the flood cache? I think the answer is no, because in any case the source can repeat
> the Flood message if the value changes. If anybody sees a use case for a TTL,
> please explain it.

We discussed in Berlin methods to discover sources that went away:

a) IMHO, we do not want to send rapid updates from sources to discover quickly when
   they die because of the absence of those announcements. (aka: no fast aliveness).
b) IMHO we do not want to work out the details of the GRASP infra quickly detecting
   when a sources goes away and flood that information (no IGP like fast source revocation).
c) I suggestested that GRASP clients could help:
   - GRASP thinks source is alive (long or no TTL)
   - GRASP API client tries to connect to objective, and fails
   - GRASP API client tells GRASP "objective seems dead".
   - GRASP removes objective source from cache
   - When GRASP receives another flood message from objective source,
     it repopulates cache and tells client.

With c) in mind, we might not need a TTL, but I am not aware of existing service
announcement mechanisms that tried to rely on this "c)" approach, so i am a bit
weary to solely rely on it:

When there is a network disconnect between objective and client, the client would not
be able to reach the objective and the objective would be declared dead. Aka: If we
do "c)", we do need to periodically flood from the objective source the announcement
to recover from this situation. The periodicity should be determined by the source
itself. But it would be good if the source also announced this periodicity.

Aka: I would suggest that intead of TTL, the source could simply announce the periodicity
of it's announcements (in seconds). Default 30 seconds.

If a client does not perform "c)", then it can use the periodicity value to
calculate when it can declare the source dead - eg: after three periods non-receipt
of a message.

There is also a fundamental limitation with "c)", and that is when the announcer
is not tightly coupled with GRASP:
   - GRASP could announce the objective, but the process providing the objective
     could be running on a different system and could have died. Or maybe this is
     even possible on the same system.
   - Client connecting to the objective  would declare it dead to GRASP, but
     GRASP would continue to see announcement from the source.

Ideally, we would not have this case, aka: GRASP API should always know when
the actual source of the objective is really alive. But i am not sure if it's possible
to always guarantee this. So it would be good to have in the flooded objective
some indiciation that the GRASP announcement is not coupled to the objective aliveness
itself.

I am extrapolating this case from what we do with mDNS: Most services today do
not couple to mDNS via API. Instead when we use something like Avahi, then there
are Avahi config-files that describe the services running on the same system (or
on another system). Which of course does not guarantee that each of those services
is actually working fine. So i't seasy to announce services when they do have
problems (eg: can't start, fail, ...).

Aka: objective parameter "GraspObjectiveCoupled" (or come up with a better name).
If true, it means that the GraspSource and Objective Owner feel that it's safe
to do c) on the client side to discover death of the service, and false means it is not.

> 51.2 We do have a potential use case where more than one source floods the same
> objective, and the recipient ASAs need to distinguish the results. That use case
> is when there are multiple bootstrap proxies advertising on the same LAN, and the
> joining nodes (pledges) can choose which one to try. I'm sure there will be others.

The most important one being multiple registrars announcing the registrar service,
and the bootstrap proxies want to discover them, and then choose the best one based
on network distance (aka: remaining loop count).

> My proposal is to specify that a node receiving a Flood message will cache
> the received objective plus its source locator (which is already in the message
> for other reasons). We'll also have to upgrade the API so that the recipient
> ASA can get multiple flooded answers.

Yes.

Thanks!
    Toerless
> 
> Comments?
> 
> Regards
>    Brian
> 
> _______________________________________________
> Anima-signaling mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/anima-signaling

-- 
---
Toerless Eckert, [email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 5 Aug 2016 15:24:43 +1200

So, here is where my head is now on flooding.

1. We specify that the source address is cached with the flooded objective, and that
multiple copies are kept if there are multiple sources.
2. To cover the common case, if a regular ASA calls synchronize(obj) it will
by default get back the first entry in the flood cache, and if there is nothing
in the flood cache, GRASP will discover and synchronize the objective.
3. To cover the case where we want multiple flooded values, we'll add a function
to the API called something like get_flood(obj) which will return a list
of versions of the objective, each tagged with its source address.

Text? I have put some words into my working version of the GRASP spec.
I will push the latest XML to GitHub shortly.

Running code? Yes, I've coded all that in my prototype and tested all the code
paths. (I don't intend to publish that code until we get a -07 draft out.)

In terms of freshness, here's some text:

   If the value of an objective has time-sensitive properties such as a
   defined lifetime, a defined expiry time, or a defined refresh rate,
   these SHOULD be expressed as part of the definition of the objective
   itself; they are not foreseen in the GRASP protocol.  For example, an
   objective that is flooded using Flood Synchronization might include a
   parameter specifying its expiry time as a UTC string.

Regards
   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 5 Aug 2016 01:34:57 -0700

To validate the proposal:

Can you write down pseudo-code for registrar discovery, eg:

  code for registrar ASA:
   API-call(s) and JSON parameters

  code for bootstrap-proxy ASA:
    API-call(s) and JSON parameters

  I would primarily like to validate that the parameters we brainstormed
  in Berlin will be carried and how they are put into the API,
  specifically:

     loop-count, remaining-loopcount
  
  And just for the fun of it, i would love to have in the example
  the parameters "prio", "weight" because we could show then how
  we could map mDNS service announcements to GRASP - because prio and
  weight are the two standard mDNS service parameters.

  Eg: algorithm on bootstrap proxy:

> multiple copies are kept if there are multiple sources.

Yes, but "source" really must be the ASA transport address, aka: 
   source = {ACP-ULA-IPv6-address, proto=TCP, ASA-GRASP-TCP-port}
  
Example why:

   I want to support "ISSU" - In-service-software-upgrade without interruption,
   meaning: i have version 1 of the ASA running, i now want to change to
   version 2 of the ASA without ever having service interruption.

      - Start version 2 of ASA
      - Version 2 ASA calls API to announce its objectives
        these are likely the same objectives as those from version 1,
        so we now have for a while 2 ASA on the same host (same ACP-ULA IPv6-address)
        that announce the same objective(s). So we need those transport
        addresses as "source" to distinguish them.
      - Version 2 ASA indicates to version 1 of ASA: stop service
      - Version 1 ASA calls API to stop announcing its ojectives
      - Version 1 ASA waits until it has finished all open services
        (eg: Registrar-ASA: all ongoing EST connections).
      - Version 1 ASA (optionally) indicates to Version 2 ASA
        that it's now done. exists
      - lots of variations. For example, instead of stopping announcement
        immediately, Version 1 could first announce with lower priority
        (when there is priority).

> in the flood cache, GRASP will discover and synchronize the objective.

a) Whether or not to do discovery&sync if the flood-cache is empty should be
a decision of the ASA, not a hardcoded GRASP function. In general it would
be a logic defined by the objective. Eg: For objectives with a dense set
of clients, i would define to not do it. Instead, a callback-API would
be nice so the client will get an API callback when the first entry shows
up in the flood-cache.

> of versions of the objective, each tagged with its source address.

Yes. It would be nice to have example (pseudo?) code to vet if the
API is complete. Eg: registrar announce & discover with functions as we discussed.

  Registrar-ASA:
    # get TCP port for EST
    s = socket(TCP,...)
    bind(s, ACP-ULA-address)
    est-port = getsockaddr(s).port

    API-call grasp.flood-syn("AN registrar") with parameters
      - loopcount = orig-loopcount = maxloopcount
      - service-address = { EST-v1, est-port } // details TBD
      # Even if we decide we do not want prio and weight, i would love to
      # use them in the example, because prio/weight are the mandatory DNS-SD
      # parameters, so it would be good if we showed how easy we could translate
      # them into GRASP:
      - weight = n
      - prio = m
  
  Bootstrap-proxy-ASA:

   try-to-connect:

     bestprio = maxprio+1
     bestdistance = maxdistance+1
     bestsources = {}
     forall instance in grasp.get_flood("AN registrar")
         instance.distance = instance.orig_loopcount - instance.loopcount
         if instance.distance = bestdistance
             if instance.prio = bestprio
                 bestsources += instance
             if instance.prio < bestprio
                 bestprio = instance.prio
                 bestsources = instance
         if instance.distance < bestdistance
             bestdistance = instance.distance
             bestsources = instance
     totalweight = 0
     forall instance in bestsources 
         totalweight += instance.weight
     weight = rnd(0,totalweight)
     forall instance in bestsources 
         weight -= instance.weight
         if weight <= 0
             break instance
    
     if instance.service-address = "EST"
         addr = instance.addr.ipv6addr
         port =  instance.service-address.est-port
         open TCP socket to destination {addr, port}
         if connection_timed_out
             # ASA seems to be dead. Do API call to grasp to declare this
             # ASA objective dead:
             grasp.delete_objetive_source(instance)
             goto try_to_connect.
         else
             # continue with EST

     else
         HELP, unsupported protocol

> I will push the latest XML to GitHub shortly.

Great.

> paths. (I don't intend to publish that code until we get a -07 draft out.)

Great.

>    parameter specifying its expiry time as a UTC string.

If the announcer of a flood sync announce dies, how would its cache entries
ever be deleted ? On nodes that have an ASA interested in the objective,
the ASA can look at expiry time and tell GRASP to expire the entries. But
how about nodes without such ASA ?

  Registrar is a bad example, because every node should run a bootstrap-proxy.
  But think for example ASA that are only required by nodes connecting to hosts.
  Only autonomic access-switches would be interested in those objectives,
  not any of the intervening core/distribution routers. But those would still
  need to have caches to make flooding work correctly. Nut they wold love
  not to maintain obsolete months old objectives that are stale.

Second discussion point:
A node without prior state (sleeping or booted) becomes active, it wants
to know  specific objectives that are only flood synchronized. How does it
do that ?
  a) Wait until it's periodically repeated
  b) Multicast a discovery to speed up discovery.
     This can be IMHO quite complex: The most simple option is
     b.1) The discovery is flooded network wide without taking
          cache entries into account. Every node that has the objective
          and sees such a discovery request will trigger a new 
          flood discovery. So the client would then need to
          convert to examine after a while the flood discovery cache.
     b.2) If there are a lot of periodically sleeping/waking up
          nodes then this will be inefficient. so we would need to
          take the cache into account and stop flooding of the
          discovery once we have cache entries - and generate new
          flood sync messages only to the link with the just woken-up
          device.

b) is complex. So IMHO the easiest approach right now is to use a) and
make ASA periodically repeated flood sync.

Aka: expiry_time looks to me like a logical GRASP header parameter so that
we can GRASP without ASA expire flood sync cache entries.

Cheers
    Toerless

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 6 Aug 2016 14:39:35 +1200

Hi Toerless, just drilling down on two points (I have not ignored the rest,
but these condition other aspects):

On 05/08/2016 20:34, Toerless Eckert wrote:

...
>    source = {ACP-ULA-IPv6-address, proto=TCP, ASA-GRASP-TCP-port}

Here I have a little problem. Unless you plan to use GRASP messages for
the entire process, it isn't GRASP's business what protocol and dynamic
port are to be used. So I think the protocol and the port need to be
fields in the flooded objective itself, not derived automatically by
GRASP. Does that make sense?

...
> how about nodes without such ASA ?

Is it a problem? The useless flooded objective would sit there until the
cache overflows and it gets deleted by the LRU algorithm.

...
> we can GRASP without ASA expire flood sync cache entries.

We can do that. It's a choice: extra bytes in every flood message versus
no default expiry mechanism, just LRU cache expiry.

I'll review all your points after we clear these ones up.

Thanks
    Brian

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 8 Aug 2016 07:03:26 -0700

Inline

On Sat, Aug 06, 2016 at 02:39:35PM +1200, Brian E Carpenter wrote:
> port are to be used.

I do not parse that sentence. Can you rephrase ?

> So I think the protocol and the port need to be

We need to be able to distinguish in the caches announcements for the same
objective from two different ASA on the same autonomic node. Therefore the
"source" needs to have another distinguisher beside the IPv6 address of the
autonomic node. This is not an issue that the individual ASA can solve becaue
the cache is kept by GRASP.

The distinguisher can be any ID, but i proposed to use the ASA's TCP port number
through which it runs GRASP becaude this ID does already exist and is unique.
So no new ID-allocation code to write.

This TCP port number will NOT be used to construct the transport port of any
service connection!

Example:

    Registrar ASA objective announcement:
        source = { ACP-ULA-IPv6-address, proto=TCP, Registrar-GRASP-TCP-Port }
        ....
        Payload:
        EST-adress: { proto=TCP, port=<Registrar-EST-port> }

Registrar ASA starts, it gets a TCP port for its GRASP unicast messages
GRASP puts that <Registrar-GRASP-TCP-Port> into the source of all Objectives from this ASA.

Then the Registrar ASA opens a TCP server port for the EST protocol. Then it
announces the objective.

The Bootstrap-Proxy constructs the transport address of the EST connection
by combining the TCP port from the payload EST-address with the IPv6 address
of the source. The TCP port of the source is only used to send/receive GRASP
messages from/to that ASA - which happens by the GRASP library. 

And of course, this could be a TCP or UDP port number. I personally think TCP
is easier even if there is more overhead because i don't need to bother about
fragmentation (for GRASP).

> cache overflows and it gets deleted by the LRU algorithm.

Normally, U) means "used", and i think we can not do this:
Check my algorithm for a bootstrap proxy finding the best registrar. It will retrieve
from GRASP all those registrar cache entries, so you can't figure out which one
it "Uses" without additional API. The only thing the currently planned APIs
would give you is an indication when the client ASA tries to use an objective
and it's broken. But it will try only what it thinks are the best. Not the others,
so they won't get good LRU treatment.

We need to have a fairly high frequency of periodic flood-sync messages anyhow.
Else a newly rebooted device will not learn them well. Or we need to make
the protocol more complex by generating additional triggered flood syncs for newly
booted devices. Aka: one flood sync every 30 seconds . Then it makes a lot of sense
to also time them out.

> no default expiry mechanism, just LRU cache expiry.

I'll vote for extra bytes.

Cheers
    Toerless

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 9 Aug 2016 13:22:48 +1200

Thanks Toerless, this is a very useful discussion in terms of detecting
unspoken assumptions, so I have to continue with more questions:
> I do not parse that sentence. Can you rephrase ?

How can the GRASP library and code know what protocol and port the ASA intends
to use for non-GRASP messages? It doesn't create the sockets for that, only
the ASA can do so. So in this case, if the ASA wants to talk over CoAP or HTTPS,
GRASP doesn't know anything about it. (I'm looking at
https://tools.ietf.org/html/draft-ietf-anima-bootstrapping-keyinfra-03#section-3.2)

> objective from two different ASA on the same autonomic node.

Now here's another hidden assumption. The GRASP spec currently says (in the
security considerations):

   - Authorization and Roles

      The GRASP protocol is agnostic about the role of individual ASAs
      and about which objectives a particular ASA is authorized to
      support.  It SHOULD apply obvious precautions such as allowing
      only one ASA in a given node to modify a given objective, but
      otherwise authorization is out of scope.

I have been working on the assumption that only one ASA is allowed to
register itself to manage a given objective in a single GRASP instance.
'Manage' means being a source of synchronization or flooding or being
a negotiation responder. Why? Because otherwise, other nodes will see
inconsistent behaviour by the node in question.

I don't understand an application scenario where it's useful to have two
ASAs managing the same objective in the same node. Why would you allow
two Registrars in the same node, for example?

But in fact, as far as I can see, GRASP discovery could work in such a
case, returning two different discovery responses from the same node, with
different GRASP-managed ports. (I suspect my code would already support that.)
That would allow synchronization or negotiation independently with either of
the ASAs. Hence the SHOULD in the above extract.

However, if we also allow two different ASAs in the same node to flood the
same objective, that does require some extra tagging, as you say below.
So I want to be very clear about it. Why do we want to do that?

Since it would be a protocol change, we would need to present it to the whole WG.

> Therefore the

Fully agree about TCP. I haven't even coded the UDP option ;-)

> Normally, U) means "used",

Right. It needs to be deleted by a Least Recently Updated algorithm!

> and i think we can not do this:

The alternative is a fixed lifetime but I don't like that for flooded
objectives, because some of them (e.g. Intent) might have infinite validity.

> I'll vote for extra bytes.

If we're happy that flooding is relatively rare, that seems OK. Of course that's
another protocol change to be discussed with the whole WG.

Rgds
   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 8 Aug 2016 23:39:09 -0700

Inline

> two Registrars in the same node, for example?

I gave that two registrars example in an earlier mail: semless software
upgrade/downgrade:
  - v1 of registrar running
  - start v2 of registrar. It sends message (eg: GRASP) to v1 instance
  - v1 withdraws its GRASP objective announcements but continues
    to accept EST connections
  - v2 instance starts sync flood of objective.
  - After a while v1 has no ongoing EST connections, and hopefully
    we also will have expiry of flood objective, so eg: after it has
    no active EST connection AND 3 * timeout of flood-objective, v1 can
    declare itself done. Maybe signal to v2 instance for purpose of
    diagnostics - o am done - and quit.

Maybe i will also have two versions of registrar run in parallel when
we come up with a different protocol beside EST for example. Those of
course would run in parallel. And in another mail i did have the pseudo-code
for the client-code searching for best objective. And on that side it 
doesn't make a difference whether different version/type(eg: protocol)
of an objective are running on the same autonomic node or different ones.
But for deployment, it is a lot easier if you have to bother about
variety of ASA only on a single node.

ASA driver for some type of service with different
models, eg: printer driver. Each printer has different printer driver,
but maybe the objective is just "printer". 

Parallelization. Ok yes, you should write high performance ASA
multi-threaded. But maybe its easier not to and just have a cluster of
these ASA on a node. Thats even how today a lot of services are scaled.

What you are trying to do is "exclusivility". Which is a nice option.
But it shouldn't be the mandatory base for all objectives. If we have
a good enough example for exclusivity, it should be added to the API,
aka: it would have to be something GRASP library tries to guarantee.

> the ASAs. Hence the SHOULD in the above extract.

Right. I would rephrase this like: "ASAs supporting specific objectives
may want exclusivity for that objective, eg: no other ASA on the same
autonomic node offering that objective. In that case, the GRASP library needs
to support approproate API for ASA to offer this option. Such API
options are outside the scope of this spec".

> Since it would be a protocol change, we would need to present it to the whole WG.

Sure.

> objectives, because some of them (e.g. Intent) might have infinite validity.

My concern is to get dead instances of objectives quickly out of the world and
to have a simple and known stable update mechanism that scales well. Which
is periodic broadcast (aka: multicast) for these.

When we announce the lifetime in the GRASP flood objective, then it could be
very long. The issue making it shorter is primarily reliability. When we
use link-local-multicast, there is no guarantee that it will be received
by all nodes. I would actually prefer to use the ACP TCP connections also
for the flood-sync inside the ACP to make them reliable. One for each ACP
neighboring autonomic node. No UDP multicast.

And of course we can change easily from TCP to CoAP if need be (that's also reliable
AFAIK). But as Carsten reminded me: Even in a TCP connection, the sender has
no guarantee that a receiver has received a message unless it also sends some
payload acknowledgement ("500 OK" ;-). Which i think you have not defined for
GRASP messages, have you ? 

If we'd be using CoAP, then CoAP itself provides the ACK of messages ;-)

So, if we assume hop-by-hop reliable flood-sync, we could use very long
time-to-live. Could even be 0 = never expires.

BUT: This does not solve all problems:

What happens with a node newly rebooting. How does it get those existing
flood objectives ? 

     backbone/registrars - AN1 - AN2 - AN3 - AN4 - AN5
                                       |-------------|

Power outage in building with AN3,4,5. AN4,AN5 immediately recover after power1,
AN3 power supply blows, takes 8 hours longer to be brought back online.

How will AN3, AN4, AN5 get intent after AN3 comes up ?

AN3 i guess could explicitly query objectives from neighbors and will ege
get sync of objectives from AN2. But would AN4, AN5 continue for 8 hours
to do the same ? 

And lets imagine an objective that only edge-AN-nodes need, eg: AN5. For
those objectives not needed on every node, it gets even trickier.

Aka: periodic, low-frequency broadcast is a proven stable solution. Its
not the nicest. It may not be the only one we want in GRASP, but i'd
feel a lot safer if we could make that option work.

One solution for above problems:
  When AN3 comes up, it does send some GRASP query with a hop-count of 1
  that's specifically asking for content of all neighbors flood-sync
  cache. YOu hve to figure out how to craft that message (eg: IMHO
  it's a special query because it's for all cached objectives).

  When AN3 receives replies from AN2, those replies must have the same
  loopcount/remianin-gloopcount as when AN2 first resent the flood-sync
  from eg: a registrar in the backbone.

  So, AN3 then simply continues to flood-sync those replies it receives
  as if they where "normal" flood sync it receives.

Once this is defined and in our review we think it works reliable,
we can reduce frequency of retransmissions for those objectives
that we think are very long-lived. But we will still be left with
the issue of withdrawal...

Example: Registrar announces lifetime of LONG. WHole network learns it
AN3 goes down, AN4, AN5 stay up. Registrar is replaced by other registrar.
AN3 comes up. AN4, AN5 will still have ld registrar flood sync cache
entry - which will not time out for a LONG time.

This is something like state of the art problem. I have ideas how to
fix this, but not sure we should go to tht trouble initially, because
it gets quite complex.

Maybe rather look into what we specifically would need for intent to
vet how far we want to drive GRASP:

Do you have any thoughts on what Intent would be for grasp ? A
specific objective ? Announced by which autonomic node ? ...
> another protocol change to be discussed with the whole WG.

Sure. Maybe we put together an update of the discussion on the list ?o

Cheers
    toerless

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 10 Aug 2016 13:07:55 +1200

On 09/08/2016 18:39, Toerless Eckert wrote:
...
> Maybe we put together an update of the discussion on the list ?

Yes. We're doing exactly what we are supposed to: confronting
the design of GRASP with details of some of the use cases.
I will draft something - but there's quite a lot of meat
here so it will take a bit of chewing.

   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 9 Aug 2016 23:41:03 -0700

Let me know if a higher bandwidth connection helps. I am currently
in germany time zone and could set up a webex.

Cheers
    Toerless

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 Aug 2016 09:04:31 +1200

Hi Toerless, and design team,

Here's a draft message to the WG. Comments? (It doesn't cover everything
in the thread - when we get these points settled, there are other points
we need to come back to.)

<draft message>

Hi WG,

This rather long message expands on GRASP issue 51, which is summarised as:

51.  Should flooded objectives have a time-to-live before they are
     deleted from the flood cache?  And should they be tagged in the
     cache with their source locator?

This has led to a complicated discussion in the signaling design time (mainly
between me and Toerless) and we would like feedback from the WG before taking
any firm decisions.

Breaking it down into separate issues:

the flood cache?

Cutting a complex discussion short, it seems that we will surely have use cases
where a time-to-live is needed. It might be infinity, of course, but we might also
want to be sure that a flooded objective vanishes after 30 seconds. To do that, we
need to add a TTL field.

Q1: Does the WG agree to add a TTL to the Flood message?

51.2 We do have at least one use case where more than one source floods the same
objective, and the recipient ASAs need to distinguish the results.
(That use case is when more than one Registrar or Proxy announces its availability
via a flooded objective, so that joining nodes don't need to send an insecure
Discovery message during bootstrap.) So the flood cache needs to record the
source of the flooded objective.

51.3 That leads to another complication. Suppose that more than one ASA in a given
node floods the same objective.

Side issue: I (Brian) had been working on the assumption that only one ASA in
a given node can manage a given objective. "Manage" means being willing to act as
a negotiation responder, a synchronization responder, or as a source of flooding.
But in fact, although there might be a reason for that limitation in a particular
use case, GRASP as a protocol could perfectly well handle several ASAs managing
the same objective. [My prototype implementation cannot really do so, but that's
an implementation choice.]

Return from side issue: If more than one ASA in the same node floods the same
objective, recording the source (as mentioned under 51.2) is not enough. We would
need to separate multiple instances from the same source node. There is a use
case for this too: if we want to make a soft changeover from version N to version N+1
of a given ASA, where version N is not switched off until all current sessions
end, but all new sessions will involve version N+1.

This will not cause any protocol changes for GRASP discovery (which already
returns a locator as a 3-tuple {IP address, protocol, port}. Therefore,
it won't cause any changes for synchronization or negotiation, which are
based on discovery results.

But it does mean that to distinguish flooded objectives from each other, we will
need to tag them too with a 3-tuple {IP address, protocol, port}.

Q2: Does the WG agree to add a locator 3-tuple to the Flood message?

When considering these additions to Flood messages, which add a few bytes
of overhead, please consider that for most use cases, the frequency of
flooding would be quite low.

</draft message ends>

Regards
   Brian

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 11 Aug 2016 14:08:35 -0700

Thanks, very well stated.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 21 Aug 2016 10:41:08 +1200

Design team,

So, assuming that silence from the WG means assent, I will propose
a change to the Flood message format that adds an optional TTL field
and an optional locator field (which can be the same format as
in a Discovery Response).

While I'm there, should I add an optional TTL field to the Discovery
Response too? It would make the Flood and Discovery Response messages
essentially identical, and if it's optional there will be no
unnecessary overhead.

Regards
   Brian

On 12/08/2016 09:15, Brian E Carpenter wrote:
>    Brian + signaling design team
>

---

From: Michael Richardson <[email-masked]>
Date: Tue, 23 Aug 2016 23:17:22 -0400

> essentially identical, and if it's optional there will be no

I think this is a good idea.

-- 
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-                  (Camping this week!)

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 26 Aug 2016 08:50:48 +1200

> I think this is a good idea.

Yes, I think it's better to be consistent. Also, my experiments with abusing
the prototype code (by forcing an address change while the code was running)
showed that a TTL on discovery results is highly useful.

Considering how to make the messages easy to parse, I decided that making the
TTL optional was a bit clumsy. As far as the user goes, it can be optional by
defining a default.

So, suggested changes in the CDDL syntax for the messages:

OLD:

message /= response-message ;response to Discovery
response-message = [M_RESPONSE, session-id, initiator,
                   (+locator-option // divert-option), ?objective]

message /= flood-message
flood-message = [M_FLOOD, session-id, initiator, +objective]

NEW:

message /= response-message ;response to Discovery
response-message = [M_RESPONSE, session-id, initiator, ttl,
                   (+locator-option // divert-option), ?objective]

message /= flood-message
flood-message = [M_FLOOD, session-id, initiator, ttl,
                (locator-option / []), +objective]

ttl = 0..4294967295 ; in milliseconds

I have running code for the above.

Comments? My next step would be to make corresponding changes to the draft.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 30 Aug 2016 13:57:57 +1200

Hi Toerless,

Catching up on your issues. I think most of them are covered by
adding the TTL and locator info to Flood. A couple of points remain:

> When we

Actually this gets back to something I was beating Michael (Behringer) up
about. I would love it if the ACP could do this. From a coding point of
view it would be nicest if the ACP could catch packets sent to a
LL multicast socket (by behaving like a virtual link) but that's a detail.
It would be OK too if the ACP simply provides functions to call instead
of sendto(msg,0,(ALL_GRASP_NEIGHBOR_6, GRASP_LISTEN_PORT)) and the
matching recvfrom().
...
...
> feel a lot safer if we could make that option work.

Yes, and it's easy - because the sending ASA can just do it, at a
frequency that suits the particular objective. Also, a node that
doesn't want to wait for the next flood can do a standard GRASP
unicast synchronize action.

> But we will still be left with

Like DNS TTLs. If you know in advance that something should be
withdrawn soon, you refresh it with a short TTL. But if there's

> entry - which will not time out for a LONG time.

When a client (a proxy in this case) tries to connect to the old registrar,
there will be a hard socket failure. So the client needs to look for
another registrar by repeating discovery or waiting for a new flood.

But looking at sone of the pseudocode you sent earlier in the thread,
I added a new call to my Python API:
 expire_flood()
# Mark a flooded objective as expired
#
# This is a call that can only be used after a preceding
# call to get_flood() by an ASA that is capable of deciding
# that the flooded value is stale or invalid. To be used
# with care.

It needs 8 lines of python :-). It's only local; to expire
something globally would need a new GRASP message type, but
wouldn't actually be hard to do if we wanted.

Regards
   Brian

---

From: "Michael Behringer (mbehring)" <[email-masked]>
Date: Tue, 30 Aug 2016 08:56:30 +0000

Sorry for long silence. Yes, vacations and business travel... 

> about. I would love it if the ACP could do this. From a coding point of view it
> would be nicest if the ACP could catch packets sent to a LL multicast socket
> (by behaving like a virtual link) but that's a detail.
> It would be OK too if the ACP simply provides functions to call instead of
> sendto(msg,0,(ALL_GRASP_NEIGHBOR_6, GRASP_LISTEN_PORT)) and the
> matching recvfrom().

Hesitating here. Our current pre-standard implementation is not implemented as a node-wide service, but as a set of interfaces. It's pure plumbing, essentially. In that, you'd send a packet to the ACP interface to node x, rather than to "the ACP" as a whole. That doesn't mean we couldn't provide an API that does what you describe, of course. 

Bottom line, the translation from the high-level "send packet to all neighbors" to mapping to specific interfaces needs to happen somewhere. We could create a more "clever" ACP which hides the actual interface and provides a higher level interface. But then we'd have two "clever" state machines, one for GRASP, one for the ACP. 

Wouldn't it be easier if GRASP could map to specific interfaces, such that we only have one such beast? And that the ACP could remain just "plumbing"?  

> ...
> > What happens with a node newly rebooting. How does it get those
> > existing flood objectives ?
> ...
> > Aka: periodic, low-frequency broadcast is a proven stable solution.
> > Its not the nicest. It may not be the only one we want in GRASP, but
> > i'd feel a lot safer if we could make that option work.
> 
> Yes, and it's easy - because the sending ASA can just do it, at a frequency that
> suits the particular objective. Also, a node that doesn't want to wait for the
> next flood can do a standard GRASP unicast synchronize action.
 
Just to be sure I understand: You're essentially saying GRASP shouldn't deal with issues like rebooting, but an ASA needs to understand that it just got re-started, and if required, ask specifically its neighbors for current state? That would make sense to me. 

Michael

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 31 Aug 2016 08:21:35 +1200

Hi Michael,

Hope you had a good vacation.

> Wouldn't it be easier if GRASP could map to specific interfaces, such that we only have one such beast? And that the ACP could remain just "plumbing"?  

Just to be clear, my implementation (and I suspect any implementation) figures
out which interfaces exist and sets up a socket for multicasting to each one. That scales
if you have a reasonable number of interfaces. How many interfaces do you expect an ACP user
to see in a typical case?

> Just to be sure I understand: You're essentially saying GRASP shouldn't deal with issues like rebooting, but an ASA needs to understand that it just got re-started, and if required, ask specificall…

Yes (either ask, or simply wait for the next flood, according to the semantics of
the particular autonomic function).

GRASP needs to do a few things after a restart, like repeat the determination of
which interfaces it has, what its own IP address is, etc., and I'm sure the ACP will
need to do the same sort of thing. We do need to emphasise for ASA writers that
they have to write robust code - however robust we make GRASP and the ACP, the ASAs
must be written to a high standard too.

Regards
   Brian