Subject: Re: [NSIS] QoS NSLP: rerouting issues

From: Andrew McDonald <[email-masked]>
Date: Thu, 02 Mar 2006 17:50:14 +0000

Hi Roland,

Thanks for looking at this. Sorry for the slow reply, this e-mail got 
half-written and I didn't get round to finishing it. Some 
replies/comments below.

Roland Bless wrote:
>>Proposed updates:
>>
>>##### section 3.2.10 #####
>>
> 
> [previous text ok.]
> 
>>Reservation on the new path happens when a refreshing RESERVE message
>>arrives at the QNE where the old and the new path diverge.  The
>>refreshing RESERVE will be interpreted as a new RESERVE on the new
>>path.  Depending on the transfer mode, this may require installation
> 
> 
> Hm, I'm a little bit confused about this statement. The incoming
> refreshing RESERVE will usually _not_ trigger a transmission of
> a new RESERVE along the new path. Refreshing state is independently
> triggered at each node rather than a refreshing RESERVE traversing
> the path downstream. The branching node (i.e. the QNE
> where the old and the new path diverge) will send a refresh
> to his downstream peer. If rerouting happens, there
> are different cases to be considered:
> - GIST will detect a new path by its own mechanism, i.e. while
>   sending an NTLP refresh. In this case a notification may be given
>   to the NSLP that a downstream SII has changed. This should
>   trigger immediate sending of a refreshing RESERVE (usually
>   with full QSPEC and RII, otherwise we'll get an error back first).
> - NSLP has to send a refresh to its downstream peer, because the refresh
>   timer expired. GIST will find the new route then, notify NSLP of a
>   new SII and will also get back an error message from its downstream
>   peer, because he needs a full QSPEC. NSLP then should send the RESERVE
>   with RII and full QSPEC again.

Yes, rereading this I'm a little bit confused too. :-)

Most of the 3.2.10 text is unchanged from -08 (mostly deleted/reworded
rather than added text), so we should probably have noticed this before. :-)

I think we could say:

Reservation on the new path happens when a RESERVE message arrives at
the QNE beyond the point where the old and new paths diverge. A
refreshing RESERVE (with no QSpec) will be identified as an error by a
QNE on the new path. It will send back an error message which results in
a full RESERVE message being sent.  Depending on the transfer mode, this
may require installation ...

>>of a new messaging association.  Rapid recovery at the NSLP layer
>>therefore requires short refresh periods.  Detection before the next
>>RESERVE message arrives is only possible at the IP layer or through
>>monitoring of GIST peering relations (e.g. by TTL counting the
>>number of GIST hops between NSLP peers or the observing changes in
>>the outgoing interface towards GIST peer).  These mechanisms can
>>provide implementation specific optimisations, and are outside the
>>scope of this specification.
> 
> 
> Yep.
> 
> 
>>When the QoS NSLP is aware of the route change, it needs to set up
>>the reservation on the new path.  This is done by incrementing the
>>RSN and then sending a new RESERVE message.  On links that are common
>>to the old and the new path, this RESERVE message is interpreted as a
>>refreshing RESERVE.  On new links, it creates the reservation.
> 
> 
> Not quite correct. Incrementing the RSN is not necessary in this
> case and  contradicts to the statement that an increased RSN indicates
> a changed session. Furthermore, remember that an RSN has only
> local significance.
> Because the RSN is different it will not directly
> be considered as refresh by nodes that know this session.
> Only if the QSPEC and MRI did not change,
> then there is no need to send it further along the path.
> I would suggest that we clarify text when to "forward"
> a RESERVE (at end of sec. 5.4.1) or TEAR anyway.
> Section 5.3.1. states
>    It MUST be incremented for each new RESERVE message where the
>    reservation for the session changes.
> Strictly speaking the above statement does not infer that
> a new RSN necessarily means that the session changes. But
> in case nothing changed, the RSN must stay the same.
> 
> Let's look what happens: the branching node should send
> a full refreshing RESERVE (RSN not changed, added RII) along the new
> link. If the next QNE is really a new peer that doesn't
> know about the reservation, it will setup a new reservation
> anyway. In this case this QNE will also emit a _new_ RESERVE
> (include a new RSN, because the QNE is new). In case the
> RESERVE hits the merging node again, the latter should
> detect that: it knows the referenced session and that it
> has a new upstream peer. But because neither MRI nor QSPEC
> are changed, a new RESERVE along the unchanged path will
> not be triggered (the normal refreshing RESERVE will be sent
> when the refreshing timer fires).

Yes, RSN does not need to be incremented since the same reservation is
being requested. (I'd probably say 'changed reservation' or 'changed
state' rather than 'changed session').

However, there is the issue of wanting to send a TEAR and not being 100% 
certain whether the next peer has changed (third paragraph of section 
5.2.5.2).

> The last two sentences are confusing, because
> the RESERVE will not be forwarded (esp. not with the very same RSN).
> Correct would be: The RESERVE may be sent along a new link but possibly
> hits the same QNE as before (multi-homing scenario). In this case the
> QNE will consider the RESERVE as refresh. If a new QNE is hit, it will
> establish a new reservation (if possible) and send a new RESERVE downstream.

Agreed. I think this is remnants of a time when we weren't so clear what
service abstract-NTLP/GIMPS/GIST would provide.

I suggest replacing this paragraph and removing the details which fit
better into section 5.

Suggested replacement:
When the QoS NSLP is aware of the route change, it needs to set up the
reservation on the new path. This is done by sending a new RESERVE
message. If the next QNE is, in fact, unchanged then this will be used
to refresh/update the existing reservation. Otherwise it will lead to
the reservation being installed on the new path.

>>After the reservation on the new path is set up, the branching node
>>or the merging node may want to tear down the reservation on the old
>>path (faster than what would result from normal soft-state time-out).
> 
> 
> As already written earlier
> (http://www.ietf.org/mail-archive/web/nsis/current/msg05826.html):
> I don't know how the merging node (i.e. the node where the paths
> join again) should TEAR down the session.
> Currently this is not possible because a RESERVE with TEAR can only
> flow in the direction from QNI to QNR.

Agreed.

This paragraph also has some old 'requests GIST to provide' text.

Suggested replacement:
After the reservation on the new path is set up, the branching nodenode
may want to tear down the reservation on the old path (sooner than would
result from normal soft-state time-out). This functionality is supported
by keeping track of the old SII-Handle provided over the GIST API. This
handle can be used by the QoS NSLP to route messages explicitly to the
next node.

>>When the QoS NSLP is aware of the route change, it needs to set up
>>the reservation on the new path.  This is done by incrementing the
>>RSN and sending a RESERVE message.  On links that are common to the
>>old and the new path, this RESERVE message is interpreted as a
>>refreshing RESERVE.  On new links, it creates the reservation.
> 
> 
> This is the same text as above. I'm not really sure that we need
> this redundancy.
> 
> 
>>After the reservation on the new path is set up, the branching node or
>>the merging node may want to tear down the reservation on the old path
> 
> 
> Only the branching node... see above

Would it be better to fold the remaining non-redundant parts of 4.8 into
3.2.10?

>>downstream peer.  A QNE that has detected the route change via the
>>SII change sends a RESERVE message towards the QNR on the old path
>>(using the old SII) with the TEAR flag set.  Note that in case of
> 
> 
> This TEAR would IMHO be too early. The TEAR should be triggered by the
> RESPONSE to the RESERVE on the new path indicating that the reservation
> is already installed.

Yes.

>>receiver-initiated reservations, this involves a QNE that is notified
>>of the route change in another way and wants to tear down the old
>>branch needs to send the RESERVE on the new path with an RII object.
>>When it receives the RESPONSE message back, it can check whether its
>>peer has effectively changed and send a RESERVE with the TEAR flag
>>set if it has.  Otherwise, teardown is not needed.  A QNE that is
>>unable to support an RII or does not receive a RESPONSE needs to rely
>>on soft-state timeout on the old branch.
> 
> 
> This is ok, but also applicable to the sender-initiated case.

Yes.

I find the last sentences of this paragraph unclear. I'm not sure that
they're saying anything particularly useful (i.e. provides useful
information beyond that in section 3.2.10).

Generally 4.8 feels a bit repetitive of 3.2.10. There's possibly some
new content in the last paragraph. Maybe that could be folded in to
3.2.10, and remove 4.8?

>>QoS NSLP is notified by GIST when GIST believes that a rerouting event
>>may have occurred. As part of the API, GIST provides an SII-Handle
>>which can be used by the NSLP to direct a signalling message to a
>>particular peer. The current SII-Handle will change if the signalling
>>peer changes. However, it is not guaranteed to remain the same after a
>>rerouting event where the peer does not change. Therefore, the QoS
>>NSLP mechanism for reservation maintenance after a route change
>>includes robustness mechanisms to avoid accidentally tearing down a
>>reservation in situations where the peer QNE has remained the same
>>after a 'route change' notification from GIST.
>>
>>When a QNE detects that its neighbour QNE in the direction of the QNR
>>has changed, it SHOULD send a RESERVE message, including the full
>>QSPEC. This RESERVE message MUST include an RII, to request a response
>>from the QNR. When sending this message the RSN MUST be incremented by
>>two (and this becomes the current RSN value). An SII MUST NOT be
>>specified when passing this message over the API to GIST, so that it
>>is correctly routed to the new peer QNE.
>>
>>When the QNE receives the RESPONSE message that relates to that the
>>RESERVE message sent down the new path, it SHOULD send a RESERVE
>>message with the TEAR flag sent down the old path, by specifying the
>>SII relating to the old peer QNE. When sending this RESERVE message
>>the RSN field MUST be set to one less than the current RSN value (and
>>the current RSN value remains unchanged). This 'RSN-1' will cause the
>>message to be reject by the next peer if the old and new next peers
>>are actually the same.
>>
>>Depending on the particular route change, there may be a 'merge point'
>>(or 'crossover node') where the old and new paths merge. Initially this
>>has a reservation installed by the old peer QNE, identified by an old
>>SII. When the new reservation is installed after the route change the
>>new peer QNE will be identified by a new SII. At a later point the
>>crossover node may receive a RESERVE message with the TEAR flag set
>>from the QNE identified by the old SII. A node receiving a RESERVE
>>message with the TEAR flag set which does not come from the current
>>peer QNE, identified by its SII, MUST be ignored and MUST NOT be
>>forwarded.
> 
> 
> Yep. Therefore, we should exactly describe at the end of the
> RESERVE section 5.4.1 that a RESERVE with a TEAR should
> not be sent to the next peer in this case.

I'd probably put this in after paragraph 4 of 5.4.1 (The one beginning 
"When the RESERVE is authorized..."), so it is with the other text 
discussing the TEAR flag.

Suggested text:
As discussed in section 5.2.5.2, to avoid incorrect removal of state 
after a rerouting event, a node receiving a RESERVE message with the 
TEAR flag set which does not come from the current peer QNE, identified 
by its SII, MUST be ignored and MUST NOT be forwarded.

>>.sh 4 "Upstream route change notification"
>>
>>GIST may notify QoS NSLP that a possible upstream route change has
>>occurred over the GIST API. On receiving such a notification, QoS NSLP
>>SHOULD send a NOTIFY message with Informational code ????? for
>>signalling sessions associated with the identified MRI.
>>
>>On receiving such a NOTIFY message, QoS NSLP SHOULD use the
>>InvalidateRoutingState API call to inform GIST that routing state
>>may be out of date. QoS NSLP SHOULD send a NOTIFY message upstream.
> 
> 
> When should forwarding of NOTIFY stop?
> I'm currently not sure how the branching node will detect this
> case. Maybe if the NOTIFY arrives via an inactive/invalid routing
> state.

I'm not sure what we should do here.

It is quite probable (if the possible route change indiciation from GIST 
is correct) that the node that the NOTIFY is initially sent to is no 
longer on the path. Some possible options:
- don't forward it at all (but since the node receiving the NOTIFY might 
not be on the path, but would still have the node sending the NOTIFY as 
its next downstream hop for that flow, it is not clear that sending the 
NOTIFY message is useful)
- let it go all the way back to the QNI/QNR
- let it go all the way back to a QNE which determines that it's 
downstream peer has changed
- something else

Any thoughts on the best answer? ('best' rather than 'correct' since 
this is, at best, a hint)

>>.sh 4 "Route change oscillation"
>>
>>In some circumstances a route change may occur, but the path than
> 
> typo: then

yes

>>falls back to the original route.
>>
>>After a route change the routers on the old path will continue to
>>refresh the reservation until soft state times out, or an explicit
>>TEAR is received.
>>
>>After detecting an upstream route change a QNE SHOULD consider the new
>>upstream peer as current and not fall back to the old upstream peer
>>unless:
>>
>>- it stops receiving refreshes from the old upstream peer for at least
>>the soft state timeout period
> 
> ??? in this case it really makes no sense to fall back to the old
> upstream peer ???

There is a vital bit missing from the end of this sentence:
"and then starts receiving messages from the old upstream peer again"

regards,

Andrew
-- 
Andrew McDonald, Senior Engineer
Roke Manor Research Ltd, Romsey, Hants  SO51 0ZN
Phone +44 (0)1794 833833   Fax +44 (0)1794 833616

_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: Roland Bless <[email-masked]>
Date: Sat, 11 Mar 2006 00:33:30 +0100

Hi Andrew,

> half-written and I didn't get round to finishing it. Some

Dito. I was moving within the last weeks, so I had time for
another look at the draft in the last days merely.
I think it's good that the WGLC was extended since I found more
fundamental issues :-( Please see below.

> may require installation ...

Sounds better, but the second sentence holds only in case
we hit a "new" QNE: in case the QNE is the same
as before, it will not send back an error message

> certain whether the next peer has changed (third paragraph of section

Nice trick with the RSN+2 and RSN-1, but is it really necessary?
Last paragraph of 5.2.5.2 should be sufficient:
A node receiving a RESERVE message with the TEAR flag set which does not
come from the current peer QNE, identified by its SII, MUST be
ignored and MUST NOT be forwarded.
So even when the tearing RESERVE arrives at the same QNE as before
but over a different incoming link or i/f, the SII will be different
and the last condition will hold, or am I missing something?

Further and NEW but also more general issue in a similar context:
if I remember correctly RSVP sends a PathTEAR in case state
expires. QoS NSLP doesn't do so. However, in case we have a node failure
etc. re-routing will possibly happen which will then trigger the
explicit TEAR from the branching node, right? In this particular case
reservations along the old branch will be torn down relatively fast.
But the forwarding of the tearing RESERVE may, however, stop at a
failed node. State in subsequent nodes may take a long time until they
expire if we don't send a tearing RESERVE upon state expiration.

> the reservation being installed on the new path.

Ok.

> After the reservation on the new path is set up, the branching nodenode

s/nodenode/node

> handle can be used by the QoS NSLP to route messages explicitly to the

Ok.

> Would it be better to fold the remaining non-redundant parts of 4.8 into

Section 4,8 should describe an example, so protocol behavior should
better be described in section 3.2.10 and in more detail in section 5.2.5.

> new content in the last paragraph. Maybe that could be folded in to

You are right. The current section 4.8 (of draft -10), however, is still
not correct. I would really describe a scenario and which protocol
actions happen (which is not so easy, because the details are defined
in section 5).

> by its SII, MUST be ignored and MUST NOT be forwarded.

See above...

> I'm not sure what we should do here.

I guess that it makes sense to send a NOTIFY from the merging/crossover
node to the previous active hop. The latter should stop sending
refreshes due to the InvalidateRoutingState (at the GIST level) that is
useful to avoid route oscillation. Lets take the example from Markus
Ott's mail: Old path A-B-E-D, new path A-C-D

                 /-\     /-\
              /--|B|-----|E|---\
             /   \-/     \-/    \
          /-\                    /-\
      ----|A|                    |D|--------
          \-/                    \-/
             \   /-\            /
              \--|C|-----------/
                 \-/

Let's assume the link A-B is broken and A detects re-routing
to C. After GIST indicated the re-routing to QoS- NSLP, it will
send a new RESERVE with RII. C will install new state and "forward"
the RESERVE to D. GIST in D will detect that the RESERVE was received
via another interface and indicates a new SII-handle to QoS NSLP.
D will detect that it is a cross-over node and stop forwarding
of the RESERVE in case nothing else (MRI or QSPEC) has changed.
It will send back a RESPONSE towards A, which will try to send
a RESERVE w/ TEAR flag along the old path. This will fail because
the link to B is broken. Note that B may still send a refreshing
RESERVE towards E. If D sends a NOTIFY towards E, E may stop to
send a GIST refresh avoiding to trigger another route change
at D. However, if the reservation state at E is not torn down
it will send another refreshing reserve (and thus
re-establishing the GIST route). But tearing down the state
at E would not be enough since B can also re-establish state
by sending its periodic refresh.

A further issue comes to my mind:
the RSN at D was the one from E before the route change.
But what happens after the rout change when C sends its new
RESERVE with a new RSN (but even potentially lower RSN)?
So D must update its locally stored RSN in case it is a rerouting
event, right? This should be mentioned in section 5.3.1.

> longer on the path. Some possible options:

Which one would it be in the example above? Do you mean E or B?
>From the discussion above follows that forwarding should not stop
at E, but at the branching node A at last. So notify should be forwarded
up to the branching node if possible. However, I'm not completely sure
that it's really safe to tear down state due to the NOTIFY.

> this is, at best, a hint)

In this context I would also repeat Markus Ott's question again:
(section 3.1.2).
The sentence "Examples would be notification to an upstream peer about
state being torn down or..." is not clear to me.
Take for instance following example:

                 /-\     /-\
              /--|B|-----|E|---\
             /   \-/     \-/    \
          /-\                    /-\
      ----|A|                    |D|--------
          \-/                    \-/
             \   /-\            /
              \--|C|-----------/
                 \-/

All the nodes are QNEs. The flow is going from A to D. The old flow was
going through A-B-E-D. Then because of rerouting the new flow is going
now through A-C-D. The A sends a RESERVE with the TEAR_DOWN bit set to
B. B tears down its state and forwards the RESERVE with TEAR_DOWN bit
set to E. If the node B notifies node A with NOTIFY message about its
state being torn down - there will be no problem. But if the next node E
after tearing down its state will send a NOTIFY message to B, then what
will happen? Will the node B just forward this message because it will
not find the appropriate SESSION_ID or just drop it? If it will just
drop it, then there is no sense to send this NOTIFY message at all. If
the node B will forward this message to A, then what can A do with this
NOTIFY message? There can be 10-20 nodes between A and D on the old
path. What will node A do in this case with 10-20 NOTIFY messages?

So upstream notification in case of state expiration is questionable...

> "and then starts receiving messages from the old upstream peer again"

Oops, missed that one...

Regards,
 Roland