Subject: [Isis-wg] a few comments on ietf restart isis draft

From: Naiming Shen <[email-masked]>
Date: Fri, 19 Apr 2002 23:52:30 -0700

Hi Mike,

I have a few comments on draft-ietf-isis-restart-00.txt, I apologize
for the late discussion. It was in my "stuff I gotta read" folder
for too long (something just learned from Jeff) ;-)

 (1) TLV code [TBD]

     seems we have a code for this now.

 (2) page 4, the 2nd paragraph,

     "a) DO NOT refresh the timer on the adjacency, ..."

     I have big problem with this. The receiving router has a good
     adjacency to the neighbor which just restarted, this neighbor
     asks for help.  The restarting neighbor sent out an IIH
     containing RR bit set, and also has its "holding time" value
     in this IIH. Why should the receiving router not to refresh the
     timer using the latest "holding time" value in this re-start IIH?

     I think in this case, the receiving router is "lucky" enough to
     get an IIH(with RR bit) from the restarting router before it times
     this adjacency out, so the receiving router should do everything
     to help the neighbor to complete the restart process.

     for example: the receiving router only has 1 second left on this
     adjacency, and received the RR bit set IIH on this adjacency. This
     IIH has the "holding time" of 30 seconds. If it wants to help
     the neighbor to complete a good restart, should the receiving
     router times out this adjacency in 1 second, or should the receiving
     router holds on this adjacency for another 30 seconds? I would
     definitely go for the later.

     The only "harm" I can see if updating the timer is that, if the
     restart router after sending out the re-start IIHs, then crashed
     again and never come back. If this happens, then the convergence
     is a little slow, but this may not affect the forwarding and its
     a very rare case. 

     Yes, the vendors should make the restart come back way before the
     neighbors holding time expires, but it may not always be so easy;
     Yes, the normal holding time can be configured long enough so it
     always handles whatever time restart takes, but that seems going
     the opposite direction from IGP milli-second convergence.

     I would change "DO NOT" into "MUST" here.

 3) page 4, paragraph 3 and 4

     Probably need to say something explicitly about the ordering of
     those packets. If we have IIH(with RA bit), CSNPs and LSPs to be
     sent to help the neighbor restart, we need to make sure the IIH goes
     out first. Otherwise the CSNPs or LSPs can be dropped by
     the re-starting neighbor since it does not have an adjacency yet.
     "without waiting for any currently running timer interval to expire"
     may refer to the same idea, but it should be spelled out.

 4) page 4, paragraph 5

     "i.e. if there was no adjacency"

     Should add "or the adjacency is not in the "UP" state".

 5) page 4, last two paragraphs

    "contains an empty ..." and "option with state "Down" and with empty .."

     Probably those two sentences should be removed. Because on page 4 the
     first paragraph says:

      "irrespective of the other contents of the "Intermediate System
      Neighbors" option (LAN circuits), or the "Point-to-Point Adjacency
      State" option"...
    
     Which means, as long as the RR bit is set in this re-start tlv,
     we don't care about the ISN tlv is empty or NOT empty and we don't
     care about what state this IIH has. Which is good.
     
     I'm not just nit-picking on language here, basically this RR bit
     is a mechanism for IS-IS to re-acquire LSPs from neighbors. It can
     be used for some other applications other than graceful restart.
     In those cases, we don't want to send out empty ISN TLVs or in
     a "Down" state. Also we don't want the neighbors to ignore my
     "holding time" either;-) (see the comment #2).

 6) levels of IIH with RR bit set
 
     The draft probably should say clearly about the levels:

       on the LAN, if both levels are active on the interface, it
       should send out two IIHs with the RR bit set, one for each
       level. The receiving router may respond with two IIHs with RA
       bit, one for each level where the adjacencies are in UP state.
       a) b) and c) on page 4 is for each level. on p2p intf, only send
       one IIH, but the receiving router still needs to check for
       both levels base on the adjacency usage to send out csnps and lsps.

 7) page 3, last paragraph

     This comment is closely related to comment #2.

     Assume we DO(opposite from the current draft) update the adjacency
     timer by the receiving router, then this "Remaining Time" field is
     no use here. This value set by the Receiving router would be the
     same as the "holding time" value in the IIH the re-start router
     just sent out. Thus this re-start TLV can use just one octet in
     length.

     The T3 is the minimum value of all the IS-IS intf holding times.
     And it can be figured out immediately without waiting for
     neighbors to tell us. It is based on local configure settings.

     I think about this way: it's reasonable to expect all the interfaces
     go through the similar amount of time to finish this re-start
     job. But if we ask the neighbors NOT updating the timers on
     the adjacencies, it will be very random. Some adjacencies will be
     timed out in 1 second, some will be in 20 seconds, etc. It's much
     better to ask them to time out me in, say 20 seconds, across
     the board. "Since I'm back, give me another chance, and I know
     my restart process takes xx seconds". Implementations can
     even offer a config knob to send a different(base on the
     local configuration size, network topology size and database size,
     also the vendor implementation, those are all closely related
     to the re-start time) holding time in this re-start IIH.

     T3 is the minimum of this "Remaining Time" right now. So if T3
     times out, all the normal IIHs are starting to go out on all
     the interfaces, this means, if one adjacency has low timeout
     from our neighbors, all our adjacencies may be affected during
     the restart since they sees the incomplete ISN TLVs in normal IIHs.

 8) page 4, paragraph 4

     "excluding the transmitting router"

     An optimization would be, "excluding the transmitting router
     and only include the routers on the LAN who had re-start
     TLV in their IIHs". There maybe someone on the LAN running old
     code, that router just happen to have the highest priority but
     it can not help the restart neighbor with CSNP and LSPs. So
     some router with new code should step in now since the information
     is available.

 9) page 5, first paragraph

     "On expiry of the timer T1, it is restarted and the IIH is
      re-transmitted as above."

     I take it that, this is to avoid the first IIH packet got
     lost, so let's retry. and have a maximum number of retry
     mentioned in page 6.

     I think we should not even retry. This graceful restart is
     not meant to be bullet proof. If the IIHs happen to get lost,
     we just have a bad day. This is also the same problem if you
     have multiple neighbors on the same LAN. Your IIH can be lost
     to one of the neighbors, but as soon as you receive from
     other neighbors and complete the CSNPs, the T1 will be cancelled.
     But you have no way to know, that one neighbor didn't
     receive your re-start IIH, and it will reset the adjacency.
     The current scheme does not cover that case, and I think its
     not worth to have retries in general. Thus actually we don't
     even need the T1.

 10) page 8, the 5th paragraph

     "are then flooded with the overload bit set to indicate that
      the router's LSPDB is not yet synchronized"

      I would be very careful setting OL bit here. T3 times out just
      means we are sending out normal IIHs, that should not affect
      anything even we have not acquired all the LSPs from neighbors
      or we haven't finished our SPF.

      The goal of this graceful restart is to do non-stop forwarding,
      if we send out our LSPs with OL on, it's not non-stop forwarding
      any more. Yes our SPF has not finished yet, so what's the hurry?
      why not let it to finish? Its much better to delay sending out
      our LSPs(assume topology has not changed during re-start) since
      routers in the area has our old LSPs containing the same
      information, than to send out LSPs immaturely with OL bit set.

      Also in 4.3.1.1, "with the overload bit clear", just a simple
      comment, some implementation/configurations couple this OL bit
      with bgp convergence, and usually bgp "converge" takes longer
      than the IGP lsp synchronization.

 11)  3way hello state

      The draft suggested to send the re-start IIH, with RR bit set,
      and 3way state in "Down" on the p2p interface. Assume the
      neighbor sends back the IIH with "UP" state with RA bit set.
      If we follow the 3way hello draft, we are in "Down", when
      receives an "Up", the result state is still "Down". And this
      is not good. We need to setup this adjacency to "Up" state
      immediately because we need to receive and accept CSNPs and
      LSPs over this link soon. So the draft needs to suggest the
      re-starting router either sending out the re-start IIH with
      "Init" or "Up" state, or specifying not to follow the 3way
      hello state table in this case and bring the adjacency up
      immediately. I think the former is better.

 12) re-start router lsp acquiring and flooding optimization?

      If the re-start router has one hundred IS-IS interfaces, at
      restart, it will receive 100 copies of the databases from
      all the interfaces assume all the neighbors can help. We
      also need to flooding all of them to the other 99 interfaces
      100 times. If we have a huge database, it can take a while;-)
      I'm wondering if the re-starting router and all its neighbors
      can periodically exchange a number of CSNP sets during this
      synchronization process until the T3 expires on the re-starting
      router, and until the neighbors sees a "normal" IIH from the
      re-starting router. This way, the neighbors may not need to
      send some redundant lsps, and the re-starting router may not
      need to do unnecessary lsp flooding.

thanks.
- Naiming
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://spider.juniper.net/mailman/listinfo/isis-wg

---

From: Dave Katz <[email-masked]>
Date: Sat, 20 Apr 2002 10:31:16 -0700 (PDT)

Nischal came up with a variant of this which accomplishes both goals--
accept the hold time from the IIH that causes you to become a restart
helper, and then don't refresh after that.

This allows scheduled restarts while normally using hold times too
short to accomplish the restart.  Imagine running with one second hold
times--the restarting router could change its announced hold time to
20 seconds in the RR IIH and then restart at its leisure.

This still provides bounding for the restart period, just a wee bit longer
than it would have been otherwise (at worst the "normal" hold time plus
the "restart" hold time.)
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://spider.juniper.net/mailman/listinfo/isis-wg

---

From: Naiming Shen <[email-masked]>
Date: Sat, 20 Apr 2002 12:24:03 -0700

]  (2) page 4, the 2nd paragraph,
 ] 
 ]      "a) DO NOT refresh the timer on the adjacency, ..."
 ] 
 ]      I have big problem with this. The receiving router has a good
 ]      adjacency to the neighbor which just restarted, this neighbor
 ]      asks for help.  The restarting neighbor sent out an IIH
 ]      containing RR bit set, and also has its "holding time" value
 ]      in this IIH. Why should the receiving router not to refresh the
 ]      timer using the latest "holding time" value in this re-start IIH?
 ] 
 ] Nischal came up with a variant of this which accomplishes both goals--
 ] accept the hold time from the IIH that causes you to become a restart
 ] helper, and then don't refresh after that.

why not refresh after that? I think we should always refresh. Well,
it's an implementation issue also, then I don't go into that.

there are two cases of restart, scheduled and unscheduled.

 for scheduled one: since you know you are going down, and you know
   how long it will take, then send out a "normal"(not with RR bit set)
   IIH with long-holding-time value, so the neighbor does not even
   know its a helper, but will hold this adjacnecy for that long.
   If you set RR bit on this IIH before you going down, I don't think
   this is good. Since the helper will keep sending LSPs to you
   which is a waste of resource.
   At scheduled restart come back, you should send the IIH with RR bit
   one, this is just part of the the normal re-start. This turns the
   neighbor into the helper, and it SHOULD refresh the holdtime.

 for unscheduled scheduled one: its the same as the last paragraph,
   the IIH turns the neighbor into the helper. So the only difference
   between those two cases should be the first one will send a normal
   IIH with large value of holdtime before going down.

There is no problem of refreshing it afterwards, since the re-start
router is not suppose to send any IIH until the all the adjacencies
are well established(T3 expires as in the draft), so it will be
good to always refresh the holdtime, no exceptions.

thanks.
 ] 
 ] This allows scheduled restarts while normally using hold times too
 ] short to accomplish the restart.  Imagine running with one second hold
 ] times--the restarting router could change its announced hold time to
 ] 20 seconds in the RR IIH and then restart at its leisure.
 ] 
 ] This still provides bounding for the restart period, just a wee bit longer
 ] than it would have been otherwise (at worst the "normal" hold time plus
 ] the "restart" hold time.)

- Naiming

---

From: Nischal Sheth <[email-masked]>
Date: Mon, 22 Apr 2002 10:53:42 -0700

> it's an implementation issue also, then I don't go into that.

Naiming,

Not refreshing after the first time can help cover two possible corner 
cases:

o the restarting router crashes again while coming up
o the hello with the restart ACK gets lost

I agree that it's much better to refresh the timer on the adjacency than 
not to (like the draft says currently).  Whether to do it the first time 
or every time is up to the implementation.

Thanks,
Nischal.

---

From: Naiming Shen <[email-masked]>
Date: Mon, 22 Apr 2002 21:48:18 -0700

Hi Nischal,

I agree that "first time" or "every time" is an implementation detail.
Here I just want to cover the two corner cases for someone wants to
implement "every time" refresh.

 ] o the restarting router crashes again while coming up

If we want to prevent a neighbor repeated restart again and again,
we can put a counter for the restart adjacency, if exceeds the limit,
tear down the adjacency. So even if we refresh the holding time
for every re-start IIH, thats ok. The advantage of refreshing every
time in this case is that, it gives every restart the same chance.

Why does anyone need this? for some implementation, if a router tries
a new version of software 3 times and all crashes, it can bring back
the older version of software. In this case, the 4th restart will still
have enough time to succeed.

Just some implementation detail.

 ] o the hello with the restart ACK gets lost

If the restart ACK gets lost, or if the restart IIH gets lost,
according to the draft, we'll resend the restart IIH again. If they
are lost again, .... So basically the same issue as the first case,
do we want to deduct time from each restart IIH, or do we want to
give each the same chance. Thus it's the same implementation detail.
I personally don't even think we should resend again if they got lost.

thanks.

 ] 
 ] I agree that it's much better to refresh the timer on the adjacency than 
 ] not to (like the draft says currently).  Whether to do it the first time 
 ] or every time is up to the implementation.
 ] 
 ] Thanks,
 ] Nischal.

- Naiming

---

From: mike shand <[email-masked]>
Date: Wed, 24 Apr 2002 14:53:26 +0100

>      seems we have a code for this now.

Yes. That needs to go in.

>      I would change "DO NOT" into "MUST" here.

I think, given the various scenarios, we need a bit of flexibility here. My 
intent was that the whole thing was "fail safe". i.e. if the restarting 
router didn't come back up properly, the neighbors would drop their 
adjacencies as if it had just plain failed. I  didn't want the restarting 
to delay the normal recovery.

However, I understand that some implementations may not be able to get back 
up within this time, especially for short hello times, and that some 
measure of "hello I'm back, could you just give me a little extra time to 
get my act together" would be helpful. I like Nischal's suggestion here.

>      may refer to the same idea, but it should be spelled out.

Yes. That was the idea. You need to get the IIH out first, but saying so 
explicitly wouldn't hurt.

>      Should add "or the adjacency is not in the "UP" state".

Yes. Agreed.

>      "holding time" either;-) (see the comment #2).

Yes. I agree. I was being over prescriptive. I think I was just pointing 
out that you didn't need to "know" that information or include it. But as 
you say, there is no reason why you shouldn't if you want to.

>        both levels base on the adjacency usage to send out csnps and lsps.

Yes. I think someone else already pointed this out.

>      just sent out. Thus this re-start TLV can use just one octet in

Yes. Of course the idea was to figure out how long you had been unconscious 
when you woke up, and hence how much time you still had left.

You are right that if we refresh when we come back up we don't need this.

>      the restart since they sees the incomplete ISN TLVs in normal IIHs.

I think you've convinced me. Anyone think otherwise?

>      some router with new code should step in now since the information

Good idea. Yes.

>      lost, so let's retry. and have a maximum number of retry

Yes.

>      not worth to have retries in general. Thus actually we don't

I'm not sure. I didn't like the idea that we may be stalled waiting for 
multiple retries, but OTOH it goes against the grain to design a protocol 
which "fails" on the loss of a single packet :-)

I guess the question is how many times will we sit around waiting for a 
response when we will never get one, as opposed to the times when we 
genuinely loose a packet and this allows us to re-start when we otherwise 
wouldn't.

Any other opinions?

>       information, than to send out LSPs immaturely with OL bit set.

The point is that we have time that we are prepared to run headless. That 
time has expired. We may be generating all sorts of loops and confusion (or 
we may not of course - we don't know). What do we do? Continue to screw up 
the traffic, or admit defeat and take ourselves out? I suggest that the 
only honest thing to do is the latter. We COULD do this by dropping all our 
adjacencies and getting our neighbors to sends LSPs saying we had gone 
away. However it seems cleaner simply to set our overload bit. This is 
after all what we would expect to do if we were coming up without re-start 
and we we were using the OL bit in this way.

However, it may be undesirable to actually insist on this behaviour. Maybe 
it should say that if you are following the setting the OL bit when you 
come up stuff, now would be a good time to do it.

>       than the IGP lsp synchronization.

Yes, absolutely. It shouldn't say turn it off it should say stop turning it 
on for this reason (or words to that effect).

>       immediately. I think the former is better.

To be honest, I've completely forgotten why I wanted to send it in down 
state. I agree absolutely that it needs to go straight into up on receipt 
of the reply (and I thought it said that somewhere, but I can't find it). 
So I guess what I thought was happening was the latter. If there is no 
problem with sending in init or up, then I wouldn't object, but I'm sure 
there was a good reason to avoid it. Maybe it will come back to me....

>       100 times.

Um... 100 times? Why? Yes we flood the first copy we see of each LSP to all 
the other 99, but subsequent copies are just acked.

>  If we have a huge database, it can take a while;-)
>       re-starting router.

I'm not sure how much this helps. 'A' sends us a bunch of LSPs, and we 
flood them out to B through Z, which causes B through Z to clear SRMflags 
for those LSPs (and set SSNflags instead on a p2p). Or if we already 
receive it from (say B) we won't flood it - just set SSNflags back to A. It 
would help of course if we randomized the order of LSP transmission. 10589 
says to do this on LANs for exactly this reason, but doesn't require it on p2p.

I don't quite see how the extra CSNPs help. Unless you are saying you can 
get a set of CSNPs out BEFORE you actually flood.

Or you could just start up on a few interfaces to  begin with, and get the 
bulk of LSPs over them, then turn on the rest and just get any remaining 
LSPs. Of course you have to choose the "right" few:-)

I'm not sure whether these sort of optimizations need to be spelled out in 
the spec. There is some considerable scope for individual implementations 
ingenuity within the basic framework here. I don't think we want to 
over-constrain.

>  This way, the neighbors may not need to

Thanks for some good comments.

         Mike

---

From: Russ White <[email-masked]>
Date: Thu, 25 Apr 2002 08:30:33 -0400 (EDT)

> I think, given the various scenarios, we need a bit of
> flexibility here. My intent was that the whole thing was
> "fail safe". i.e. if the restarting router didn't come back
> up properly, the neighbors would drop their adjacencies as if
> it had just plain failed. I didn't want the restarting to
> delay the normal recovery.
> 
> However, I understand that some implementations may not be
> able to get back up within this time, especially for short
> hello times, and that some measure of "hello I'm back, could
> you just give me a little extra time to get my act together"
> would be helpful. I like Nischal's suggestion here.

Agreed.... It's better to assume that since we've received a
hello with the proper bits set, we should give the adjacent is a
little more time. 

> I'm not sure. I didn't like the idea that we may be stalled
> waiting for multiple retries, but OTOH it goes against the
> grain to design a protocol which "fails" on the loss of a
> single packet :-)
> 
> I guess the question is how many times will we sit around
> waiting for a response when we will never get one, as opposed
> to the times when we genuinely loose a packet and this allows
> us to re-start when we otherwise wouldn't.

We could prescribe a smaller number of retries, perhaps. I don't
like the idea of dropping a single packet causing the restart to
fail, either.

:-)

Russ

__________________________________
[email-masked] CCIE <>< Grace Alone

---

From: Naiming Shen <[email-masked]>
Date: Thu, 25 Apr 2002 23:25:45 -0700

Hi Russ,

 ] 
 ] Agreed.... It's better to assume that since we've received a
 ] hello with the proper bits set, we should give the adjacent is a
 ] little more time. 
 ] 
 ] > >      not meant to be bullet proof. If the IIHs happen to get lost,
 ] > >      we just have a bad day. This is also the same problem if you
 ] > >      have multiple neighbors on the same LAN. Your IIH can be lost
 ] > >      to one of the neighbors, but as soon as you receive from
 ] > >      other neighbors and complete the CSNPs, the T1 will be cancelled.
 ] > >      But you have no way to know, that one neighbor didn't
 ] > >      receive your re-start IIH, and it will reset the adjacency.
 ] > >      The current scheme does not cover that case, and I think its
 ] > >      not worth to have retries in general. Thus actually we don't
 ] > >      even need the T1.
 ] > 
 ] > I'm not sure. I didn't like the idea that we may be stalled
 ] > waiting for multiple retries, but OTOH it goes against the
 ] > grain to design a protocol which "fails" on the loss of a
 ] > single packet :-)
 ] > 
 ] > I guess the question is how many times will we sit around
 ] > waiting for a response when we will never get one, as opposed
 ] > to the times when we genuinely loose a packet and this allows
 ] > us to re-start when we otherwise wouldn't.
 ] 
 ] We could prescribe a smaller number of retries, perhaps. I don't
 ] like the idea of dropping a single packet causing the restart to
 ] fail, either.

"prescribe" is a little strong for me:-) even though I've nothing
against retries.

retries, at least the mechanism in the current draft, only solves
some easy cases, such as the re-start IIHs get lost on p2p
links. Let's see what can happen on a LAN: assume we have routers
A, B, C and D on the LAN. A is the re-start router. A sends out
a re-start IIH on the LAN, only C and D get it. There is a way
for A to know B is on the LAN by looking the re-start ACKs sent
back by C and D(assume they didn't get lost). B should be listed
as one of the neighbors of C and D. should we retry?

- if we don't retry, than its the same "dropping a single packet
  causing the restart to fail".

- if we do try,
  (a) just send the re-start IIH again on this LAN, then C and
      D will go through this re-start again. 
  (b) somehow to only inform B to ack on this re-start IIH.
      we can include systemIDs in the re-start TLV, and have a bit
      to indicate only below systems need to respond to re-start.

- a new mechansim, since C and D all sent re-start Ack, if B sees
  those Acks, B knows it missed the re-start IIH. it would be
  equivalent of recv a re-start IIH:-) but it does not know who
  restarted.

- need to do this for both levels. what if B received re-start
  IIH on level-1 but not level-2? since re-start usually
  affects both levels, maybe should let level-1 inform the
  level-2 on B.

thanks.
- Naiming

---

From: Russ White <[email-masked]>
Date: Sun, 28 Apr 2002 21:31:26 -0400 (EDT)

Then perhaps we should recommend them for p2p links.

> retries, at least the mechanism in the current draft, only
> solves some easy cases, such as the re-start IIHs get lost on
> p2p links. Let's see what can happen on a LAN: assume we have
> routers A, B, C and D on the LAN. A is the re-start router. A
> sends out a re-start IIH on the LAN, only C and D get it.
> There is a way for A to know B is on the LAN by looking the
> re-start ACKs sent back by C and D(assume they didn't get
> lost). B should be listed as one of the neighbors of C and D.
> should we retry?

-- In the case where you are the dis, then you need to send the
retries to all of the is' on the network. In that case, any which
receive two would just ignore the subsequent ones.

-- In the case where you're not the dis, you're primary concern
is the dis (obviously your dropping out isn't going to effect who
the dis is if you're not currently the dis), so retries should
directed at the dis, I would think. The rest of the adj's can be
straightened out as needed.

Does this help your broadcast network case any?

:-)

Russ

---

From: Naiming Shen <[email-masked]>
Date: Tue, 30 Apr 2002 13:30:27 -0700

Mike,

 ] >
 ] >      I would change "DO NOT" into "MUST" here.
 ] 
 ] I think, given the various scenarios, we need a bit of flexibility here. My 
 ] intent was that the whole thing was "fail safe". i.e. if the restarting 
 ] router didn't come back up properly, the neighbors would drop their 
 ] adjacencies as if it had just plain failed. I  didn't want the restarting 
 ] to delay the normal recovery.
 ] 
 ] However, I understand that some implementations may not be able to get back 
 ] up within this time, especially for short hello times, and that some 
 ] measure of "hello I'm back, could you just give me a little extra time to 
 ] get my act together" would be helpful. I like Nischal's suggestion here.
 ] 

Ok, if the restarting router does not come back, then it can not even
send out this IIH, thats fine. There should not be any delay here. If it
does send out the re-start IIH, let's trust it. Since the neighbors have no
way to judge if its a good or bad re-start.
 
My take on this is that, we either want to help this re-start neighbor
or we don't want to help this neighbor. If we do decide to help, lets help
it fully. And the re-starting router knows better how long it need to
take for recovery. The "fail safe" measure we need is to prevent this
neighbor repeatedly re-start, and that can be an implementation issue  
and/or a configuration issue(e.g. user can decide only allow 5 re-start
helps). A router can be instructed to load new image, try three times
max, if all fail, then default back to the old image. In this case,
we want to make sure the 4th re-start will have enough time.

 ] 
 ] >      I think we should not even retry. This graceful restart is
 ] >      not meant to be bullet proof. If the IIHs happen to get lost,
 ] >      we just have a bad day. This is also the same problem if you
 ] >      have multiple neighbors on the same LAN. Your IIH can be lost
 ] >      to one of the neighbors, but as soon as you receive from
 ] >      other neighbors and complete the CSNPs, the T1 will be cancelled.
 ] >      But you have no way to know, that one neighbor didn't
 ] >      receive your re-start IIH, and it will reset the adjacency.
 ] >      The current scheme does not cover that case, and I think its
 ] >      not worth to have retries in general. Thus actually we don't
 ] >      even need the T1.
 ] 
 ] I'm not sure. I didn't like the idea that we may be stalled waiting for 
 ] multiple retries, but OTOH it goes against the grain to design a protocol 
 ] which "fails" on the loss of a single packet :-)
 ] 
 ] I guess the question is how many times will we sit around waiting for a 
 ] response when we will never get one, as opposed to the times when we 
 ] genuinely loose a packet and this allows us to re-start when we otherwise 
 ] wouldn't.

Retry is fine as long as it leaves to the implementation for the N retries,
the N can also be zero.  As I mentioned in another email exchange, the
current retry seems only cover the p2p case, the LAN case is more
complicated. If we lose IIHs, even in normal cases, we will drop
adjacencies. I don't feel particularly bad when this re-start IIH
get lost:-) the worst case is this adjacency is dropped during the
restart, it helps reducing unnecessary flooding also:-)

 ] 
 ] Any other opinions?
 ] 
 ] >       I would be very careful setting OL bit here. T3 times out just
 ] >       means we are sending out normal IIHs, that should not affect
 ] >       anything even we have not acquired all the LSPs from neighbors
 ] >       or we haven't finished our SPF.
 ] >
 ] >       The goal of this graceful restart is to do non-stop forwarding,
 ] >       if we send out our LSPs with OL on, it's not non-stop forwarding
 ] >       any more. Yes our SPF has not finished yet, so what's the hurry?
 ] >       why not let it to finish? Its much better to delay sending out
 ] >       our LSPs(assume topology has not changed during re-start) since
 ] >       routers in the area has our old LSPs containing the same
 ] >       information, than to send out LSPs immaturely with OL bit set.
 ] 
 ] The point is that we have time that we are prepared to run headless. That 
 ] time has expired. We may be generating all sorts of loops and confusion (or 
 ] we may not of course - we don't know). What do we do? Continue to screw up 
 ] the traffic, or admit defeat and take ourselves out? I suggest that the 
 ] only honest thing to do is the latter. We COULD do this by dropping all our 
 ] adjacencies and getting our neighbors to sends LSPs saying we had gone 
 ] away. However it seems cleaner simply to set our overload bit. This is 
 ] after all what we would expect to do if we were coming up without re-start 
 ] and we we were using the OL bit in this way.
 ] 

But "That time" is defined mainly for timeout the adjacencies, its the
time when we start to send out "normal" IIHs. This has a little to do
with if we have finished the SPF or not. My point is that, each router
has its own "time" to finish the re-start. Its affected by its
position in the network, its configuration size, does it have a huge
bgp download at the time(not applicable if the implementation have
isis and bgp running on diff CPUs), does someone have a big image
downloading or bulkstats collections at the time, etc. If for example
this "time" is 25 seconds, and at re-start a particular router may
need 27 seconds. Yes, there needs to be a fixed maximum time, say 5
minutes not finished, lets shut the isis down. But only the users can
decide that. If the router has serious problem, i doubt it can properly
send out this OL bit labelled packet, or even read time correctly.

 ] However, it may be undesirable to actually insist on this behaviour. Maybe 
 ] it should say that if you are following the setting the OL bit when you 
 ] come up stuff, now would be a good time to do it.
 ] 
 ] >       If the re-start router has one hundred IS-IS interfaces, at
 ] >       restart, it will receive 100 copies of the databases from
 ] >       all the interfaces assume all the neighbors can help. We
 ] >       also need to flooding all of them to the other 99 interfaces
 ] >       100 times.
 ] 
 ] Um... 100 times? Why? Yes we flood the first copy we see of each LSP to all 
 ] the other 99, but subsequent copies are just acked.

I was just saying a huge number. Yes, if they happen to occur at the
same time, lots of them will cancel each other. But some of them will
not. Say I get LSP X from intf a, flood this to intf b and c. At the
same time I get X from b also(b sent out before it got X from me), but
the SRM is cleared on intf c since I already sent X out. So I need to
flood X from b to intf a and c again. True, definitely not 100 times.

 ] 
 ] I'm not sure whether these sort of optimizations need to be spelled out in 
 ] the spec. There is some considerable scope for individual implementations 
 ] ingenuity within the basic framework here. I don't think we want to 
 ] over-constrain.

Ok, I agree no need to mention optimization in the draft. Lets leave it to
implementations.

thanks.
- Naiming

---

From: Naiming Shen <[email-masked]>
Date: Tue, 30 Apr 2002 13:53:30 -0700

] 
 ] Then perhaps we should recommend them for p2p links.

fine, but lan has more chance to lose pkts i would think.

 ] 
 ] -- In the case where you are the dis, then you need to send the
 ] retries to all of the is' on the network. In that case, any which
 ] receive two would just ignore the subsequent ones.

how do we decide if the neighbor restarted again, or is doing
retries? even the re-start router is the dis, someone else on
the LAN suppose to serve as a "temp" dis for help. even though
we can put a re-start counter in the re-start IIH saying this
is numberX retry, still how do we know it's due to someone else
on the LAN didn't send ACK, or my ACK actually got lost?

 ] 
 ] -- In the case where you're not the dis, you're primary concern
 ] is the dis (obviously your dropping out isn't going to effect who
 ] the dis is if you're not currently the dis), so retries should
 ] directed at the dis, I would think. The rest of the adj's can be
 ] straightened out as needed.
 ] 
 ] Does this help your broadcast network case any?
 ] 

I think the simplist way is NOT to retry. Simply just sit and wait.
Your neighbor WILL send you a "normal" IIH, since it does not know
you just restarted. Just treat this "normal" IIH as it sends back
the ACK. Ok, what if this neighbor is the DIS or "temp" DIS,
we don't get all the lsps from it? but over other interfaces, we
may get more than enough of them. Ok, what if we only have one
interface? then adjacency reset or not does not affect traffic flow,
since there is only one way to go; Ok, what if all the interface
we lose all the re-start IIHs? then we probably should not being
graceful anymore:-)

- Naiming