Subject: [bmwg] Comments on draft-ietf-bmwg-igp-dataplane-conv-*-16 drafts

From: "Kris Michielsen" <[email-masked]>
Date: Thu, 23 Oct 2008 14:20:52 +0200

Al, Scott and BMWG,

My comments on the draft-ietf-bmwg-igp-dataplane-conv-*-16 drafts are below.
I hope you find them useful and please feel free to contact me if you have
any questions or need clarification.

Thanks,
Kris

------
General

In the drafts I found no clear answer to the question "What needs to be
measured: 1) time to complete the convergence process, based on data-plane
measurements or 2) duration/amount of packet loss that occurs during the
convergence process?". When the Convergence Event causes instantaneous
traffic drop for all routes/destinations (all "pull the cable" testcases),
there is no difference between 1) and 2). For the other test cases (with
Convergence Event inserted by the test equipment (aka "Tester")), the
Convergence Event may or may not cause this instantaneous traffic drop,
depending on testcase, UUT and also test equipment implementation. In that
case 1) may differ from 2).
I see a need to measure and report both 1) and 2) in all test cases where
the Convergence Event is not causing instantaneous traffic loss (including
cases where e.g. the Tester sends a POS alarm but keeps accepting incoming
packets). Not measuring 1) in these cases (without instant loss) may not
give the measurements that one would expect e.g. because in the lab
conditions the convergence is lossless.

To measure time to complete convergence (option 1) above) - even in cases
not normally exhibiting packet loss - I see two possibilities:
a) instantanously start dropping traffic - for all destinations - on the
Tester port connected to the Preferred Egress Interface at the time instant
the Convergence Event occurs
b) the Tester collects timestamps at the time instant traffic is started and
at the time instant the Convergence Event occurs and the Tester only
considers packets arriving on the Tester port connected to the Next-Best
Egress Interface.

When measuring packet loss during the convergence process (option 2) above),
the behaviour of the test equipment needs to be defined or at least
reported. Does it drop packets on the port connected to the Preferred Egress
Interface? If yes, when exactly? If this is not defined or reported, simply
replacing the test equipment in the same test setup may lead to different
measurement results.

------
Methodology draft

Each testcase procedure starts with "Advertise matching IGP routes from
Tester to DUT on Preferred Egress Interface and Next-Best Egress Interface
(...)". What does it mean? It could mean one can advertise a separate
topology on both interfaces having the same prefixes but different
nodes/routers. This would obviously lead to an unrealistic setup and
unrealistic results. It needs to be explicitely defined that a single
link-state IGP topology needs to be advertised with both interfaces
connected to this one topology.

page 8, section 3.3
Why is Loss-Derived Convergence Time not measured/reported for Reversion?

Test cases 4.4, 4.5, 4.6 and 4.7 are not unambiguously defined:
(4.4) questions/comments:
- What needs to be measured? Convergence time and/or packet loss during
convergence? See general comment above.
- In reality a layer2 failure can be without (instantaneous) packet loss
(e.g. crash of RP on distributed platform) or with packet loss (e.g. failure
of an interface with intermediate devices not propagating this failure).
Both cases can be measured and need to be described (or included in the
topic of the first bullet above). Currently the test only seems to consider
the case without (instantaneous) packet loss? In that case it needs to be
made clear layer2 failure detection time is not included in the measurement.
- In the case layer2 failure detection time would be included, any layer2
timer setting would have to be reported
- On page 5 of the methodology draft, IGP timers are recommended to be tuned
(1sec hello, 3sec hold). For this test case  the IGP adjacency should not
time out before layer2 goes down.

(4.5) questions/comments:
- What needs to be measured? Convergence time and/or packet loss during
convergence? See general comment above.
- In reality an IGP adjacency failure can be without (instantaneous) packet
loss (e.g. crash of RP on distributed platform) or with packet loss (e.g.
failure of an interface with intermediate devices not propagating this
failure). Both cases can be measured and need to be described (or included
in the topic of the first bullet above). Currently the test only seems to
consider the case without (instantaneous) packet loss? In that case it needs
to be made clear IGP adjacency failure detection time is not included in the
measurement. Note as well that convergence will likely be lossless in that
case.

(4.6) questions/comments:
- What needs to be measured? Convergence time and/or packet loss during
convergence? See general comment above.
- There are a couple of issues with the route withdrawal test case. One
issue is related to how the link-state IGP topology is injected by the
Tester. The correct way to inject such a topology is to advertise a single
topology with both Preferred and Next-Best Egress interfaces connected to
it. This is NOT equal to injecting one topology on the Preferred Egress
Interface and another topology (having the same prefixes but different
nodes) on the Next-Best Egress Interface. Having a single topology implies
the Tester cannot withdraw the LSPs/LSAs from that topology since it would
effectively withdraw them from both interfaces and convergence would not be
possible.
For the single advertised topology case, the only routes the Tester can
withdraw are leaf routes that are advertised by at least two nodes in the
emulated topology. In that case one of the nodes advertising these leafs can
"withdraw" its advertisement of these leafs and the DUT will then reroute to
another node also advertising these leafs.
One realistic case of multiple nodes advertising a decent number (10s, 100s
or 1000s) of the same leaf networks are ASBRs. The recommendation would be
for the Tester to emulate (besides a realistically sized number of nodes and
prefixes) two ASBRs advertising the same external networks. Call these
"extPfx1...extPfxn".
Call the Tester port connected to the Preferred Egress Interface "portA" and
the Tester port connected to the Next-Best Egress Interface "portB". Call
the two emulated ASBRs "ASBR1" and "ASBR2". Call the metric to go from portX
to ASBRY "metric(portX->ASBRY)".
Before the withdrawal, ASBR1 and ASBR2 are both advertising
extPfx1...extPfxn. To assure that before the withdrawal the UUT prefers
ASBR1 via portA to reach extPfx1...extPfxn, metrics in the emulated topology
need to be such that: metric(portA->ASBR1)<metric(portA->ASBR2),
metric(portA->ASBR1)<metric(portB->ASBR1) and
metric(portA->ASBR1)<metric(portB->ASBR2). After ASBR1 withdraws
extPfx1...extPfxn, UUT should prefer ASBR2 via portB. To ensure this,
metrics need to be such that metric(portB->ASBR2)<metric(portA->ASBR2).
A second issue is that there is a dependancy on the test equipment to
advertise the LSAs/LSPs to withdraw the routes. Each test equipment can
differ in - for the ospf example - the amount of LSAs per LS Update packet
and the amount of pacing between LS Updates. These are not defined in the
draft. Preferably all LSAs would be sent in a single packet.
- In reality route withdrawal (from e.g. an ASBR) can be without
(instantaneous) packet loss (e.g. due to change of a redistribution policy)
or with packet loss (e.g. failure of an interface with BGP peering over it).
Both cases can be measured and need to be described (or included in the
topic of the first bullet above). Currently the test only seems to consider
the case without (instantaneous) packet loss? Note that convergence will
likely be lossless in that case.

(4.7) questions/comments:
- What needs to be measured? Convergence time and/or packet loss during
convergence? See general comment above.
- There should be no assumption on the result (Result section says: "There
should be no measured packet loss for this case.")

------
Terminology draft
 
In the evolution of the methodology draft to what it is today, lots of good
and essential improvements have been made. But these improvements only seem
to have led to patches in the terminology draft, touching as little as
possible of the original version. This leads to confusing and unclear
definitions, superfluous comments, contradictions between Definitions and
Discussions and incorrect use of defined terms. These need to be corrected.
It's worthwhile to rethink/rework the terminology draft as a whole since
there is a big difference between defining one measurement using two
measurement methods - as it used to be - and defining three (or more)
measurements using three methods - as it is now. The draft would also become
better referable from other (in progress or future) convergence benchmark
documents.
One point would be to separate measurement methods from measurements (e.g.
the current Rate-Derived Convergence Time) since the same method (e.g.
rate-derived) can be used for more than one measurement (e.g. first route
and last route convergence times). It also makes it easier to describe the
characteristics (e.g. precision or accuracy, ...), the pros and cons of the
different methods and the required test equipment functionality (e.g.
polling interval, number of flow statistics, ...) for each method.

page 3
"flow" needs to be added to the existing definitions, ref [Po06]

page 7
 28    3.7 Convergence Packet Loss
...
 34         Discussion:
 35         Convergence Packet Loss includes packets that were lost and 
 36         packets that were delayed due to buffering.  The Convergence 
What is the maximum acceptable delay and where is it specified?

page 8
  4    3.8 Convergence Event Instant
  5 
  6         Definition:
  7         The time instant that a Convergence Event becomes observable in 
  8         the data plane.
  9 
 10         Discussion:
 11         Convergence Event Instant is observable from the data 
 12         plane as the precise time that the device under test begins 
 13         to exhibit packet loss.  
I find it confusing to define Convergence Event Instant to be different from
the intuitive definition "The time instant that a Convergence Event occurs".
Both time instants (Convergence Event and start of loss) are useful but not
necessarily concurrent, so I would rename the current "Convergence Event
Instant" to "First Route Loss Instant" and make the definition of
"Convergence Event Instant" being "The time instant a Convergence Event
occurs".

page 11
 15    3.14 Loss-Derived Convergence Time
 16 
 17         Definition:
 18         The amount of time it takes for Full Convergence to be
 19         completed as calculated from the amount of Convergence 
 20         Packet Loss.  Loss-Derived Convergence Time can be 
It's incorrect to state that you can measure Full Convergence completion
time by using the loss-derived method. This is only correct (as correctly
pointed out in the Discussion section) if both Convergence Event Transition
and Convergence Recovery Transition are instantaneous. A correct definition
would say: "The average convergence time over all routes in the RIB as
calculated from the amount of Convergence Packet Loss."

page 12, 14: sections 3.15 and 3.17
In general it's incorrect to say that rate-derived first route convergence
time is equal to minimum(route-specific convergence times) (see 3.17) and to
say that rate-derived full convergence time is equal to
maximum(route-specific convergence times) (see 3.15). This is dependent on
characteristics of Convergence Event Transition, Convergence Recovery
Transition and the route update order in both transitions.

page 15
  4    3.18 Reversion Convergence Time
  5         Definition:
  6         The amount of time for the DUT to complete Full Convergence 
  7         to the Preferred Egress Interface, instead of the Next-Best 
  8         Egress Interface, upon recovery from a Convergence Event.
The reversion of a Convergence Event is a Convergence Event by itself, so
the same convergence measurements are applicable to this as well. I don't
see a benefit to have a separate "Reversion Convergence Time" term for the
reversion phase of the test procedure.

_______________________________________________
bmwg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bmwg