Subject: [bmwg] comments on draft-asati-bmwg-reset-03

From: David Newman <[email-masked]>
Date: Fri, 19 Mar 2010 16:08:56 -0700

http://tools.ietf.org/html/draft-asati-bmwg-reset-03

I support this effort moving forward in the bmwg, but have some comments
on this draft.

ABSTRACT and INTRODUCTION
s/reset/reset performance/

INTRODUCTION
The "other factors" list can be expanded:

7. Data plane traffic -- orientation and distribution of data plane
traffic (as those terms are defined in RFC 2285), and on how many
interfaces, just prior to reset

8. Control plane traffic -- type and amount of routing protocol traffic
just prior to reset

SECTION 3
typos in first paragraph -- DUTs and SUTs do not use a possessive apostrophe

The definitions given in sections 3.1 and 3.2 do not sufficiently
distinguish hardware and software resets. If a "hardware reset" and a
"software reset" both can be triggered from the command line, how are
they different from the perspective of what's externally observable?

SECTION 3.1.1.1 and ALL OTHER PROCEDURES

Suggest replacing "mandatory" with "MUST" keyword in text (not section
heading) for better compliance with RFC 2119

The procedures require completion of "some or all" of a list of
operational tasks. To ensure reproducibility across different test beds,
this should be changed to "all" of a specific list of tasks.

The procedure sections could use more specificity about test traffic:

- where is traffic forwarded? To all or some routes held in the RIB?

- is correct forwarding of all traffic on all interfaces prior to reset
a MUST requirement?

- how many data-plane interfaces are in use? (this also belongs under
reporting requirements in each section)

- what is the orientation and distribution of traffic?

- what burst size and inter-burst gap are in use? (and see question
below about constant rate traffic)

Also in section 3.1.1.1 et seq, there is no such metric as "maximum
forwarding throughput." Per RFC 1242, throughput is a single rate.

Finally, section 3.1.1.1 and other procedures derive reset time from a
packet loss measurement. This is a valid method of determining how long
the router was off the air, *provided* the traffic generator(s) offer
traffic at a constant rate. If traffic rates are bursty or in any other
way random, packet loss may not accurately describe reset time. This can
be fixed by adding a requirement for a constant offered load from
traffic generator(s).

SECTION 3.1.1.2
This makes assumptions about default and nondefault configurations of
devices with redundant route processors. While all routers I know are
indeed nonredundant by default, it probably would be safer to use the
terms "redundant" and "nonredundant" rather than default/non-default.

SECTION 3.1.2
The line card is responsible for data-plane packet forwarding, but may
or may not be responsible for control-plane packet forwarding. An RP or
other element with OOB interfaces may handle control-plane packet
forwarding.

SECTION 3.1.2
The procedure is missing specifics on the number of routes and the
routing protocol involved for this test, and how data-plane traffic uses
these routes.

SECTION 3.2
The process reset test is a "white box" procedure in that it requires
direct interaction with the DUT/SUT rather than external stimuli. This
contradicts both longtime bmwg practice and also language in section 4
on security considerations saying testing is done on a "black-box basis."

I'm in favor of adding this "white box" test, but it is a change from
past procedure and we'd need to reconcile language in section 4 and
perhaps elsewhere in the bmwg to allow it.

SECTION 3.3
wording nit -- s/DUT or/DUT and all of its/

SECTION 3.3.1
"The actual power interruption should last at least 15 seconds." Not
disagreeing, but why 15 seconds and not 5s or 60s? Is there something
about capacitator discharge or fan spindown that makes 15 the right number?

SECTION 4
Language on "black box" testing should come out if section 3.2 on
process reset stays in

dn