Subject: Re: [bmwg] WG Last Call: draft-ietf-bmwg-benchres-05

From: Al Morton <[email-masked]>
Date: Fri, 11 Feb 2005 15:27:36 -0500

At 12:16 PM 01/30/2005, Al Morton wrote:
>A WG Last Call period for the Internet-Draft on
>
>"Benchmarking Terminology for Routers Supporting Resource Reservation"
>
>will be open from 31 January 2005 through 14 February 2005.

This has been a quiet Last Call so far, so here are a few
comments and suggestions, below. This is mostly about tightening
a few of the definitions and consistency, but there are a few
formatting comments, too.

Al
(as a working group participant, so not replying to myself)

Status of the Memo
The paragraph "By submitting this Internet Draft..." usually appears
first.  There are acceptable rewordings for drafts with multiple authors,
here's what I have been using:
>This document is an Internet-Draft and is subject to all provisions of 
>section 3 of RFC 3667. By submitting this Internet-Draft, each author 
>represents that any applicable patent or other IPR claims of which he or 
>she is aware have been disclosed, and any of which he or she becomes aware 
>will be disclosed, in accordance with RFC 3668.
(when I ran the nits checker, v1.58, it complained about boilerplate,
but it complains about my paragraph, too)

Abstract
(there seems to be a stray page break between the title and the text)
>    The purpose of this document is to define terminology specific to
>    the benchmarking of resource reservation signaling of Integrated
>    Services IP routers.
There is some applicability beyond IntServ, right?
You have looked ahead to NSIS terminology.  How about
The *primary* purpose of this document...

1.  Introduction
... 4th para
>    In order to avoid any confusion we would like to emphasize that this
>    terminology considers only signaling protocols that provide IntServ
>    resource reservation; the DiffServ world, for example, is out of our
>    scope.
This paragraph responds to an old AD-level comment, if memory serves.
But several definition sections discuss packet markings,
though they refer to them as TOS markings (precedence markings were
grand-fathered into Diffserv).
The example of exclusions in the last phrase needs to be tightened
so that it's clear.

3.2.5 Resource Reservation Protocol Orientation
>    Issues:
>      The location of the reservation initiator affects the basics of
>      the resource reservation protocols and therefore it is an
>      important design decision.
s/design decision/aspect of characterization/

3.3 Router Load Factors
Some of these seem to be Load Rates, specifically anywhere I see
>    Measurement unit:
>      bits per second (bps)
that's a rate.   BTW, what overhead bits are included in the count?

3.3.3 Session Load Factor
>    Measurement unit:
>      This factor is measured by the number of QoS sessions impacting
>      the router, thus it has no unit.
I suggest:
    Measurement unit:
      This factor (a count) is given in units of QoS sessions impacting
      the router.

3.3.4 Signaling Intensity Load Factor
>    Definition:
>      The signaling intensity load factor is defined as the number of
>      signaling messages that hit the router during one second.
I suggest:
    Definition:
      The signaling intensity load factor is defined as the number of
      signaling messages that are presented at input interfaces during
      one second.
>    Measurement unit:
>      The unit of this factor is 1/second.
I suggest:
    Measurement unit:
      This rate is given in signaling packets/second.
Also, how do you plan to handle variability in this Load?

3.3.5 Signaling Burst Load Factor

>    Measurement unit:
>      This load factor is measured by the number of messages in the
>      burst, thus it has no unit.
I suggest:
    Measurement unit:
      This load factor is a count in units of messages in the
      burst.

3.4.1 Signaling Message Handling Time

>    Definition:
>      The signaling message handling time (or, in short, signal handling
>      time) is the latency ([1]) of a signaling message passing through
>      the router.
I suggest:
...
      time) is the latency ([1], for store and forward devices) of a
      signaling message passing through the router.
(same for Definition in 3.4.2 Distinguished Traffic Delay)

3.4.4 Signaling Message Loss

>    Definition:
>      Signaling message loss is the ratio of the actual and the expected
>      number of signaling messages leaving a resource reservation
>      capable router, subtracted from one.

This is more like a definition of Signaling Message Survival,
until you subtract from one in the last phrase.  It's a
calculation based on what you can measure, not the definition of loss.
What we need here is more like this:
from RFC 1242:
>3.6  Frame Loss Rate
>
>         Definition:
>                 Percentage of frames that should have been forwarded
>                 by a network device under steady state (constant)
>                 load that were not forwarded due to lack of
>                 resources.

And I suggest changing the name to:  3.4.4 Signaling Message Loss Ratio

3.5 Scalability Limit
>    Definition:
>      The scalability limit of the router is the critical load
>      condition, when the router is still in the steady state but the
>      smallest amount of additional load would drive it to the
>      overloaded state.
"steady state" does not necessarily mean loss-free, it just means
stable/constant, as in RFC 1242 Frame Loss Rate, above.
I suggest:
    Definition:
      The scalability limit of the router is the critical load
      condition, when the router is still in the loss-free state but the
      smallest amount of additional load would drive it to the
      overloaded state.

Also, there should be some related definitions cited here,
all the other "Loads" provide the units of measure.

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Al Morton <[email-masked]>
Date: Thu, 24 Feb 2005 19:45:37 -0500

Krisztian,

Thanks for your replies. Here's some feedback on the
open issues.

Al

At 08:39 PM 02/23/2005, Nemeth Krisztian wrote:
>Well, it's a bit harder question, and reading over the related sections, 
>maybe it's really not written clear enough. So, we state in 3.1.1, 
>Discussion, 1st paragraph, that "The flow identifier can be an arbitrary 
>subset of the packet header fields", e.g., the 5-tuple, or the Flow Label 
>field (IPv6). In the next paragraph we intended to say, that the flow 
>classification is a resource consuming task, and this can be saved if the 
>flow is a best-effort (BE) flow only (this latter part is not explicitly 
>written), so there's no need for classification. This can be achieved by 
>marking the non-BE packets with perhaps as few as only one bit (again, not 
>written) e.g. in the ToS field. Accordingly, in 3.1.2, and 3.1.3 we again 
>mention marking, which may be misleading, as ToS-like marking is not 
>strictly necessary for correct operation.
>
>Perhaps we should rephrase this part by putting the whole ToS-marking 
>optimization part under 3.1.3 (BE Data flow), Discussion. Comments?

Try the clarifications you proposed for section 3.1.1.

>Other than that, we are not considered with marking, and as this really 
>has nothing to do with DiffServ, I'd propose to keep the Intro 4th para 
>part as it is now.

I still think the example could be more clear. The term "world"
may raise some eyebrows. Something like:
"For example, techniques in the DiffServ toolbox are predominantly
beyond our scope."

>>that's a rate.
>This seems to be a remark on the usage of English, and as you could see, 
>we're not that perfect at it :-) Anyway, here our idea was to segment the 
>total load into "load components" or "load factors", and list these 
>factors. These includes different kinds of "traffic load factors", 
>"session load factor", "sign. intensity load factor". Yes, the traffic 
>load factors are quantified by the data rate, but for me its sounds odd to 
>write "Distinguished Traffic Load Rate". Maybe "Distinguished Traffic Data 
>Rate Load Factor" would be more appropriate, but is sounds quite 
>complicated. So we'd keep it as it was, but are looking for your answer!

I see your point now, many different factors.
After introducing this section as Router Load Factors
in section 3.3, maybe it's enough to say something like:
"The various factors defined below are rates, session counts,
and messages per burst."

>>BTW, what overhead bits are included in the count?
>
>Good question, have to think about it. My instant answer is to count all 
>the bits in the IP packet, but don't consider PPP/Ethernet/anything else 
>frames overhead bits. We didn't discuss it so far, but are looking for 
>everyone's opinion, too.
>
>Well, we took the physicist approach, so that the number of sessions has a 
>unit of [1]. Nevertheless, it can be changed. What about simply: "This 
>factor (a count) is measured by the number of QoS sessions impacting the 
>router."?

I can live with this wording (better).

>My problem with your suggestion is that it might suggest that a qos 
>sessions have got a unit other then their quantity.
>
>...
>
>Do you mean if the signaling messages are not coming in a CBR style 
>(different interarrival times),

Yes

>or if different messages are coming?
>Anyway, in our view, both will go to the methodology part. [RFC1242] in 
>Section 3.17 (Throughput) doses not deal with the former question (CBR) either.

If the observation interval is long enough, it should smooth-out
periodic variations. Perhaps it is best to deal with this in the
methodology, as you suggest.

>Well, maybe it because of my imperfect English, but I think that although 
>our version is truly more complicated, it's more difficult to 
>misunderstood, too. But there's a more fundamental argument: we did not 
>write "should have been forwarded" as we also consider a message lost, if 
>it should have been generated by the device, but it didn't happen.

I don't think "failed generation" is the same as Loss.
Packet loss always refers to packets that enter, but fail to exit/arrive.
Maybe another term in needed to cover the "failed generation" form of defect.

>Nevertheless, it's a good idea to include the phrases "steady state" and 
>"due to lack of resources".
>
>Ok.
>Right, but I'd propose to extend it like this: "in the loss-free state" -> 
>"in the loss-free state (i.e., it is not losing messages, states, etc., 
>see the Discussion below)".
>
>To be honest, we don't understand, what related definitions should be 
>cited here.

Sometimes, other authors have included a "Related Definitions" section
under each of the terms.  The Scalability Limit will be expressed in
terms of all the Router Load Factors, right.  So just say:

Related Definitions:
      Best-Effort Traffic Load Factor
      Distinguished Traffic Load Factor
      Session Load Factor
      Signaling Intensity Load Factor
      Signaling Burst Load Factor

>About the units: in the early versions it was under 3.4, Performance 
>Metrics, but it didn't really fit (partly because of the lack of unit). So 
>we moved it to an own subsection, 3.5.
>
>Sorry for being so long.

Not a problem.  Glad to help with the language where possible, too.
If I had to write in another language, my messages would be very short.
:-)