Subject: On the ATM benchmarking terminology draft.

From: Kevin Dubray <[email-masked]>
Date: Wed, 21 Apr 1999 08:43:34 -0400

Hi!

Section 2.2.2.3 of <draft-ietf-bmwg-call-04.txt>, Cell Error Ratio,
states in the discussion: "CER causes ATM cells to be lost."  Isn't
CER a statistic and not a cause?

Likewise Section 2.2.2.5, Cell Misinsertion Rate, discussion states 
"CMR causes ATM streams to be corrupted."   Isn't CMR an indicator to 
the degree of disruption in an ATM stream as opposed to a cause?

Contrast the above to section 2.2.2.7, CRC Error Ratio, whose
discussion presents, "CRC errors causes ATM cells to be lost."
While I would have preferred the term "discarded" to lost, I find 
this statement closer to the mark...

Two general comments:  

1) Are _all_ the related definitions of section 1.2, admittedly
   subsumed from other sources, needed to support the definition
   proposed in section 2.2?  Perhaps the document might benefit by
   including only those background definitions needed to provide 
   a basis for section 2.2's definitions.

2) Section 2.2's definitions do a good job of instrumenting the
   ATM layer (a communicated goal), but that layer's instrumentation
   alone doesn't help me benchmark the cross-technology of an 
   IP/ATM forwarding device.  For instance, how can I emperically
   ascertain the impact of cell loss or AAL 5 reassembly time on IP
   packet forwarding?  As I understand it, this was a major 
   differentiator for this work from other bodies' work. 

-Kevin

---

From: Jeffrey Dunn <[email-masked]>
Date: Wed, 21 Apr 1999 21:02:44 -0400

Kevin,

Thank you for your comments.  I will address each point below.  When we
are in agreement, I will make the appropriate changes to the document.

> CER a statistic and not a cause?

Fair enough.  Perhaps this should read something like: "CER measures
cell, and therefore higher layer PDU, integrity.  As a result, larger
values of CER will result in more SAR failures and, consequently, more
packets discarded at the AAL5 layer."

> the degree of disruption in an ATM stream as opposed to a cause?

Again, perhaps this should read: "CMR measures cell stream, and
therefore higher layer PDU, integrity. As a result, larger values of CMR
will result in more SAR failures and, consequently, more packets
discarded at the AAL5 layer."

> this statement closer to the mark...

Noted.  "CRC errors cause cells to be discarded at the physical layer
convergence sub-layer.  As a result, SAR fails and packets will be
discarded at the AAL5 layer."

>    a basis for section 2.2's definitions.

At the last meeting, I asked the group whether we should include TM4.0
terminology and benchmarking issues.  As i recall, we never reached a
consensus.  We can greatly reduce the number of definitions in section
1.2 if we agree to handle TM4.0 issues in a seperate document.  I am
open to suggestions.

>    differentiator for this work from other bodies' work.

I disagree.  The purpose of the terminology document is to provide the
metrics which will be used in the methodology document.  It is the job
of the methodology document, not the technology document, to specify
the  procedures used to quantify the impact of the value of a specific
metric on a IP and above.  As a result, the impact of AAL5 reassembly
time will be called out in a specific test for this value in the
methodology document.

Again, thank you for your comments.

-- Jeff

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 22 Apr 1999 09:10:59 -0400

>time will be called out in a specific test for this value in the

Jeff, no, I think we agree on the purpose of the terminology 
document as you specified above. ;-) I think I'm possibly confused
on visibility/context/scope.  For instance, you define a CER metric 
in this draft. As I read it, this appears to be an ATM-centric metric. 
If, through the CER metric's methodology, you are quantifying 
something other than cell behavior (i.e., higher layer behavior) as
specified in the CER metric definition, shouldn't the definition
reflect what you're quantifying?  What I'm hearing, I'm to go
the methodology document to see whether your metric collection heuristic
paints Cell Transfer Delay in the context of packet 
latency or packet forwarding rate, right?

---

From: Jeffrey Dunn <"Jeffrey,Dunn"@worldnet.att.net>
Date: Fri, 23 Apr 1999 23:16:17 -0400

Kevin,

I see where you are going with this.  When I looked at RFC 1242, I
noticed that mostof the discussions were longer than the definitions. 
>From that point of view, there needs to be a lot more background
associating the ATM layer imparements with IP and higher layer
performance issues.  I have no problem calling out the impact specific
metrics will have on higher layer performance; however, I offer three
cautions.  First, even if we remove all mention of TM4.0 issues and
specify UBR as the only Really Useful Service Class, this will greatly
increase the size of the document (and, consequently, the work which I
need to do on it ;->).  I heard comments at the last meeting about
shorter documents being better.  Second, ATM test is somewhat a black
art.  If you do not understand the implications, do not try this at
home.  Third, I do not want the reader to miss the forest for the trees.
Most of the metrics will effect several of the higher layers and effect
them differently.  At this juncture, one has to ask: Where do I stop? 
Is it sufficient to call out IP and TCP related performance issues or do
I need to explore FTP and OSPF performance issues.  Agreed, I am playing
devil's advocate with  point three; however, it remains an issue.  I
agree that we need to get a consensus within the working group as to the
scope of this document.  As always, I welcome comments from the silent
majority.

-- Jeff

>Kevin Dubray wrote:

---

From: Kevin Dubray <[email-masked]>
Date: Mon, 26 Apr 1999 09:44:12 -0400

Jeffrey Dunn wrote:

> 
> Second, ATM test is somewhat a black

I think part of the problem is that the B-ISDN model, as defined
in the document, abstracts anything above the cell/aa5 level as
an all encompassing service layer.

My suggestion would be to bound the work to addressing  
characterizing IP (as opposed to TCP/FTP/OSPF) packet
forwarding/handling with respect to ATM being the primary factor.
Subsequent documents could handle other issues.  I think this
fufills your points to ATM instrumentation, focus, and brevity.

What do others think?

-Kevin