Subject: bmwg Internet tunnelling protocol benchmarking

From: Ron Johnson <[email-masked]>
Date: Tue, 2 Feb 1999 15:48:35 -0800

Hello BMWG-

At Kevin Dubray's suggestion, I would like to solicit your opinions on an
appropriate way to benchmark tunnelling protocol implementation such as GRE.
In my experience, I find that performance measurements of networking gear as
per Scott Bradner's informational RFC fairly closely represents what we can
expect in "real" world use.  However, due to the variability in Internet
conditions, it seems nearly impossible to come up with a testing methodology
for tunnelling protocols that will provide the same applicability of data.

Outside of the actually variability of Internet conditions, the protocol
and/or encryption implementation could further aggravate the end node's
actual throughput.  It is possible such a protocol implementation could be
excellent in a no/low loss situation and very poor once packet rates and
latency increases.  This is what I am trying to avoid.

Any experience or thoughts you may have on this are invited.  If such
discussion leads to a possible methodology and the consensus is that IP
tunnelling is a viable technology, I would be very interested in helping to
draft an informational RFC.  

Sincerely-
Ron Johnson
Sr. Network Engineer

---

From: Kevin Dubray <[email-masked]>
Date: Tue, 09 Feb 1999 10:05:56 -0500

> draft an informational RFC.

Hi Ron,

I would presume that the folks working on MPLS are betting that
tunnelling is a viable Internet technology! ;-)

In any event, your original mail brings up several good points.  I
glean two central themes:

     1. Ways to benchmark tunnelling technologies.
     2. Need to inject "realism" into clinical evaluations.

But in order to make progress, I believe you need to answer the 
first question Mr. McQuaid put forth, namely, what do you want
to measure and where?

If attempting to characterize application-level performance using a
given end-to-end network technology, then simulating the aggregated
loss/delay/sequencing errors/etc. propagated by an abstracted network
may give the understanding that you seek.(Much work has been done in
this area for TCP, so you already have founding work.)

It may be beneficial to try to characterize the forwarding 
performance of networking element using protocols that are dependent 
on a historical event as a condition of forwarding (packet sequencing,
packet reassembly, etc.).  Then, again, not all tunnelling protocols
have such requirements, so loss may have no impact on the
discrete action of forwarding by NEs.

Bottomline, there are many ways to skin the cat. The question is
_which_ cat?  ;-)  If the BMWG were to formally undertake such work, I
for one, would like to see a crisper problem set so that we could
make forward progress. 

-Kevin