Subject: Re: [bmwg] Draft BMWG Agenda Posted for IETF-79

From: Al Morton <[email-masked]>
Date: Tue, 09 Nov 2010 20:06:05 -0500

I'd like to propose a small revision to our draft
agenda.  Fred Baker has prepared a benchmarking proposal
and has asked for a few minutes at the beginning of our
session to discuss it, face2face with those present:
http://tools.ietf.org/html/draft-baker-bmwg-testing-eyeball-happiness-00

I've exchanged some messages with Fred and suggested
a few changes that would make this work more consistent with
our charter, such as removing the Pass/Fail threshold.
We can discuss these points on the list and at the session
on Thursday afternoon, Beijing time.

More background on this topic is available in the
Wing and Yourtchenko draft Fred references:
http://tools.ietf.org/html/draft-wing-v6ops-happy-eyeballs-ipv6-01
and the Sept 2010 edition of IPJ.

So, we'll take a few minutes for this at the start of
our session, and Fred can attend another session as soon
as we are done.

Al
bmwg chair

At 08:02 AM 10/27/2010, Al Morton wrote:
>BMWG,
>
>The draft agenda is posted:
>http://www.ietf.org/proceedings/79/agenda/bmwg.txt
>
>There will be a short status on several WG drafts that
>do not have presentations proposed, then we'll get to
>the full presentations and discussion.
>
>Re-chartering is proceeding, but not yet finalized.
>
>Please let me know if I've missed any agenda requests,
>Al
>bmwg chair
>
>_______________________________________________
>bmwg mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/bmwg

---

From: David Newman <[email-masked]>
Date: Wed, 10 Nov 2010 17:17:24 -0800

Interesting draft. I've seen this behavior but hadn't thought about
testing it.

In lieu of a pass/fail metric it might make sense to define
response-time goalposts, such as (warning: strawman example) the
interval between Alice's first DNS request and Bob's SYN/ACK.

Also, it probably would be helpful to clarify the procedure for testing
non-determinism in DNS servers (how many trials is adequate?).

Same thing for clearing Alice's resolver between iterations. I
understand the concept of trying with and without Bob's info in Alice's
resolver, but I'm less clear on the statement that "The fact that Alice
was able to contain Bob at a given address should not preclude Alice
trying other addresses on subsequent attempts."

Different implementations do different things, but in general I think
most prefer to use info from their own resolvers if it exists over
running another DNS query (and some do this even when they're not
supposed to). A simple requirement that resolvers should/should not be
cleared before a given test would address this.

dn

---

From: Al Morton <[email-masked]>
Date: Wed, 10 Nov 2010 20:42:22 -0500

Changing the subject line for WG Chair sanity...

Thanks for chiming-in, David.

>interval between Alice's first DNS request and Bob's SYN/ACK.

I shared a different possibility with Fred, reproduced below,
where the "Bob" Device would make the measurement:

 >>>>The question for Fred was this: Can this sort of design (with a
continuous benchmark metric, rather than Pass/Fail criteria)
meet the needs of the industry as you see them? I'll let Fred
add his response...

Using Figure 1 from Wing&Yourtechenko:

The "Time to complete ANY TCP 3-way Handshake" would be the interval
from the time when the first TCP SYN is received (any address family),
to the time when the ACK for the Server's SYN is received, all
observations made at the Server.

       DNS Server                Client (Alice?)         Server (Bob?)
           |                          |                       |
     1.    |<--www.example.com A?-----|                       |
     2.    |<--www.example.com AAAA?--|                       |
     3.    |---192.0.2.1------------->|                       |
     4.    |---2001:dba::1----------->|                       |
     5.    |                          |                       |
     6.    |                          |--TCP SYN, IPv6--->X   |<***********
     7.    |                          |--TCP SYN, IPv6--->X   |     |
     8.    |                          |--TCP SYN, IPv6--->X   | TCP 3wHS
     9.    |                          |                       |   Time
     10.   |                          |--TCP SYN, IPv4------->|(any family)
     11.   |                          |<-TCP SYN+ACK, IPv4----|     |
     12.   |                          |--TCP ACK, IPv4------->|<***********

             Figure 1: Existing behavior message flow

This Benchmark would have (at least) two categories
   - Initial Attempt
   - Subsequent Attempts  (where performance probably improves if
     Alice remembers the address family that worked last time)

---

From: Rick Jones <[email-masked]>
Date: Wed, 10 Nov 2010 17:58:10 -0800

> where the "Bob" Device would make the measurement:

If Bob is taking the measurement is that you have to instrument Bob's stack or 
use a packet trace.  Not impossible, but still more unpleasant than something 
that can be measured with some nice simple C code :)  And while ostensibly the 
RTT here is quite small, it will miss the half RTT it took Alice's first SYN to 
get to Bob.

I was under the impression from the problem statement that the important 
"Happiness" is Alice's, not Bob's, so it would be how soon does Alice have a 
successful/completed connect() call no?  That does not care about how long it 
takes Alice's ACK of Bob's SYN|ACK to get to Bob.

Of course, I suspect that what Alice *really* cares about is not simply how long 
it takes to get to a successfully completed connect() call, but how long it 
takes until the first bytes of what she wants from Bob to be displayed.  In that 
case, it would be how long to get a successfully completed connect() call, send 
a request to Bob and get a response back - more akin to a netperf TCP_CRR test 
than a netperf TCP_CC test.

That, or I've just had my Emily Litella moment for the day :)

rick jones

---

From: Al Morton <[email-masked]>
Date: Wed, 10 Nov 2010 21:57:18 -0500

Hi Rick,

I don't think you attained full-Emily status, but it
may help to know that we live in the lab here at old BMWG.

Since Alice appears would be the Device Under Test (DUT),
we can construct "Bob" ourselves to do what we want.
Also, we treat Alice as a Black-box, so observing standard
messages from Alice is a normal thing...

Al

At 08:58 PM 11/10/2010, Rick Jones wrote:
>If Bob is taking the measurement is that you have to instrument 
>Bob's stack or use a packet trace.  Not impossible, but still more 
>unpleasant than something that can be measured with some nice simple 
>C code :)  And while ostensibly the RTT here is quite small, it will 
>miss the half RTT it took Alice's first SYN to get to Bob.
>
>I was under the impression from the problem statement that the 
>important "Happiness" is Alice's, not Bob's, so it would be how soon 
>does Alice have a successful/completed connect() call no?  That does 
>not care about how long it takes Alice's ACK of Bob's SYN|ACK to get to Bob.
>
>Of course, I suspect that what Alice *really* cares about is not 
>simply how long it takes to get to a successfully completed 
>connect() call, but how long it takes until the first bytes of what 
>she wants from Bob to be displayed.  In that case, it would be how 
>long to get a successfully completed connect() call, send a request 
>to Bob and get a response back - more akin to a netperf TCP_CRR test