Subject: [Gen-art] Gen-ART Telechat review of draft-ietf-lmap-use-cases-05

From: Ben Campbell <[email-masked]>
Date: Tue, 2 Dec 2014 15:46:41 -0600

I am the assigned Gen-ART reviewer for this draft. For background on
Gen-ART, please see the FAQ at
< http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Please wait for direction from your document shepherd
or AD before posting a new version of the draft.

Document: draft-ietf-lmap-use-cases-05
Reviewer: Ben Campbell
Review Date: 2014-12-02
IETF LC End Date: 2014-10-07
ESG Telechat date: 2014-12-04

Summary: The draft is basically ready for publication as an informational RFC. All of the actionable issues from my last call review of version 04 have been addressed. There are a few remaining editorial issues, below:

Nits/editorial comments:

-- General: This version is much improved from 04. However it still tends to overuse parentheses in ways that are unnecessary and tend to break the flow of reading. These can probably be handled by the RFC editor.

The following are editorial comments from my original review that I think need further work:

[...]

> 
> -- 2.1, third bullet, last sentence:
> 
> The sentence hard to parse. Is the first comma intended?

The sentence needs work. Suggestion:

"The ISP requires visibility into the end-to-end performance of home and enterprise networks,..."

[...]

> 
> -- 3.1, 1st para, sentence starting with "The panel..."
> 
> I'm confused by the nested lists, nested parentheses, and unexplained ellipses. Also, it seems to contain a comma splice. Are there missing words?

The comma splice is fixed. The nested lists and ellipses are still confusing. You might consider splitting lists out into separate "For example" sentences.

For example:
"For example, the operator's access technology might include fiber, HFC, or DSL. It might offer broadband speeds of ...."

I also suggest dropping "(say)" and and the sentence-starting "So..." from later in the paragraph.

> 
> 
> - 3.1, 1st para:
> 
> Can you provide a definition or reference for "mean opinion score"?

Not addressed.

> 
> 
> -- 3.2:
> 
> Overly complex sentence structure. Consider breaking into bullet lists. Something seems messed up near " along the lines..." . Maybe a cut and paste error?

The bullet list improves things. Bullet 2 still contains a list of examples  in the form of comma-spliced sentences.

[...]

> 
> 
> -- 4.1, 1st para, last sentence: "... mandate transparent information made available..."
> 
> Should that be "... be made available..."?

Fix attempted, but new typo "imade" introduced".

> 
> -- 4.2, 3rd paragraph:
> 
> Can you offer a definition for "probes"?

Not addressed.

---

From: Jari Arkko <[email-masked]>
Date: Thu, 4 Dec 2014 15:29:02 +0200

Thanks for your review, Ben!

Jari

On 02 Dec 2014, at 23:46, Ben Campbell <[email-masked]> wrote:

> _______________________________________________
> Gen-art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/gen-art

---

From: <[email-masked]>
Date: Mon, 15 Dec 2014 22:32:13 +0000

Ben, Jari,

thanks for the follow-up, here are some further proposals 
phil

> "The ISP requires visibility into the end-to-end performance of home and enterprise networks,..."

OLD

      o Understanding the quality experienced by customers. Alongside
      benchmarking competitors, gaining better insight into the user's
      service through a sample panel of the operator's own customers.
      The ISP requires a performance viewpoint of the end-to-end
      perspective, which includes: home/enterprise networks; peering
      points; Content Delivery Networks (CDNs); etc.

PROPOSED NEW

      o Understanding the quality experienced by customers. The network operator would like to gain better insight into the end-to-end performance experienced by its customers. This could incorporate home and enterprise networks, and the impact of peering, caching and Content Delivery Networks (CDNs). 

> The comma splice is fixed. The nested lists and ellipses are still confusing. You might consider splitting lists out into separate "For example" sentences.

OLD
The panel needs to include a representative sample for
   each of the operator's technologies (fiber, Hybrid Fibre-coaxial
   (HFC), DSL...) and broadband speeds (80Mb/s, 20Mb/s, basic...). 

PROPOSED NEW
The panel needs to include a representative sample of the operator's technologies and broadband speeds. For instance it might encompass speeds ranging from sub 8Mbps to over 100Mbps.

> I also suggest dropping "(say)" and and the sentence-starting "So..." from later in the paragraph.

agreed.
OLD
The operator would like the end-to-end view of
   the service, rather than (say) just the access portion.  So as well as
   simple network statistics like speed and loss rates, they want to
   understand what the service feels like to the customer. 

NEW
The operator would like the end-to-end view of
   the service, rather than just the access portion.

>> Can you provide a definition or reference for "mean opinion score"?

If a reference is really needed, then i guess http://en.wikipedia.org/wiki/Mean_opinion_score will do.

> The bullet list improves things. Bullet 2 still contains a list of examples  in the form of comma-spliced sentences.

OLD
      o Understand the impact of IPv6 before it makes it available to
      customers (will v6 packets get through, what will the latency be
      to major websites, what transition mechanisms will be most
      appropriate?)

PROPOSED NEW
      o Understand the impact of IPv6 before it is made available to
      customers. Questions such as these could be assessed: will v6 packets get through? what will the latency be
      to major websites? what transition mechanisms will be most appropriate?

> Fix attempted, but new typo "imade" introduced".

OLD
In some jurisdictions regulators mandate that transparent
   information imade available about service offers.

NEW
In some jurisdictions regulators mandate that transparent
   information be made available about service offers.

>> Can you offer a definition for "probes"?

I don't want to address this one. Formal terminology is in the LMAP framework, including the definition of "measurement agent". In this document we used the more informal term "probe", as it seems easy to understand without a formal definition. Since this use cases doc might have a wider readership, we wanted to avoid terminology and just give the reader the motivations and usages of LMAP measurement information.

---

From: Ben Campbell <[email-masked]>
Date: Mon, 15 Dec 2014 18:24:48 -0600

Thanks for the response. I removed sections that do not seem to need further discusson.

> On Dec 15, 2014, at 4:32 PM, <[email-masked]> <[email-masked]> wrote:
> 

[...]

>      o Understanding the quality experienced by customers. The network operator would like to gain better insight into the end-to-end performance experienced by its customers. This could incorporat…

Looks good, but do you expect the last sentence to be exhaustive, or are those just examples?

> If a reference is really needed, then i guess http://en.wikipedia.org/wiki/Mean_opinion_score will do.

On a quick search, it looks like that's at least discussed (and maybe defined) in ITU-T P.800.

http://www.itu.int/rec/T-REC-P.800-199608-I/en

[...]

> I don't want to address this one. Formal terminology is in the LMAP framework, including the definition of "measurement agent". In this document we used the more informal term "probe", as it seems…

Could you offer an informal definition, or a "such as [example]" sort of thing? "Probe" has many meanings. Context might be enough for some readers, but possibly not all.

---

From: <[email-masked]>
Date: Tue, 16 Dec 2014 09:12:39 +0000

Thanks!
phil

> >      o Understanding the quality experienced by customers. The
> network operator would like to gain better insight into the end-to-end
> performance experienced by its customers. This could incorporate home
> and enterprise networks, and the impact of peering, caching and Content
> Delivery Networks (CDNs).	
> 

"End-to-end" could, for instance, incorporate home and enterprise networks, and the impact of peering, caching and Content Delivery Networks (CDNs).

> > If a reference is really needed, then i guess
> http://en.wikipedia.org/wiki/Mean_opinion_score will do.
> >	
> 

I'll add whichever reference you prefer - wiki or P.800 which is specifically about telephony.
The context is:
<<This involves
   relating the pure network parameters to something like a 'mean
   opinion score' which will be service dependent (for instance web
   browsing QoE is largely determined by latency above a few Mb/s).

> > I don't want to address this one. Formal terminology is in the LMAP
> framework, including the definition of "measurement agent". In this
> document we used the more informal term "probe", as it seems easy to
> understand without a formal definition. Since this use cases doc might
> have a wider readership, we wanted to avoid terminology and just give
> the reader the motivations and usages of LMAP measurement information.
> 
> Could you offer an informal definition, or a "such as [example]" sort
> of thing? "Probe" has many meanings. Context might be enough for some
> readers, but possibly not all.

In S3.1 probe is first mentioned. So then I'll add this text:-

NEW
(A probe is a device or piece of software that makes measurements and reports the results, under the control of the measurement system. Implementation options are discussed in Section 5.)

And in S5
OLD
One way involves a special hardware device that is connected directly
   to the home gateway.
...
Another approach involves implementing the measurement capability as
   a webpage or an "app" that end users are encouraged to download onto
   their mobile phone or computing device.

NEW
One type of probe is a special hardware device that is connected directly
   to the home gateway.
...
Another type of probe involves implementing the measurement capability as
   a webpage or an "app" that end users are encouraged to download onto
   their mobile phone or computing device.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 16 Dec 2014 09:26:26 -0600

Thanks! More inline:

> On Dec 16, 2014, at 3:12 AM, [email-masked] wrote:
> 
> Thanks!
> phil
> 
>> 
>>> OLD
>>> 

Works for me!

>   browsing QoE is largely determined by latency above a few Mb/s).

On reflection, given the context of this being a very "informational" reference, and that there doesn't seem to be an authoritative source for the term when used outside of telephony, it's not a big deal one way or another. The wiki link would be sufficient, as would a one sentence explanation along the lines as an "aggregation of subjective user experience measurements."

>   their mobile phone or computing device.

That helps, thanks!