Subject: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06

From: Ben Campbell <[email-masked]>
Date: Mon, 2 Oct 2017 16:28:07 -0500

Hi,

This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I would like to resolve at least the substantive comments prior to IETF LC.

Thanks!

Ben.
—————————

Substantive:

- General: If I understand correctly, this document lists and describes XR metrics that a WebRTC application might choose to negotiate and expose via the WebRTC API. It does not define any new XR metrics. That brings up two questions:

1) Why is this specific to WebRTC? For the most parts, all the arguments here would apply to any sort of RTP endpoint.

2) Why is this standards track? This sort of material is typically informational, or occasionally a BCP. There are a small number of 2119 keywords, but I’m not sure they are needed or appropriate. (Specifics below.) My initial instinct is that this should be informational.

- General: This needs an IANA considerations section, even if it just contains the statement to the effect of “This document makes no requests for IANA”.

- 1, last paragraph: “ The document also creates a registry containing identifiers from the metrics reported in the RTCP Sender, Receiver, and Extended Reports.”

It doesn’t actually do that. I understand from the shepherd’s report that this was intentionally removed.

— “All identifiers proposed in this document are RECOMMENDED to be implemented by an endpoint.  An endpoint MAY choose not to expose an identifier if it does not implement the corresponding RTCP Report. “

Does the “RECOMMENDED” apply to all endpoints or just WebRTC endpoints? If the latter, doesn’t that requirement belong (or need to update) some requirement from an RTCWEB (maybe rtp-usage) document, or even the API itself? (I suspect that this draft does not have standing to state this normatively.) What is meant by “MAY choose not to expose”. Is that talking about via the WebRTC API? If so, isn’t that up to the API definition? (That is, it shouldn’t be normative here.)

-2: If this document keeps the normative keywords, please use the updated boilerplate from RFC 8174. (I note that there are at least some uncapitalized “may”s that do not seem normative.)

- 5.2.2, last paragraph: “The following metrics can also be considered…”
Be considered by whom? Implementors? API designers?

- 6, first paragraph: “In practice the application MUST be able to query the statistic identifiers on both an incoming (remote) and outgoing (local) media stream.”
What does the MUST requirement apply to? The WebRTC API? This seems more like a statement of fact that “ the application needs to be able…”

-8: "Therefore encryption procedures, such as the ones suggested for a Secure RTCP (SRTCP), need to be used.”
The text should describe the reasons encryption is needed. Also, is this a new normative requirement, or a restatement of an existing requirement?

Editorial:

-1, paragraph 1:" If sufficient information (metrics or statistics) are provided to the applications, it can attempt to improve the media quality. “

s / “are provided” / “is provided”
s / applications / application

-3, 2nd paragraph: It would help to clarify that the references to section 5 and 6 are to this document, not RFC 3611.

— 3rd paragraph: "At the moment…”
Please clarify that is at the time of writing (not reading)

-5, 2nd paragraph: “ Application impact metrics mainly collect the information in the viewpoint of application … “
s / in / from
Also, please expand “FEC” on first mention.

- 5.1.1, first paragraph: “ Duplicate packets may be a result of network delays, which causes the sender to retransmit the original packets.”
s/ :delays, which” / “delays that”

— last paragraph: "For those RTCWEB services with jitter buffer…”
s/buffer/buffers

- 5.1.2, first paragraph:
First sentence is a comma splice.
 — “some transitory nature of the impairments”
Should this say “the transitory nature of some impairments”?
— “ Distributed burst provides a higher subjective quality”
“Burst distribution …”?
— Last paragraph: “ Hence, if WebRTC application”
Missing article (or should application be plural?)

- 5.2.1, title: Should “Discard” be “Discarded”?

-7: Please clarify that this is at the time of this writing.

---

From: Varun Singh <[email-masked]>
Date: Sun, 8 Oct 2017 06:11:08 -0700

Hi Ben,

See inline.

> 1) Why is this specific to WebRTC? For the most parts, all the arguments here would apply to any sort of RTP endpoint.

This draft was created initially as an input to RTCWEB and later as an
input to WebRTC Statistics API. Hence the use of WebRTC in the title
and the introduction. We could add to the introduction: "In general,
the metrics listed in this document can be exposed by any real-time
communications endpoint." Or something along those lines

> 2) Why is this standards track? This sort of material is typically informational, or occasionally a BCP. There are a small number of 2119 keywords, but I’m not sure they are needed or appropriate.…

The reason until recently (perhaps still exist) is that this was an
input from the IETF to W3C, I am not sure if there was an official
liaison statements sent to them? nonetheless, the W3C Statistics API
document needed a specification needed (a specification in any
standards body would suffice) for metrics to be accepted into the W3C
document.

In practice, it helped that I was the co-author of both the IETF and
W3C document and hence was able to make the case for the metrics in
W3C.

> - 1, last paragraph: “ The document also creates a registry containing identifiers from the metrics reported in the RTCP Sender, Receiver, and Extended Reports.”

This should be removed. Thanks for pointing this out.

> Does the “RECOMMENDED” apply to all endpoints or just WebRTC endpoints? If the latter, doesn’t that requirement belong (or need to update) some requirement from an RTCWEB (maybe rtp-usage) document…

It should be applicable to any endpoint, although it is pertinent or
timely for the WebRTC endpoint to expose these.
The purpose of the statement was to highlight that if the
middlebox/endpoint sends RTCP XRs then it should expose the data
available via the API.

I am open to changing to a non-normative lower case guidance if that
would be more suitable, although if this document is making
recommendations to the W3C document -- wouldn't we keep the uppercase?

co-authors, what do you think?

> -2: If this document keeps the normative keywords, please use the updated boilerplate from RFC 8174. (I note that there are at least some uncapitalized “may”s that do not seem normative.)

depending on the above, we should fix these to uppercase.

> Be considered by whom? Implementors? API designers?

Both.

> What does the MUST requirement apply to? The WebRTC API? This seems more like a statement of fact that “ the application needs to be able…”

correct, "needs to" is better.

> The text should describe the reasons encryption is needed. Also, is this a new normative requirement, or a restatement of an existing requirement?

I believe it is restatement alluding to the guidance in the
corresponding RTCP XRs.

> -7: Please clarify that this is at the time of this writing.

Thanks for your feedback and suggestions, we will fix these.

Regards,
Varun

-- 
Founder, CEO, callstats.io
http://www.callstats.io

Interested in networking, media quality, and diagnostics.
We are hiring!: www.callstats.io/jobs/

---

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Tue, 10 Oct 2017 05:56:15 +0000

Hi

Please see inline.

BR,
Rachel

> -----Original Message-----
> From: Varun Singh [mailto:[email-masked]]
> Sent: Sunday, October 08, 2017 9:11 PM
> To: Ben Campbell
> Cc: [email-masked]; xrblock
> Subject: Re: [xrblock] AD Evaluation of
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
> 
> Hi Ben,
> 
> See inline.
> 
> > This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I
> would like to resolve at least the substantive comments prior to IETF LC.
> >
> > Thanks!
> >
> > Ben.
> > —————————
> >
> > Substantive:
> >
> > - General: If I understand correctly, this document lists and describes XR
> metrics that a WebRTC application might choose to negotiate and expose via
> the WebRTC API. It does not define any new XR metrics. That brings up two
> questions:
> >
> This draft was created initially as an input to RTCWEB and later as an input to
> WebRTC Statistics API. Hence the use of WebRTC in the title and the
> introduction. We could add to the introduction: "In general, the metrics listed in
> this document can be exposed by any real-time communications endpoint." Or
> something along those lines
> 
> 
> > 2) Why is this standards track? This sort of material is typically informational,
> or occasionally a BCP. There are a small number of 2119 keywords, but I’m not
> sure they are needed or appropriate. (Specifics below.) My initial instinct is that
> this should be informational.
> >
> 
> The reason until recently (perhaps still exist) is that this was an input from the
> IETF to W3C, I am not sure if there was an official liaison statements sent to
> them? nonetheless, the W3C Statistics API document needed a specification
> needed (a specification in any standards body would suffice) for metrics to be
> accepted into the W3C document.
> 
> In practice, it helped that I was the co-author of both the IETF and W3C
> document and hence was able to make the case for the metrics in W3C.
> 
> > - General: This needs an IANA considerations section, even if it just contains
> the statement to the effect of “This document makes no requests for IANA”.
> >
> > - 1, last paragraph: “ The document also creates a registry containing
> identifiers from the metrics reported in the RTCP Sender, Receiver, and
> Extended Reports.”
> 
> > — “All identifiers proposed in this document are RECOMMENDED to be
> > implemented by an endpoint.  An endpoint MAY choose not to expose an
> > identifier if it does not implement the corresponding RTCP Report. “
> >
> > Does the “RECOMMENDED” apply to all endpoints or just WebRTC
> > endpoints? If the latter, doesn’t that requirement belong (or need to
> > update) some requirement from an RTCWEB (maybe rtp-usage) document, or
> > even the API itself? (I suspect that this draft does not have standing
> > to state this normatively.) What is meant by “MAY choose not to
> > expose”. Is that talking about via the WebRTC API? If so, isn’t that
> > up to the API definition? (That is, it shouldn’t be normative here.)
> >
> 
> It should be applicable to any endpoint, although it is pertinent or timely for the
> WebRTC endpoint to expose these.
> The purpose of the statement was to highlight that if the middlebox/endpoint
> sends RTCP XRs then it should expose the data available via the API.
> 
> I am open to changing to a non-normative lower case guidance if that would be
> more suitable, although if this document is making recommendations to the
> W3C document -- wouldn't we keep the uppercase?
> 

[Rachel]: The sentences are meant for WebRTC endpoints, but surely, it is applicable to any endpoints. RTP-usage document just describes RTCP SR and RR, and leaves RTCP XR undiscussed. That's also part of the reason why we have this draft. So I think it's better to keep the uppercase. But maybe some modification of the sentences should be made to make it clear that this is making recommendations to webRTC applications. How about
"
WebRTC endpoints are RECOMMENDED to implement all the metrics proposed in this document. An endpoint MAY choose not to expose an identifier if it does not implement the corresponding RTCP Report.
"
?

BR,
Rachel

---

From: Ben Campbell <[email-masked]>
Date: Thu, 12 Oct 2017 16:51:42 -0500

> 
> On Oct 8, 2017, at 8:11 AM, Varun Singh <[email-masked]> wrote:
> 
> Hi Ben,
> 
> See inline.
> 

To make sure I understand:That API document will reference this one? Do they require that specification to be standards track?

Was there discussion in xrblock about standards track verses BCP?

> W3C document and hence was able to make the case for the metrics in

Is that case already made? That is, does the need for a published RFC still exist?

> recommendations to the W3C document -- wouldn't we keep the uppercase?

In general, 2119 keywords are better used when we are talking about interoperability issues in a protocol. I suggest switching to lower case unless there’s some reason to believe the W3C won’t pay attention unless it’s in all-caps. (I don’t think that’s usually the case.)

---

From: Ben Campbell <[email-masked]>
Date: Thu, 12 Oct 2017 16:56:08 -0500

> On Oct 10, 2017, at 12:56 AM, Huangyihong (Rachel) <[email-masked]> wrote:
> 
> Hi
> 
> Please see inline.
> 
> BR,
> Rachel
> 

My concern is that it can be problematic to add normative requirements to things that already exist without updating those things. Otherwise, implementors may not learn about the new requirements. That is, they may not realize they need to even read this document. 

Would it make sense to ask for a reference to this from rtp-usage?

Ben.