Subject: Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06

From: "Huangyihong (Rachel)" <[email-masked]>
Date: Thu, 2 Nov 2017 10:00:33 +0000

Hi Ben,

After discussion among the co-authors, we reached the consensus to make it informational. W3C stats API still refers to it, but in an informational way. We'll submit a new version to address your comments.

BR,
Rachel

> -----Original Message-----
> From: Ben Campbell [mailto:[email-masked]]
> Sent: Friday, October 13, 2017 5:52 AM
> To: Varun Singh
> Cc: [email-masked]; xrblock
> Subject: Re: [xrblock] AD Evaluation of
> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06
> 
> >
> > On Oct 8, 2017, at 8:11 AM, Varun Singh <[email-masked]> wrote:
> >
> > Hi Ben,
> >
> > See inline.
> >
> >> On Tue, Oct 3, 2017 at 12:28 AM, Ben Campbell <[email-masked]>
> wrote:
> >> Hi,
> >>
> >> This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I
> would like to resolve at least the substantive comments prior to IETF LC.
> >>
> >> Thanks!
> >>
> >> Ben.
> >> —————————
> >>
> >> Substantive:
> >>
> >> - General: If I understand correctly, this document lists and describes XR
> metrics that a WebRTC application might choose to negotiate and expose via
> the WebRTC API. It does not define any new XR metrics. That brings up two
> questions:
> >>
> >> 1) Why is this specific to WebRTC? For the most parts, all the arguments
> here would apply to any sort of RTP endpoint.
> >>
> >
> > This draft was created initially as an input to RTCWEB and later as an
> > input to WebRTC Statistics API. Hence the use of WebRTC in the title
> > and the introduction. We could add to the introduction: "In general,
> > the metrics listed in this document can be exposed by any real-time
> > communications endpoint." Or something along those lines
> >
> >
> >> 2) Why is this standards track? This sort of material is typically informational,
> or occasionally a BCP. There are a small number of 2119 keywords, but I’m not
> sure they are needed or appropriate. (Specifics below.) My initial instinct is that
> this should be informational.
> >>
> >
> > The reason until recently (perhaps still exist) is that this was an
> > input from the IETF to W3C, I am not sure if there was an official
> > liaison statements sent to them? nonetheless, the W3C Statistics API
> > document needed a specification needed (a specification in any
> > standards body would suffice) for metrics to be accepted into the W3C
> > document.
> 
> To make sure I understand:That API document will reference this one? Do they
> require that specification to be standards track?
> 
> Was there discussion in xrblock about standards track verses BCP?
> 
> >
> > In practice, it helped that I was the co-author of both the IETF and
> > W3C document and hence was able to make the case for the metrics in
> > W3C.
> 
> Is that case already made? That is, does the need for a published RFC still
> exist?
> 
> >
> >> - General: This needs an IANA considerations section, even if it just contains
> the statement to the effect of “This document makes no requests for IANA”.
> >>
> >> - 1, last paragraph: “ The document also creates a registry containing
> identifiers from the metrics reported in the RTCP Sender, Receiver, and
> Extended Reports.”
> >
> > This should be removed. Thanks for pointing this out.
> >
> >>
> >> It doesn’t actually do that. I understand from the shepherd’s report that
> this was intentionally removed.
> >>
> >> — “All identifiers proposed in this document are RECOMMENDED to be
> >> implemented by an endpoint.  An endpoint MAY choose not to expose an
> >> identifier if it does not implement the corresponding RTCP Report. “
> >>
> >> Does the “RECOMMENDED” apply to all endpoints or just WebRTC
> >> endpoints? If the latter, doesn’t that requirement belong (or need to
> >> update) some requirement from an RTCWEB (maybe rtp-usage) document,
> >> or even the API itself? (I suspect that this draft does not have
> >> standing to state this normatively.) What is meant by “MAY choose not
> >> to expose”. Is that talking about via the WebRTC API? If so, isn’t
> >> that up to the API definition? (That is, it shouldn’t be normative
> >> here.)
> >>
> >
> > It should be applicable to any endpoint, although it is pertinent or
> > timely for the WebRTC endpoint to expose these.
> > The purpose of the statement was to highlight that if the
> > middlebox/endpoint sends RTCP XRs then it should expose the data
> > available via the API.
> >
> > I am open to changing to a non-normative lower case guidance if that
> > would be more suitable, although if this document is making
> > recommendations to the W3C document -- wouldn't we keep the uppercase?
> 
> In general, 2119 keywords are better used when we are talking about
> interoperability issues in a protocol. I suggest switching to lower case unless
> there’s some reason to believe the W3C won’t pay attention unless it’s in
> all-caps. (I don’t think that’s usually the case.)
> 
> >
> > co-authors, what do you think?
> >
> >> -2: If this document keeps the normative keywords, please use the
> >> updated boilerplate from RFC 8174. (I note that there are at least
> >> some uncapitalized “may”s that do not seem normative.)
> >>
> >
> > depending on the above, we should fix these to uppercase.
> >
> >> - 5.2.2, last paragraph: “The following metrics can also be considered…”
> >> Be considered by whom? Implementors? API designers?
> >>
> >
> > Both.
> >
> >> - 6, first paragraph: “In practice the application MUST be able to query the
> statistic identifiers on both an incoming (remote) and outgoing (local) media
> stream.”
> >> What does the MUST requirement apply to? The WebRTC API? This seems
> more like a statement of fact that “ the application needs to be able…”
> >>
> >
> > correct, "needs to" is better.
> >
> >> -8: "Therefore encryption procedures, such as the ones suggested for a
> Secure RTCP (SRTCP), need to be used.”
> >> The text should describe the reasons encryption is needed. Also, is this a
> new normative requirement, or a restatement of an existing requirement?
> >>
> >
> > I believe it is restatement alluding to the guidance in the
> > corresponding RTCP XRs.
> >
> >> Editorial:
> >>
> >> -1, paragraph 1:" If sufficient information (metrics or statistics)
> >> are provided to the applications, it can attempt to improve the media
> >> quality. “
> >>
> >> s / “are provided” / “is provided”
> >> s / applications / application
> >>
> >> -3, 2nd paragraph: It would help to clarify that the references to section 5
> and 6 are to this document, not RFC 3611.
> >>
> >> — 3rd paragraph: "At the moment…”
> >> Please clarify that is at the time of writing (not reading)
> >>
> >> -5, 2nd paragraph: “ Application impact metrics mainly collect the
> >> information in the viewpoint of application … “ s / in / from Also,
> >> please expand “FEC” on first mention.
> >>
> >> - 5.1.1, first paragraph: “ Duplicate packets may be a result of network
> delays, which causes the sender to retransmit the original packets.”
> >> s/ :delays, which” / “delays that”
> >>
> >> — last paragraph: "For those RTCWEB services with jitter buffer…”
> >> s/buffer/buffers
> >>
> >> - 5.1.2, first paragraph:
> >> First sentence is a comma splice.
> >> — “some transitory nature of the impairments”
> >> Should this say “the transitory nature of some impairments”?
> >> — “ Distributed burst provides a higher subjective quality”
> >> “Burst distribution …”?
> >> — Last paragraph: “ Hence, if WebRTC application”
> >> Missing article (or should application be plural?)
> >>
> >> - 5.2.1, title: Should “Discard” be “Discarded”?
> >>
> >> -7: Please clarify that this is at the time of this writing.
> >>
> >
> > Thanks for your feedback and suggestions, we will fix these.
> >
> > Regards,
> > Varun
> >
> >
> > --
> > Founder, CEO, callstats.io
> > http://www.callstats.io
> >
> > Interested in networking, media quality, and diagnostics.
> > We are hiring!: www.callstats.io/jobs/

---

From: Roni Even <[email-masked]>
Date: Thu, 2 Nov 2017 11:25:11 +0000

Hi 
Just to clarify, the individual draft started as an Informational draft and just before becoming a WG draft it was changed to standard track. We see no WG decision for this and as Rachel said, Informational is OK
Roni

> From: xrblock [mailto:[email-masked]] On Behalf Of Huangyihong
> (Rachel)
> Sent: יום ה 02 נובמבר 2017 12:01
> To: Ben Campbell; Varun Singh
> Subject: Re: [xrblock] AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-
> metrics-06
> 
> Hi Ben,
> 
> After discussion among the co-authors, we reached the consensus to make it
> informational. W3C stats API still refers to it, but in an informational way.
> We'll submit a new version to address your comments.
> 
> BR,
> Rachel
> 
> > >> This is my AD Evaluation of
> > >> draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I
> > metrics that a WebRTC application might choose to negotiate and expose
> > via the WebRTC API. It does not define any new XR metrics. That brings
> > up two
> > questions:
> > >>
> > > This draft was created initially as an input to RTCWEB and later as
> > > an input to WebRTC Statistics API. Hence the use of WebRTC in the
> > > title and the introduction. We could add to the introduction: "In
> > > general, the metrics listed in this document can be exposed by any
> > > real-time communications endpoint." Or something along those lines
> > >
> > >
> > or occasionally a BCP. There are a small number of 2119 keywords, but
> > I’m not sure they are needed or appropriate. (Specifics below.) My
> > initial instinct is that this should be informational.
> > >>
> > >
> > > standards body would suffice) for metrics to be accepted into the
> > > W3C document.
> >
> > To make sure I understand:That API document will reference this one?
> > Do they require that specification to be standards track?
> >
> > Is that case already made? That is, does the need for a published RFC
> > still exist?
> >
> > >
> > identifiers from the metrics reported in the RTCP Sender, Receiver,
> > and Extended Reports.”
> > >
> > >> implemented by an endpoint.  An endpoint MAY choose not to expose
> > >> an identifier if it does not implement the corresponding RTCP
> > >> Report. “
> > >>
> > >> standing to state this normatively.) What is meant by “MAY choose
> > >> not to expose”. Is that talking about via the WebRTC API? If so,
> > >> isn’t that up to the API definition? (That is, it shouldn’t be
> > >> normative
> > >> here.)
> > >>
> > >
> > interoperability issues in a protocol. I suggest switching to lower
> > case unless there’s some reason to believe the W3C won’t pay attention
> > unless it’s in all-caps. (I don’t think that’s usually the case.)
> >
> > >
> > statistic identifiers on both an incoming (remote) and outgoing
> > (local) media stream.”
> > >> are provided to the applications, it can attempt to improve the
> > >> media quality. “
> > >>
> _______________________________________________
> xrblock mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xrblock

---

From: Dan Romascanu <[email-masked]>
Date: Thu, 2 Nov 2017 08:15:02 -0400

I agree.

Regards,

Dan

On Thu, Nov 2, 2017 at 7:25 AM, Roni Even <[email-masked]> wrote:

> Hi
> Just to clarify, the individual draft started as an Informational draft
> and just before becoming a WG draft it was changed to standard track. We
> see no WG decision for this and as Rachel said, Informational is OK
> Roni
>

---

From: Ben Campbell <[email-masked]>
Date: Thu, 2 Nov 2017 10:24:19 -0500

Thanks, that sounds good to me.

Is there a specific W3C document that references (or plans to reference) this? I ask because I expect people to ask if the draft has already “done its job”; that is, if it’s still relevant to publish as an RFC.

Ben.

> On Nov 2, 2017, at 7:15 AM, Dan Romascanu <[email-masked]> wrote:
> 
> I agree.
> 
> Regards,
> 
> Dan
> 
> 
>

---

From: Varun Singh <[email-masked]>
Date: Sun, 05 Nov 2017 05:05:03 +0000

webrtc statistics API makes an informational reference to this document.

On Thu, Nov 2, 2017, 08:24 Ben Campbell <[email-masked]> wrote:

> Is there a specific W3C document that references (or plans to reference)
> this? I ask because I expect people to ask if the draft has already “done
> its job”; that is, if it’s still relevant to publish as an RFC.
>
> Ben.
>
> > > > >> The text should describe the reasons encryption is needed. Also,
> is
> > > > >> this a