Subject: [AVT] Fw: I-D Action:draft-wu-avt-rtcp-xr-quality-monitoring-02.txt

From: Qin Wu <[email-masked]>
Date: Fri, 10 Sep 2010 18:20:43 +0800

Hi, folks:
02 version is posted which 
* break mixed metrics into resuable small metrics
* catogerize all the metrics into transport layer metrics and application layer metrics.
* rephrase syncronization metrics and perceptional quality metrics
Thanks Roland to join this work. Thanks Colin to comment this I-D.
The new draft is aviaible at:
http://www.ietf.org/id/draft-wu-avt-rtcp-xr-quality-monitoring-02.txt
Your comments and suggestions are welcome!

Regards!
-Qin
----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Sent: Friday, September 10, 2010 5:15 PM
Subject: I-D Action:draft-wu-avt-rtcp-xr-quality-monitoring-02.txt

>A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> Title           : RTP Control Protocol Extended Reports (RTCP XR) Report Blocks for Real- time Video Quality Monitoring
> Author(s)       : W. Wu, et al.
> Filename        : draft-wu-avt-rtcp-xr-quality-monitoring-02.txt
> Pages           : 27
> Date            : 2010-09-10
> 
> This document defines a set of RTP Control Protocol Extended Reports
> (RTCP XR) Report Blocks and associated SDP parameters allowing the
> report of video quality metrics, primarily for video applications of
> RTP.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-wu-avt-rtcp-xr-quality-monitoring-02.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
>

--------------------------------------------------------------------------------

> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

---

From: Colin Perkins <[email-masked]>
Date: Fri, 17 Sep 2010 10:50:23 +0100

I've read this draft, and have some comments. In general, I think it's  
on the right lines, but there are a number of details to be resolved.

Initial synchronisation delay: okay

Audio-video playout offset: makes sense in principle, but is defined  
in a way that’s specific to a simple scenario with one audio and one  
video stream. This may be better defined as a general synchronisation  
offset metric, which reports the offset of this stream relative to  
other streams sent with the same RTCP CNAME (where the reference  
stream reports offset of zero, and the others are signed offsets  
relative to it). That would allow it to be used to report on more  
complex synchronisation scenarios, rather than define other similar  
metrics for those scenarios later

In the layered stream statistics report block, the layer type flag is  
phrased in terms of H.264 SVC video, but there is nothing particularly  
SVC-specific here. Can it be written in a more general way, just  
specifying base and enhancement layers, so the same metric can be used  
for other layered codecs?

It’s not clear that the loss and duplicate reports need to be optional  
in the layered streams statistics metric block. They are straight  
forward to calculate, and making them optional seems to only introduce  
unnecessary complexity into the protocol. Similar for the video  
statistics summary report block.

The definition of the lost_layered component packets and dup_layered  
component packets is under-specified.

Is the frame type indicator field used in the video statistics report  
block and the video stream loss and discard metrics block well  
defined? The characterisation of video frames into I, P, and B frames  
makes sense for MPEG-2, but is it appropriate for other codecs?

Unlike the other metrics, the TR 101 290 decodability metrics report  
block is entirely MPEG-2 specific. I suggest splitting this into a  
separate draft, since it doesn’t really relate to the other metrics  
defined in this draft.

The RTP Terminal Configuration Metrics Block seems ill-thought out.  
The PLC type field doesn’t seem appropriate to video - the definition  
looks to be copied from an audio metrics draft. The JBA field is ill- 
defined, and doesn’t give meaningful information. The FEC field is  
very limited, and too tied to particular FEC schemes in a way that is  
not desirable: the media types namespace should be used here, rather  
than defining a new, very limited, two-bit field, if you need to do  
this at all. It's not clear that this information should be reported  
in RTCP, since the same information is available in the SDP signalling  
channel.

Colin

On 10 Sep 2010, at 11:20, Qin Wu wrote:
>> Title           : RTP Control Protocol Extended Reports (RTCP XR)  
>> Report Blocks for Real- time Video Quality Monitoring

-- 
Colin Perkins
http://csperkins.org/

---

From: Qin Wu <[email-masked]>
Date: Mon, 20 Sep 2010 15:37:24 +0800

Hi, Colin:
Thank for your valuable comments. Please see my reply inline.

Regards!
-Qin
----- Original Message ----- 
From: "Colin Perkins" <[email-masked]>
To: "Qin Wu" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Friday, September 17, 2010 5:50 PM
Subject: Re: [AVT] Fw: I-D Action:draft-wu-avt-rtcp-xr-quality-monitoring-02.txt

[Qin]: Yes, you are right. The main changes we made for this draft is try to align with Monitoring Architecture draft,
which more focus on how to extend metric report using RTCP XR for monitoring purpose in the latest version.

[Qin]: Good suggestion. One generic metric will benefit for a large broad appliations.
it is reasonable to replace the audio-video playout offset report with
one more genric reusable metric. I will do this  in the next version.

[Qin]: I am not sure whether we need more finely granularity of enchancement layers
to distinguish one enhancment layer from in other layered codecs
Is it reasonable to only specify base and enhancment layers?

[Qin] Good point. It seems these two indication fields for loss and duplication are
over-designed, I will remove them in the new version.

[Qin]: Okay, thank for catching this. we will specify this in the new version.

[Qin]: I think classifying video media into  I, P and frames also works for MPEG-4, but they may take the different name like
I-VOP, B-VOP and P-VOP, for the other video Codec like V6, it also characterize the video into different picture types like I and P.

Therefore may I propose to use  key frame and reference frame to replace I,P, and B frames,
which is more generic and can be used to distinguish different picture types. 

[Qin]: I am not sure whether we should leave TR101 290 decodability metrics out from this draft.
But I think this metrics could be taken as one of input to calculate the MOS-V which reflects the 
MPEG-2 media quality in some way.

[Qin]: I agree the parameters defined in this metrics can be obtained in the SDP signaling. 
It seems this Terminal related metrics looks like a evil. :-)
But what I like to argue about
is different type of terminal with differnt capability may demonstrate different reception quality
to the users. Are we aware of that? Isn't valuable to take this terminal associated metrics into 
accout when we evaluate the media quality in the comprehensive way?

Colin

-- 
Colin Perkins
http://csperkins.org/