Subject: RE: [AVT] - Some - More Comments on draft-clark-avt-rtcpxr-video-02

From: "Fiandra Riccardo" <[email-masked]>
Date: Tue, 25 Jul 2006 20:04:51 +0200

Alan, Mark, all,

I am happy to have stimulated a bit of discussion in this items.
I'd like to restate the need for some simple metrics extension for measuring and having feedback on packet loss.
I have in mind some loss patterns that would be disclosed with the additions I proposed and some more that I think others had.
Again the target is quality evaluation and FEC design and understanding loss patterns in general.
Here is an example:
17000 as expected packets in one minute
14 as the number of lost packets in this minute.
By only knowing:

Number of loss events (e.g. = 4)
Max Loss Period 	    (e.g  = 6)
Min Loss Period	    (e.g  = 1)
Noticable Loss events  (e.g = 0) with a minimum loss distance of e.g. 100 packets

I think we have a very clear view of the loss patterns. Different examples might be done, but if the reporting interval or measurement sample is low enough (order of minutes) we can well understand how our end-to-end chain is (really) behaving.

Other metrics as pointed out by Mark and others, such as number_of_events_with_at_least_2_consecutive_loss or within a time interval are useful as well.
I believe that all these metrics are (very) easy to compute and could enrich the information we have from burst and gaps parameters.

Regards,

Riccardo

-----Original Message-----
From: Alan Clark [mailto:[email-masked]] 
Sent: martedì 25 luglio 2006 0.58
To: 'Mark Watson'; 'Hedayat, Kaynam'; Fiandra Riccardo; [email-masked]
Subject: RE: [AVT] - Some - More Comments on draft-clark-avt-rtcpxr-video-02

Mark

The loss model used in RFC3611, which is also contained in the video metrics draft, uses a sparse burst model that may also be a useful model to consider in the context of FEC.  This model divides the session into burst and gaps - where a gap is defined as a period of time during which any lost or discarded packet must be preceded and followed by at least Gmin received packets (where Gmin is typically 16).  This essentially means that during burst periods lost packets are separated by less than Gmin received packets
- and the model measures the length and average loss density of burst and gap periods.  
 
If on some stream the following parameters were measured:

Gap length 10,000 ms
Gap density 0.01 (i.e. 1%)
Burst length 1,200 ms
Burst density 0.23 (i.e. 23%)

Then you would know that on average, packet loss was occurring in bursts of 23% loss rate for periods of (on average) 1.2 seconds, separated by gaps of 10 seconds during which around 1% of packets were lost.  It should be fairly easy to relate this to FEC performance.

Regards

Alan

> -----Original Message-----
> From: Mark Watson [mailto:[email-masked]]
> Sent: Wednesday, July 19, 2006 12:44 PM
> To: Hedayat, Kaynam; Fiandra Riccardo; Alan Clark; [email-masked]
> Subject: RE: [AVT] - Some - More Comments on
> draft-clark-avt-rtcpxr-video-02
> 
> Kayam,
> 
> Yes, the P-One-Way-Loss-Noticable-Rate in RFC3357 is similar to one of 
> the counts I was suggesting: specifically the count of the events 
> where there are exactly 2 packet losses within the period of interest.
> 
> However, to be interesting for IPTV we would also like to know the 
> number of events in which there were 3, 4, 5, etc.
> losses within the period of interest. So, it would be better to have 
> several counters.
> 
> Given this, the term 'noticable' is probably not appropriate
> - since whether the losses are noticeable depends on lots of things.
> 
> The other question is whether the measurement period should be defined 
> as a number of packets or a period of time. When applying FEC 
> protection to a video stream then an FEC buffer is needed at the 
> receiver and the receiver must delay the playout by the duration of 
> this buffer (in time).
> Subsequently, FEC protection cannot be applied to blocks larger than 
> this (int time), since that would either starve the player or mean the 
> FEC data would arrive too late at the decoder. So, for FEC it's a time 
> period which is most interesting.
> 
> Of course, for constant rate streams, then a number of packets and a 
> time period are roughly equivalent, so it is only for variable rate 
> streams where this becomes interesting.
> 
> Finally, I'd just like to take the opportunity to point out a
> (non-technical) problem which I have observed with RFC3357: 
> Whilst the loss period and loss distance *streams* and other metrics 
> defined there completely describe the losses that were seen in the 
> stream, I have often seen people consider 'loss period' and 'loss 
> distance' as just two numbers (presumably averages) which are used to 
> characterize the whole stream. Of course these two numbers give very 
> limited information about the loss patterns observed and are certainly 
> not sufficient for understanding the QoE that might be obtained for a 
> video stream or what kind of loss mitigation is required.
> 
> Best regards,
> 
> Mark
> 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Alan Clark" <[email-masked]>
Date: Tue, 25 Jul 2006 15:40:03 -0400

Fiandra

With your example you would not know if the loss events were clustered
together or were widely dispersed, which would have quite an effect on
quality.

Also - loss events quite often occur as single lost packets within high loss
periods (i.e. sparse bursts) and this type of distribution is not captured
by loss period. You may want to take a look at the burst/ gap approach used
in RTCP XR before discounting it.

Regards

Alan

> From: Fiandra Riccardo [mailto:[email-masked]] 
> Sent: Tuesday, July 25, 2006 2:05 PM
> To: Alan Clark; Mark Watson; Hedayat, Kaynam; [email-masked]
> I'd like to restate the need for some simple metrics 
> extension for measuring and having feedback on packet loss.
> I have in mind some loss patterns that would be disclosed 
> with the additions I proposed and some more that I think others had.
> I think we have a very clear view of the loss patterns. 
> Different examples might be done, but if the reporting 
> interval or measurement sample is low enough (order of 
> minutes) we can well understand how our end-to-end chain is 
> (really) behaving.
> 
> Other metrics as pointed out by Mark and others, such as 
> number_of_events_with_at_least_2_consecutive_loss or within a 
> time interval are useful as well.
> I believe that all these metrics are (very) easy to compute 
> and could enrich the information we have from burst and gaps 
> parameters.
> 
> 
> Regards,
> 
> Riccardo
> 
> 
> 
>  
> The loss model used in RFC3611, which is also contained in 
> the video metrics draft, uses a sparse burst model that may 
> also be a useful model to consider in the context of FEC.  
> This model divides the session into burst and gaps - where a 
> gap is defined as a period of time during which any lost or 
> discarded packet must be preceded and followed by at least 
> Gmin received packets (where Gmin is typically 16).  This 
> essentially means that during burst periods lost packets are 
> separated by less than Gmin received packets
> Then you would know that on average, packet loss was 
> occurring in bursts of 23% loss rate for periods of (on 
> average) 1.2 seconds, separated by gaps of 10 seconds during 
> which around 1% of packets were lost.  It should be fairly 
> easy to relate this to FEC performance.
> 
> Regards
> 
> Alan
> 
> 
> 
>