Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

From: "Hedayat, Kaynam" <[email-masked]>
Date: Thu, 13 Jul 2006 12:40:17 -0400

Alan,
 
There is no need to re-interpret the draft. The same metrics can be used
to report transport unit loss. 
 
I don't disagree with your point that XR metrics as specified today are
useful. However, RFC3357 provides metrics and information that is not
available by the current draft. As far as XR deployments I don't know of
any deployments in video networks. However, I know of a handful of large
scale deployments that use RFC3357 metrics.
 
The goal of the standard should be adding flexibility and options for
use by the technology vendor and users. Not to limit the options.
 
I have serious concerns about pushing XR into reporting specific metrics
that are used by a specific model.
 
Regards,
Kaynam

________________________________

	From: Alan Clark [mailto:[email-masked]] 
	Sent: Wednesday, July 12, 2006 5:03 PM
	To: Hedayat, Kaynam; [email-masked]
	Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

	Kaynam
	 
	RFC3357's ideas on packet loss distribution seem largely based
on work done in the mid-1990's.  Several of the studies of packet loss
burstiness "only" considered the notion that bursts were consecutive -
and not surprisingly if you only look for some specific characteristic
then that is what you find.  I re-analyzed a number of the traces used
in some of these studies (e.g. Sue Moon's traces) however used a sparse
burst model instead of a consecutive loss model - and found significant
presence of sparse bursts.  Across a number of traces we've seen bursts
of 20-30% loss or loss/discard density that range up to several seconds
in length.
	 
	If you consider the impact of a consecutive loss burst vs a
sparse burst then the issue becomes apparent.  A burst of 2 or 3 lost
packets will cause a short term degradation lasting a few tens of
milliseconds for voice or audio, and several hundred milliseconds for
video.  A burst of 20% packet loss lasting for 2 seconds will completely
trash the decoded video for 2-3 seconds.  The burst model used in
RFC3611 can recognize either type of burst whereas a consecutive loss
model can only recognize consecutive loss.
	 
	If I were a service provider, I would not like the idea of a
measurement system that was insensitive to sparse bursts.
	 
	With regard to field deployment -  there are over 70 equipment
vendors that have/are deploying RFC3611 style metrics in their products,
with an installed base of several million units as of late 2005 -
expected to grow to 10-20 million units by the end of 2006. 
	 
	The video metrics draft specifies loss/discard metrics for
"transport units" and not "packets".   Video systems can use various
multiplexing algorithms, including packing multiple transport units for
the same video stream into one packet.  Are you suggesting that RFC3357
should be used as specified or should be re-interpreted to mean that
packets are transport units?
	 
	Alan

________________________________

		From: Hedayat, Kaynam [mailto:[email-masked]] 
		Sent: Wednesday, July 12, 2006 3:15 PM
		To: Alan Clark; [email-masked]
		Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

		Alan,
		 
		The argument is for not tying the model to the metrics
reported by XR or at a minimum have an option that will allow multiple
models. RFC3357 provides metrics that provide packet loss behavior in
terms of loss periods and loss distance, both applicable to real-time
applications. 
		 
		I disagree with your point that 3357 does not consider
packet discard. For the application, loss is loss, regardless of its
source (network or endpoint). Both kinds of loss can be described in
terms defined by 3357.
		 
		A lot of good work and time has been put into RFC3357
and its metrics are widely used in the field today, especially for Video
over IP applications (IPTV, VoD, etc.). I can't imagine any argument
against using these metrics.
		 
		Regards,
		Kaynam

________________________________

			From: Alan Clark
[mailto:[email-masked]] 
			Sent: Wednesday, July 12, 2006 2:29 PM
			To: Hedayat, Kaynam; [email-masked]
			Subject: RE: [AVT]
draft-clark-avt-rtcpxr-video-02

			Kaynam
			 
			You are quite incorrect.
			 
			(i) The burst and gap metrics were originally
defined in RFC3611 (RTCP XR) and, as they are extremely helpful in
understanding the impact of IP impairments on VoIP, were then adopted in
eight other standards related to VoIP performance.    Hence the original
work was done in AVT and then adopted across the industry
			 
			(ii) The concepts used in RFC3611 are quite
different to those in RFC3357 for good reason.  We analyzed the
distribution of lost packets on many traces and found that "burstiness"
tended to be exhibited as periods of high loss and NOT as periods of
consecutive loss. We also found that periods of high PDV typically
coincide with periods of high loss, and hence see merit in measuring the
impact of loss and PDV together (i.e. loss/discard distribution).
			 - The loss period you refer to from RFC3357
defines only consecutive loss and not the sparse bursts that typically
occur (particularly at the network edge), whereas RFC3611 describes
sparse bursts
			 - RFC3357 only applies to lost packets, and not
to packets discarded due to late arrival, whereas the model from RFC3611
applies to the combined loss/discard distribution
			 
			A good summary of the rationale behind the
burst/gap model is
http://www.telchemy.com/reference/ITUSG12_PacketLossDistribution.pdf

			I totally disagree with your assertion that
RFC3357 metrics are better defined and more accurately represent packet
loss behavior - you are entitled to hold any opinion you like however if
you want to express an opinion should explain the justification for it.
Any of these measurements essentially comprise parameters of some
assumed model that describes the behavior of the system being measured.
You should try and ensure that the model is relevant with regard to the
aspect of the system behavior you are interested in.  You could argue
that the assumed model is or is not a good fit for the aspect of system
behavior that is of interest, or that the method for parameter
estimation is more or less accurate.
			 
			Alan

________________________________

				From: Hedayat, Kaynam
[mailto:[email-masked]] 
				Sent: Tuesday, July 11, 2006 11:49 PM
				To: [email-masked]
				Subject: [AVT]
draft-clark-avt-rtcpxr-video-02

				I have concerns with the packet loss
metrics reported in the proposed blocks (RTP and MPEG TS). The Burst and
Gap models are based on the VoIP metrics block of RTCP-XR that were used
for consistency with other standards for voice quality algorithms. I
suggest either replacing burst and gap with packet loss period and
packet loss distance based on RFC3357 or augmenting them with these
metrics. RFC3357 metrics are better defined and more accurately
represent packet loss behavior.

				Regards, 
				Kaynam 

----------------------------------------- 
				Kaynam Hedayat 
				Chief Technology Officer 
				Vice President, Engineering 

				Brix Networks 
				285 Mill Road 
				Chelmsford, MA 01824 
				tel. 978.367.5611 
				fax. 978.367.5700 
				[email-masked] 
				www.brixnet.com <http://www.brixnet.com>

				Service Quality Matters. Test the
performance and quality of your VoIP or IP video service at
http://www.TestYourVoIP.com <http://www.testyourvoip.com/>   and
http://www.TestYourIPVideo.com <http://www.testyouripvideo.com/>  

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt