Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

From: "Hedayat, Kaynam" <[email-masked]>
Date: Fri, 14 Jul 2006 11:17:33 -0400

Alan,
 
I was on jabber and my comment to your xr presentation did not go
through. I would like to emphasize that the argument is not against the
value of gap/bust model. It is for the value of rfc3357, especially for
video applications. Gap and burst models have been used and proven
valuable in VoIP. This has not been the case for video.
 
I will follow-up for the next IETF meeting.
 
Regards,
Kaynam

________________________________

	From: Hedayat, Kaynam [mailto:[email-masked]] 
	Sent: Thursday, July 13, 2006 12:40 PM
	To: Alan Clark; [email-masked]
	Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

	Alan,
	 
	There is no need to re-interpret the draft. The same metrics can
be used to report transport unit loss. 
	 
	I don't disagree with your point that XR metrics as specified
today are useful. However, RFC3357 provides metrics and information that
is not available by the current draft. As far as XR deployments I don't
know of any deployments in video networks. However, I know of a handful
of large scale deployments that use RFC3357 metrics.
	 
	The goal of the standard should be adding flexibility and
options for use by the technology vendor and users. Not to limit the
options.
	 
	I have serious concerns about pushing XR into reporting specific
metrics that are used by a specific model.
	 
	Regards,
	Kaynam

________________________________

		From: Alan Clark [mailto:[email-masked]] 
		Sent: Wednesday, July 12, 2006 5:03 PM
		To: Hedayat, Kaynam; [email-masked]
		Subject: RE: [AVT] draft-clark-avt-rtcpxr-video-02

		Kaynam
		 
		RFC3357's ideas on packet loss distribution seem largely
based on work done in the mid-1990's.  Several of the studies of packet
loss burstiness "only" considered the notion that bursts were
consecutive - and not surprisingly if you only look for some specific
characteristic then that is what you find.  I re-analyzed a number of
the traces used in some of these studies (e.g. Sue Moon's traces)
however used a sparse burst model instead of a consecutive loss model -
and found significant presence of sparse bursts.  Across a number of
traces we've seen bursts of 20-30% loss or loss/discard density that
range up to several seconds in length.
		 
		If you consider the impact of a consecutive loss burst
vs a sparse burst then the issue becomes apparent.  A burst of 2 or 3
lost packets will cause a short term degradation lasting a few tens of
milliseconds for voice or audio, and several hundred milliseconds for
video.  A burst of 20% packet loss lasting for 2 seconds will completely
trash the decoded video for 2-3 seconds.  The burst model used in
RFC3611 can recognize either type of burst whereas a consecutive loss
model can only recognize consecutive loss.
		 
		If I were a service provider, I would not like the idea
of a measurement system that was insensitive to sparse bursts.
		 
		With regard to field deployment -  there are over 70
equipment vendors that have/are deploying RFC3611 style metrics in their
products, with an installed base of several million units as of late
2005 - expected to grow to 10-20 million units by the end of 2006. 
		 
		The video metrics draft specifies loss/discard metrics
for "transport units" and not "packets".   Video systems can use various
multiplexing algorithms, including packing multiple transport units for
the same video stream into one packet.  Are you suggesting that RFC3357
should be used as specified or should be re-interpreted to mean that
packets are transport units?
		 
		Alan

________________________________

			From: Hedayat, Kaynam
[mailto:[email-masked]] 
			Sent: Wednesday, July 12, 2006 3:15 PM
			To: Alan Clark; [email-masked]
			Subject: RE: [AVT]
draft-clark-avt-rtcpxr-video-02

			Alan,
			 
			The argument is for not tying the model to the
metrics reported by XR or at a minimum have an option that will allow
multiple models. RFC3357 provides metrics that provide packet loss
behavior in terms of loss periods and loss distance, both applicable to
real-time applications. 
			 
			I disagree with your point that 3357 does not
consider packet discard. For the application, loss is loss, regardless
of its source (network or endpoint). Both kinds of loss can be described
in terms defined by 3357.
			 
			A lot of good work and time has been put into
RFC3357 and its metrics are widely used in the field today, especially
for Video over IP applications (IPTV, VoD, etc.). I can't imagine any
argument against using these metrics.
			 
			Regards,
			Kaynam

________________________________

				From: Alan Clark
[mailto:[email-masked]] 
				Sent: Wednesday, July 12, 2006 2:29 PM
				To: Hedayat, Kaynam; [email-masked]
				Subject: RE: [AVT]
draft-clark-avt-rtcpxr-video-02

				Kaynam
				 
				You are quite incorrect.
				 
				(i) The burst and gap metrics were
originally defined in RFC3611 (RTCP XR) and, as they are extremely
helpful in understanding the impact of IP impairments on VoIP, were then
adopted in eight other standards related to VoIP performance.    Hence
the original work was done in AVT and then adopted across the industry
				 
				(ii) The concepts used in RFC3611 are
quite different to those in RFC3357 for good reason.  We analyzed the
distribution of lost packets on many traces and found that "burstiness"
tended to be exhibited as periods of high loss and NOT as periods of
consecutive loss. We also found that periods of high PDV typically
coincide with periods of high loss, and hence see merit in measuring the
impact of loss and PDV together (i.e. loss/discard distribution).
				 - The loss period you refer to from
RFC3357 defines only consecutive loss and not the sparse bursts that
typically occur (particularly at the network edge), whereas RFC3611
describes sparse bursts
				 - RFC3357 only applies to lost packets,
and not to packets discarded due to late arrival, whereas the model from
RFC3611 applies to the combined loss/discard distribution
				 
				A good summary of the rationale behind
the burst/gap model is
http://www.telchemy.com/reference/ITUSG12_PacketLossDistribution.pdf

				I totally disagree with your assertion
that RFC3357 metrics are better defined and more accurately represent
packet loss behavior - you are entitled to hold any opinion you like
however if you want to express an opinion should explain the
justification for it.  Any of these measurements essentially comprise
parameters of some assumed model that describes the behavior of the
system being measured.  You should try and ensure that the model is
relevant with regard to the aspect of the system behavior you are
interested in.  You could argue that the assumed model is or is not a
good fit for the aspect of system behavior that is of interest, or that
the method for parameter estimation is more or less accurate.
				 
				Alan

________________________________

				From: Hedayat, Kaynam
[mailto:[email-masked]] 
				Sent: Tuesday, July 11, 2006 11:49 PM
				To: [email-masked]
				Subject: [AVT]
draft-clark-avt-rtcpxr-video-02

				I have concerns with the packet loss
metrics reported in the proposed blocks (RTP and MPEG TS). The Burst and
Gap models are based on the VoIP metrics block of RTCP-XR that were used
for consistency with other standards for voice quality algorithms. I
suggest either replacing burst and gap with packet loss period and
packet loss distance based on RFC3357 or augmenting them with these
metrics. RFC3357 metrics are better defined and more accurately
represent packet loss behavior.

				Regards, 
				Kaynam 

----------------------------------------- 
				Kaynam Hedayat 
				Chief Technology Officer 
				Vice President, Engineering 

				Brix Networks 
				285 Mill Road 
				Chelmsford, MA 01824 
				tel. 978.367.5611 
				fax. 978.367.5700 
				[email-masked] 
				www.brixnet.com <http://www.brixnet.com>

				Service Quality Matters. Test the
performance and quality of your VoIP or IP video service at
http://www.TestYourVoIP.com <http://www.testyourvoip.com/>   and
http://www.TestYourIPVideo.com <http://www.testyouripvideo.com/>  

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt