Subject: [clue] My comments on draft-romanow-clue-telepresence-requirements-02

From: "Roni Even" <[email-masked]>
Date: Thu, 5 May 2011 11:32:20 +0300

Hi,
I read the draft and the requirements and also looked at the email thread.

The one thing that bothered me was the terms used.
Requirement 1 and 2 talk about describing spatial arrangement.

Requirement 3 talk about spatial matching of audio and video - I am not sure that we need to have this as a MUST requirement. I understand lip synch will be required and is not mentioned or it can be that I do not understand what spatial matching means here.

Requirement 4 talk about characteristics and attribute. I saw characteristics mentioned in section 4 "Characteristics include number, placement, capture/render angle, resolution of cameras and screens,  spatial location and mixing parameters of microphones." And noticed that Allyn in the email thread had " Since characteristics of media streams during a telepresence conference may change, such as bandwidth, resolution". I think that we need to specify what we mean by endpoint characteristics and stream characteristics and are these the terms we should use in all the requirements.

Requirement 5 uses specified information about itself (see the comment on requirement 4). Is this the received endpoint receive characteristics?

Requirement 6 based on the above comment should probably discuss dynamic changes in stream characteristics.

Requirement 7 should talk about the streams characteristics and not the description of the streams.

I think that if we go to a common language in all requirements it will be easier to see the redundancy between the requirements.

I was wondering about requirement 14 - I take it that it means that it must be possible to open all media channels after one offer answer exchange. I am not sure if we should have such a strong statement currently. I think it is an important objective but based on my current experience with dual stream support (H.239 like) and the current media capability negotiation I think it is very challenging when considering   backward computability ("support for "legacy" systems).

I am also missing a requirement for "legacy" system support like the solution MUST enable a system that does not support the mechanism to select the best set of streams to render (audio + video).

Thanks
Roni Even

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Allyn Romanow (allyn)
> Sent: Friday, April 22, 2011 10:48 AM
> To: CLUE
> Subject: [clue] FW: New Version Notification for draft-romanow-clue-
> telepresence-requirements-02
> 
> Folks,
> I revised my requirement document based on the numerous helpful
> comments made during the WG meeting in Prague. An appendix outlines the
> main changes.
> 
> I’m delighted that Stephan Wenger has significantly contributed to the
> draft and agreed to be a co-author - much appreciation, Stephan.
> 
> Looking forward to discussion of the draft-
> 
> Best regards,
> Allyn
> 
> -------
> 
> A new version of I-D, draft-romanow-clue-telepresence-requirements-
> 02.txt has been successfully submitted by Allyn Romanow and posted to
> the IETF repository.
> 
> Filename:	 draft-romanow-clue-telepresence-requirements
> Revision:	 02
> Title:		 Requirements for Telepresence Multi-Streams
> Creation_date:	 2011-04-22
> WG ID:		 Independent Submission
> Number_of_pages: 10
> 
> Abstract:
> This memo discusses the requirements for a specification that enables
> telepresence interoperability, by describing the relationship between
> multiple RTP streams.  In addition, the problem statement and
> definitions are also covered herein.
> 
> 
> 
> The IETF Secretariat.
> 
> 
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Thu, 5 May 2011 08:45:47 -0500

Roni,

Think of it as eye sync.

No ventriloquism here.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Thursday, May 05, 2011 4:32 AM
To: 'CLUE'
Subject: [clue] My commentson draft-romanow-clue-telepresence-requirements-02

Hi,

Thanks
Roni Even

---

From: "Roni Even" <[email-masked]>
Date: Thu, 5 May 2011 19:41:12 +0300

Hi,
I am not sure since we were talking about a single audio stream and multiple video streams or is this for a different use case.
Roni

> From: Mike Hammer (hmmr) [mailto:[email-masked]]
> Sent: Thursday, May 05, 2011 4:46 PM
> To: Roni Even; CLUE
> Subject: RE: [clue] My commentson draft-romanow-clue-telepresence-
> requirements-02
> 
> Roni,
> 
> Think of it as eye sync.
> 
> No ventriloquism here.
> 
> Mike
> 
> 
> Requirement 3 talk about spatial matching of audio and video - I am not
> sure that we need to have this as a MUST requirement. I understand lip
> synch will be required and is not mentioned or it can be that I do not
> understand what spatial matching means here.
> 
> Requirement 4 talk about characteristics and attribute. I saw
> characteristics mentioned in section 4 "Characteristics include number,
> placement, capture/render angle, resolution of cameras and screens,
> spatial location and mixing parameters of microphones." And noticed
> that Allyn in the email thread had " Since characteristics of media
> streams during a telepresence conference may change, such as bandwidth,
> resolution". I think that we need to specify what we mean by endpoint
> characteristics and stream characteristics and are these the terms we
> should use in all the requirements.
> 
> Requirement 5 uses specified information about itself (see the comment
> on requirement 4). Is this the received endpoint receive
> characteristics?
> 
> Requirement 6 based on the above comment should probably discuss
> dynamic changes in stream characteristics.
> 
> I think that if we go to a common language in all requirements it will
> be easier to see the redundancy between the requirements.
> 
> 
> I was wondering about requirement 14 - I take it that it means that it
> must be possible to open all media channels after one offer answer
> exchange. I am not sure if we should have such a strong statement
> currently. I think it is an important objective but based on my current
> experience with dual stream support (H.239 like) and the current media
> capability negotiation I think it is very challenging when considering
> backward computability ("support for "legacy" systems).
> 
> I am also missing a requirement for "legacy" system support like the
> solution MUST enable a system that does not support the mechanism to
> select the best set of streams to render (audio + video).
> 
> 
> Thanks
> Roni Even
>

---

From: "Duckworth, Mark" <[email-masked]>
Date: Fri, 6 May 2011 14:05:33 -0700

Hi Roni,

REQMT-3 is about making the audio sound like it is coming from the same place as the image of the person who is speaking.  This is relevant for cases that use multi channel audio, for example common cases are stereo and left/center/right channels.  I think MUST is appropriate.

Synchronization between audio and video was a requirement in an earlier draft, but I think maybe the intent was for spatial matching rather than lip sync.  It probably wouldn't hurt to add another requirement about lip sync.

About characteristics - in an earlier email, I asked about the difference between REQMT-4 and REQMT-7, and Allyn said she thought they were redundant and one could be removed.  I agree, and my point is we should talk about characteristics of streams and not characteristics of endpoints.  If we think an endpoint characteristic is important, then it probably relates to the streams somehow and should be described in terms of the streams rather than the endpoint.

For REQMT-5, I interpret it to mean characteristics about the streams the receiver can (or wants to) receive.  Again, I think we should talk about stream characteristics and not endpoint characteristics.  This might be realized by the receiver asking the sender to send streams with certain characteristics.

You might be right about REQMT-14 being too strong, I wouldn't mind relaxing it.

Regards,
Mark Duckworth

> Subject: [clue] My comments on draft-romanow-clue-telepresence-
> requirements-02
> 
> Hi,

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 06 May 2011 15:39:32 -0700

Hi Mark, all,
Please see inline.

On 5.6.2011 14:05 , "Duckworth, Mark" <[email-masked]> wrote:

>Hi Roni,
>
>REQMT-3 is about making the audio sound like it is coming from the same
>place as the image of the person who is speaking.  This is relevant for
>cases that use multi channel audio, for example common cases are stereo
>and left/center/right channels.  I think MUST is appropriate.
>Synchronization between audio and video was a requirement in an earlier
>draft, but I think maybe the intent was for spatial matching rather than
>lip sync.  It probably wouldn't hurt to add another requirement about lip
>sync.

I think that lip-sync is something that is completely up to the
implementation to enable.  All the necessary protocol mechanisms are
already available in RTP.  You may even want to be have an option to
disable lipsync from the user interface.  There are many scenarios where
lip-sync is helpful, but there are also others where it is not (where you
want to render audio as quickly as you can, even at the cost of imperfect
lipsync, for example on intercontinental teleconferences where the
additive delay of video/lip-sync and lightspeed delay add up to a
non-conversational experience)

>[...]
>For REQMT-5, I interpret it to mean characteristics about the streams the
>receiver can (or wants to) receive.  Again, I think we should talk about
>stream characteristics and not endpoint characteristics.  This might be
>realized by the receiver asking the sender to send streams with certain
>characteristics.

Req 4 and 5 were meant as device characteristics.  Stuff such as the
spatial arrangement of microphones, cameras, screens, constraints the
codecs may have, and so on.  These are a superset of the stream
characteristics that are sent in that they can include permutations of
capabilities that are mutually exclusive, for example due to cycle
restrictions.  An example would be: in the 2003 TeleSuite system, for each
camera field, we were able to either code either VGA or a variety of
scaled down formats half-VGA, QVGA, and so on, that were be simulcast.

>You might be right about REQMT-14 being too strong, I wouldn't mind

The history of this requirement is that there was a requirement "solution
is not to add delay".  Allyn and I were dissecting what types of delay
could be meant, and figured out that it can't be media path delay (as this
is independent from the CLUE concepts), so it must be some other delay
that matters.  Call setup was the only one we came up with.  I don't mind
relaxing this, either, but do we really want to allow an additional round
trip over offer-answer?  If not, I don't see where significant adidtional
delay can stem from...

Stephan

>> > I¹m delighted that Stephan Wenger has significantly contributed to
>> the

---

From: Roni Even <[email-masked]>
Date: Sat, 07 May 2011 11:15:58 +0300

Hi,
My comments were trying to highlight the point that we need to specify what
we mean by characteristics in the requirements probably in section 1 or 4 of
the draft.

Like I said I noticed that the draft sometime means streams characteristics
and sometime endpoint characteristics. On endpoint characteristic, I am not
sure if it talks about the systems itself (number of cameras, screens, ..)
or also about the whole room when discussing spatial relation between the
audio and the video.

Thanks
Roni Even

> Stephan Wenger
> Sent: Saturday, May 07, 2011 1:40 AM
> To: Duckworth, Mark; CLUE
> Subject: Re: [clue] My comments on draft-romanow-clue-telepresence-
> requirements-02
> 
> Hi Mark, all,
> Please see inline.
> 
> >> Requirement 7 should talk about the streams characteristics and not
> the
> >> description of the streams.
> >>
> >> > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> >> Of
> >> > Allyn Romanow (allyn)
> >> > comments made during the WG meeting in Prague. An appendix
> outlines
> >> the
> >> > main changes.
> >> >

---

From: Roni Even <[email-masked]>
Date: Sat, 07 May 2011 11:23:01 +0300

Inline, 

> trip over offer-answer?  If not, I don't see where significant
'
I agree that the issue was call setup delay, but this requirement in my view
calls for one offer answer exchange and I think that we can relax it to a
SHOULD since we are not sure if this is feasible or of we may agree to add
some of the media channel in a second offer answer and I do not want the
requirement to prevent it.
I agree that it will be best if we can have one offer/answer exchange.

Roni

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Mon, 9 May 2011 19:53:16 -0700

Hi Stephan and Mark,
I am trying to sort out whether we need both #4 and #7 or not.. i.e., whether stream characteristics are adequate and reflect any thing of interest in the endpoint characteristics, which, I believe, is Mark's point. To me this sounds plausible, but  I'm not entirely sure.

Stephan, I have a question for you based on your comment below-

Regards,
Allyn

> Sent: Friday, May 06, 2011 3:40 PM

AR- So, do you think that stream characteristics, as expressed in #7 and capabilities as expressed in #12 cover it, and we take out #4?

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Tue, 10 May 2011 14:28:36 -0700

Hi-

The discussion has been quite good - arguments on both sides sound convincing. :)

Here's my take on several point, we can discuss during the virtual meeting this week-

Reqmt 3 - spatial matching means as Mark described an association of audio with video, without being lip sync. Will try to make the wording more clear.

There is disagreement about whether or not we need lip sync as a requirement. I side with Stephan at the moment, feeling it shouldn't be a requirement. 

Reqmt 4 and 7 - As you all point out, 4 is about endpoint characteristics and 7 is about the characteristics of streams.. so they are different. The question is whether we feel the need to describe endpoints - which may contain multiple devices - as well as describe streams.  There have been arguments on both sides. I tend to think we do need both, but could be proved wrong. Perhaps this is something to discuss at the meeting.

Roni has suggested rewording 4, 5, and 7 to use the word "characteristic" for clarity.  I don't see any harm in this.

There has been difference of opinion on the use of "SHOULD" and "MUST" in the requirements. Whether there should be shoulds at all, whether they should vary, or all be the same. 
I'm of 2 minds - with Peter in thinking, what's the point of a requirement that is not a MUST, while on the other hand wanting wiggle room in case something turns out to be not a good idea. So, what are the guidelines here? Look at each requirement individually and divine which makes more sense??

There seems general agreement to change Reqmt 14 to a SHOULD.

All the points that have been raised will be brought up during the virtual meeting.

Allyn

> Sent: Saturday, May 07, 2011 1:16 AM
> To: 'Stephan Wenger'; 'Duckworth, Mark'; 'CLUE'
> > >lip sync.  It probably wouldn't hurt to add another requirement
> about
> > lip
> > >sync.
> >
> > >For REQMT-5, I interpret it to mean characteristics about the
> streams
> > the
> > >> Requirement 3 talk about spatial matching of audio and video - I
> am
> > not
> > >> I was wondering about requirement 14 - I take it that it means
> that
> > it
> > >> > 02.txt has been successfully submitted by Allyn Romanow and
> posted
> > to
> > >> > the IETF repository.
> > >> >

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 11 May 2011 15:42:04 +0200

Hi,

I have reviewed the new version of the requirements and I do wonder if
this document needs to have some requirements more directly related to
the media plane, i.e. RTP transport of media.

To achieve interoperability it is quite clear from my perspective that
RTP related questions like:

- How one performs multiplexing over RTP
- If and how Simulcast is to be supported
- If relayed centralized models are to be supported there will be
questions around stream control.

Sure,
   REQMT-8:   The solution MUST include a mechanism by which the
              receiver (endpoint or MCU) can specify to the sender
              (endpoint or MCU) what streams it wants to receive.
can be interpreted to be applicable to parts of the last question.

But in general we have so far not discussed much on what the issues are
on the actual RTP media plane.

Are such requirements something that we would like to develop?

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 11 May 2011 16:01:56 +0200

> There has been difference of opinion on the use of "SHOULD" 
> and "MUST" in the requirements. Whether there should be 
> shoulds at all, whether they should vary, or all be the same. 
> I'm of 2 minds - with Peter in thinking, what's the point of 
> a requirement that is not a MUST, while on the other hand 
> wanting wiggle room in case something turns out to be not a 
> good idea. So, what are the guidelines here? Look at each 
> requirement individually and divine which makes more sense??
> 
[JRE] During recent work on RFC 5947
https://datatracker.ietf.org/doc/rfc5947/
we decided to split things into requirements, using MUST, and desirables, using SHOULD. The latter was use for things that we couldn't really quantify.
Since then I have heard opinions voiced that RFC 2119 language in this sort of document is inappropriate, in which case one could use "must" and "should" instead.

John

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 11 May 2011 16:22:04 +0200

Hi,

I know that you already discussed requirement 4 and 7. The
interpretation some colleagues and I have is that there is some
important difference between requirement 4 and 7.

First of all we do see that REQMT-4 and REQMT-5 are linked and appears
to relate to Endpoint and MCU information exchange.

REQMT-7 and REQMT-12 are also linked and relates to capabilities around
media streams.

Thus REQMT-4 and REQMT-7 appears to covered different things and are not
truly redundant. However, note that I say appear. Because REQMT-4 is
incredibly vague in its formulation. So vague that I am uncertain what
it really means. I think it should be substantially clarified what type
of attributes and characteristic is considered. Otherwise one can't
determine if any proposed solution is reasonable for the function or not.

Are this static information, or information that are dynamic. How big is
it, and what impact does it have on the system?

Cheers

Magnus Westerlund

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 11 May 2011 18:50:06 -0700

On 5.11.2011 07:01 , "Elwell, John" <[email-masked]>
wrote:

>we decided to split things into requirements, using MUST, and desirables,
>using SHOULD. The latter was use for things that we couldn't really
>quantify.
>Since then I have heard opinions voiced that RFC 2119 language in this
>sort of document is inappropriate, in which case one could use "must" and
>"should" instead.

Please note that the capitalization of keywords is neither required by
2119, nor does 2119 suggest that non-capitalized "must" or "should" ought
to be interpreted any different than capitalized MUST or SHOULD.  (I have
learned that monitoring whitebeard-traffic on ietf@ietf.)

Stephan

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Wed, 11 May 2011 12:52:11 -0700

Regarding the must/should question, I think that if it is not a must, it does not belong in the requirements section and is more appropriately addressed in the other sections of the document.
If we are not sure whether something is a real requirement or not, we can have more discussion on it until we determine which way to go.
In my opinion, Reqmt 14 is a goal worth keeping in mind, but it is not a requirement.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Allyn Romanow (allyn)
> Sent: Tuesday, May 10, 2011 2:29 PM
> To: Roni Even; Stephan Wenger; Duckworth, Mark; CLUE
> Subject: Re: [clue] My comments ondraft-romanow-clue-telepresence-requirements-02
> 
> Hi-
> 
> Reqmt 3 - spatial matching means as Mark described an association of audio with video, without being
> lip sync. Will try to make the wording more clear.
> 
> There is disagreement about whether or not we need lip sync as a requirement. I side with Stephan at
> the moment, feeling it shouldn't be a requirement.
> 
> Reqmt 4 and 7 - As you all point out, 4 is about endpoint characteristics and 7 is about the
> characteristics of streams.. so they are different. The question is whether we feel the need to
> describe endpoints - which may contain multiple devices - as well as describe streams.  There have
> been arguments on both sides. I tend to think we do need both, but could be proved wrong. Perhaps this
> is something to discuss at the meeting.
> 
> There has been difference of opinion on the use of "SHOULD" and "MUST" in the requirements. Whether
> there should be shoulds at all, whether they should vary, or all be the same.
> I'm of 2 minds - with Peter in thinking, what's the point of a requirement that is not a MUST, while
> on the other hand wanting wiggle room in case something turns out to be not a good idea. So, what are
> the guidelines here? Look at each requirement individually and divine which makes more sense??
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Wed, 11 May 2011 12:57:01 -0700

Hi Magnus,

I agree the RTP related questions are very important for CLUE. However, I am thinking it is better to address them within the scope of one of more other working group documents related to the proposed solution for CLUE (e.g. architecture or protocol document), not the requirements document.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
> Sent: Wednesday, May 11, 2011 6:42 AM
> To: Allyn Romanow (allyn)
> Cc: CLUE
> Subject: [clue] draft-romanow-clue-telepresence-requirements-02: RTP Media plane requirements
> 
> Hi,
>

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Wed, 11 May 2011 14:48:14 -0700

Hi Magnus,
Yes, most definitely, we need to develop these RTP related requirements.. 

But do you see them related to this requirements draft?  I think that this requirements draft is just for the requirements for the CLUE specification.. which is about exchanging information. 

The charter says that it is in scope to consider the Interop requirements for the RTP media plane and then bring them to appropriate WGs. I see it as a separate work item.

What do you think?

Best,
Allyn

> From: Magnus Westerlund [mailto:[email-masked]]
> Subject: draft-romanow-clue-telepresence-requirements-02: RTP Media
> plane requirements
> 
> Hi,
>