Subject: [maitai] Roles of Sender and Receiver

From: Peter Musgrave <[email-masked]>
Date: Thu, 2 Dec 2010 07:53:03 -0500

Hi, 

I have been pondering possible ways of representing a room configuration - but I think I need to back up and ask a question about basic interop architecture. 

In the selection of content on a stream, who is the active decision making entity, sender - or receiver - or both?

Consider a three display, 3 camera (3d3c) room A sending to a 1d1c room B. 

Does A offer all three streams to B and let B pick? (This leads in the direction of dynamic changes based on B, which is powerful but leads us into continuous control...)
Does A determine B has only one display and select which video B should get and send only one stream?

IHMO an approach which lets B have control over what it gets is more powerful. e.g. If I have heard the pitch from the speaker 100 times, I want to watch the customer and see if they are engaged, rather than watch the active speaker blather on....

Peter

---

From: "Duckworth, Mark" <[email-masked]>
Date: Thu, 2 Dec 2010 06:35:04 -0800

Peter,

That is a very good question.  I think we don't have an easy answer yet, it needs some discussion.  The issue gets a bit more complicated for multipoint, when there could be an MCU in the middle that is also part of the decision process.

>Does A offer all three streams to B and let B pick?

That is certainly a good possibility.  If B can choose dynamically, changing the choice during the session, it is similar to the current standard ability for B to select a video source from A using the H.281 SELECT VIDEO SOURCE message.  But the list of possible sources supported in H.281 isn't good enough for this application.

Maybe B wants to receive something like "current speaker stream" or "presenter stream", no matter which of A's three cameras it comes from.  I think of this as B making a higher level choice, and delegating the specific choice to A.

Maybe A is capable of sending a single stream that includes all the people in the room (perhaps by zooming out a camera, or combining three camera images into a single stream).  B might prefer to receive that type of stream.

I tend to agree an approach that gives B some control is more powerful and useful.  But I also think there are use cases where we want everything to be automatic, so we should not require a human user at site B to do anything in order to receive useful streams.

Mark Duckworth

-----Original Message-----

Peter

---

From: Peter Musgrave <[email-masked]>
Date: Thu, 2 Dec 2010 10:01:04 -0500

> That is certainly a good possibility.  If B can choose dynamically, changing the choice during the session, it is similar to the current standard ability for B to select a video source from A using…
Oh, dear. Another H.xxx protocol I don't know...

Are the semantics in H.281 controlling which video is placed on an existing media stream (i.e. stream still appears to come from same port/IP but the content within the stream changes) - or does the sender send from a new IP/port to the same destination? 

I agree with your comments that receiver selection is a powerful thing. It seems to me inevitable that this is something which will need effort - but IIUC this is out of scope for the Maitai work (it is concerned with knowing what the choices are...and not issues which touch on 'continuous presence/floor control')

Thanks, 

Peter

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Thu, 2 Dec 2010 09:33:37 -0600

Peter,

Unless of course it is a company meeting and the speaker is your CEO!

Something has to decide what screen tunes into what camera, no matter
what.

Question is what can be programmed automatic, and what is allowed to be
manual.
And if manual, what information is available to make such a decision.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Peter Musgrave
Sent: Thursday, December 02, 2010 7:53 AM
To: [email-masked]
Subject: [maitai] Roles of Sender and Receiver

Hi, 

I have been pondering possible ways of representing a room configuration
- but I think I need to back up and ask a question about basic interop
architecture. 

In the selection of content on a stream, who is the active decision
making entity, sender - or receiver - or both?

Consider a three display, 3 camera (3d3c) room A sending to a 1d1c room
B. 

Does A offer all three streams to B and let B pick? (This leads in the
direction of dynamic changes based on B, which is powerful but leads us
into continuous control...)
Does A determine B has only one display and select which video B should
get and send only one stream?

IHMO an approach which lets B have control over what it gets is more
powerful. e.g. If I have heard the pitch from the speaker 100 times, I
want to watch the customer and see if they are engaged, rather than
watch the active speaker blather on....

Peter

_______________________________________________
maitai mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/maitai

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Thu, 2 Dec 2010 08:10:46 -0800

The way I have been thinking about it, A would advertise what cameras it
has and ways it can present them to B. B would then be able to select
the camera and/or representations it wants.
As a simplified example, A could advertise the following:

I have 3 cameras, left, center, and right.
I can provide these as:
- 3 separate streams
- 1 active speaker switched stream
- 1 stream composed of the three

Of course there would be audio and many other things to consider, but
hopefully that gets the general idea across. This allows B to have as
much of as little control over what it receives as it likes, and it
should be able to change mid call.

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Mike Hammer (hmmr)
> Sent: Thursday, December 02, 2010 7:34 AM
> To: Peter Musgrave; [email-masked]
> Subject: Re: [maitai] Roles of Sender and Receiver
> 
> Peter,
>

---

From: "Duckworth, Mark" <[email-masked]>
Date: Thu, 2 Dec 2010 10:10:55 -0800

Charles,

That sounds consistent with the way I have been thinking about it too.  B might also want to ask for a particular stream at a particular time, as in Peter's example of wanting to see the customer even though the customer isn't speaking.

Mark

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Charles Eckel (eckelcu)
Sent: Thursday, December 02, 2010 11:11 AM
To: Mike Hammer (hmmr); Peter Musgrave; [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

Cheers,
Charles

> From: [email-masked] [mailto:[email-masked]] On
Behalf

---

From: "Duckworth, Mark" <[email-masked]>
Date: Thu, 2 Dec 2010 10:34:07 -0800

Peter,

H.281 controls video on an existing stream, not selecting a new stream from a new IP/port.

Mark

-----Original Message-----
From: Peter Musgrave [mailto:[email-masked]] 
Sent: Thursday, December 02, 2010 10:01 AM

Thanks, 

Peter

---

From: Peter Musgrave <[email-masked]>
Date: Thu, 2 Dec 2010 14:00:26 -0500

On 2010-12-02, at 11:10 AM, Charles Eckel (eckelcu) wrote:

>  have 3 cameras, left, center, and right.
> I can provide these as:
> - 3 separate streams

Ok, I see what you're after. 

In the case where the receiver elects to get three separate streams, then I have a refinement for my question. 

Who now decides how the streams from A map on to screens at B?

A could send streams targeted at specific screens at B (after examining B's description of it's screens)

-or-

B could send A instructions on where to send each stream (based on examining A's description of it's cameras)

-or-

We can find use-cases in which both techniques might be required. 

I am trying to decide how complete the information in a room description really needs to be. While I like the idea of a reasonably complete physical description (since it is very future proof) - I think it imposes a burden on each side which might not be warranted. The other extreme (just label cameras left, center, right) seems to obviously limited. Where is the middle ground?

Peter Musgrave

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Thu, 2 Dec 2010 13:47:44 -0600

Ummmm....

Part of the issue is that with multiple participants with multiple
inputs and outputs, you can't send all inputs from all sites to all
other sites.  You kill the network, so some judicious control of what is
sent when and to whom is needed.  That means that some inputs
(microphone or camera) are not transmitted at times.

So, do we allow legs to be asymmetric or not?

Interested in your view of the collective impact of these types of
control decisions.

Mike

-----Original Message-----
From: Peter Musgrave [mailto:[email-masked]] 
Sent: Thursday, December 02, 2010 2:00 PM
To: Charles Eckel (eckelcu)
Cc: Mike Hammer (hmmr); [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

In the case where the receiver elects to get three separate streams,
then I have a refinement for my question. 

A could send streams targeted at specific screens at B (after examining
B's description of it's screens)

-or-

B could send A instructions on where to send each stream (based on
examining A's description of it's cameras)

-or-

I am trying to decide how complete the information in a room description
really needs to be. While I like the idea of a reasonably complete
physical description (since it is very future proof) - I think it
imposes a burden on each side which might not be warranted. The other
extreme (just label cameras left, center, right) seems to obviously
limited. Where is the middle ground?

Peter Musgrave

---

From: "James M. Polk" <[email-masked]>
Date: Thu, 02 Dec 2010 13:57:08 -0600

Charles

In this scenario - wouldn't it be more appropriate to switch between 
the streams (i.e., only one is sent at a time) than actually 
transmitting each stream (where only one is consumed), thus 
absolutely reducing network load?

James

---

From: Peter Musgrave <[email-masked]>
Date: Thu, 2 Dec 2010 15:02:13 -0500

I think this is actually a different, very interesting issue. This speaks to control DURING the call and the need for that to be dynamic. I completely agree - but as understand the charter it's out of scope...although I think it is inevitable that we will need to look at what exists (BFCP, conference event etc.) and determine if it can be used or start to define something new.

This might be subverted by sending all and letting the receiver take them on and off hold quickly...clunky but avoid protocol work. 

My specific question relates to the initial setup where I have two 3 display, 3 camera systems from different vendors. Who decides which stream goes where? Sender or receiver?

Peter

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 2 Dec 2010 15:17:19 -0500

> Who now decides how the streams from A map on to screens at B?

The MCU, of course. So, all 3 could be sent to the MCU, and it could send a 3-tuple to some end points, switched to others, composited to
still others, and even combinations to some. 

The point is that there could be middleware that knows more about what is going on than any of the endpoints, which only have partial knowledge.

Regards
Marshall

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Thu, 2 Dec 2010 12:59:48 -0800

That is my thinking as well. You advertise what you have, which may or
may not include multiple representations of the same thing, then let the
other side decide what they want. The may want to receive a simplified
view of things, such a single composite image, or they may want
everything separate and then to do some composition or something else on
their end.

Cheers,
Charles

> From: Marshall Eubanks [mailto:[email-masked]]
> Sent: Thursday, December 02, 2010 12:17 PM
> To: Peter Musgrave
> Cc: Charles Eckel (eckelcu); [email-masked]
then I have a refinement for my
> question.
> The MCU, of course. So, all 3 could be sent to the MCU, and it could
send a 3-tuple to some end
> points, switched to others, composited to
> The point is that there could be middleware that knows more about what
is going on than any of the
> endpoints, which only have partial knowledge.
> 
> 
> Regards
> Marshall
> 
> 
> 
> >
> > A could send streams targeted at specific screens at B (after
examining B's description of it's
> screens)
examining A's description of it's
> cameras)
> > I am trying to decide how complete the information in a room
description really needs to be. While I
> like the idea of a reasonably complete physical description (since it
is very future proof) - I think
> it imposes a burden on each side which might not be warranted. The
other extreme (just label cameras
> left, center, right) seems to obviously limited. Where is the middle
ground?

---

From: Peter Musgrave <[email-masked]>
Date: Thu, 2 Dec 2010 16:23:15 -0500

Hi Marshal, 

That's not really the question I'm asking. In general either of the ends of these examples could be an MCU or an endpoint. I am interested in basic decisions when one side has three receivers and the other has three senders. Who decides on the mapping? The EPs could be rooms or MCUs. 

It may be that we end up with the old "master-slave" pattern - so a room might agree to let an MCU control both decisions of where to display incoming video and where to send outgoing video - and in a room to room call one side is elected to be master or they decide to split the responsibilities in some well defined way (e.g. each makes their own sender decisions). 

Also, I think that as "video router" and/or peer to peer systems are deployed, we do not want to "bake in" an MCU as a required architectural element in the specification. 

Cheers,

Peter Musgrave

---

From: "Duckworth, Mark" <[email-masked]>
Date: Thu, 2 Dec 2010 13:36:39 -0800

> Behalf Of Peter Musgrave
> Sent: Thursday, December 02, 2010 3:02 PM
> To: Mike Hammer (hmmr)
> Cc: [email-masked]
> I think this is actually a different, very interesting issue. This
> speaks to control DURING the call and the need for that to be dynamic.
> I completely agree - but as understand the charter it's out of
> scope...although I think it is inevitable that we will need to look at
> what exists (BFCP, conference event etc.) and determine if it can be
> used or start to define something new.

I agree, except I think this issue is within the scope of the charter.
The charter version 8 does say far end camera control is out of scope,
but it also says this is in scope: "As part of the receiver telling the
sender what it wants dynamically, explicit receiver notification to the
sender of the desired video stream and video pause will be considered."
This seems to leave a lot of room for dynamic handling of video stream
selection, within scope of the charter.

> This might be subverted by sending all and letting the receiver take
> them on and off hold quickly...clunky but avoid protocol work.
> 
> My specific question relates to the initial setup where I have two 3
> display, 3 camera systems from different vendors. Who decides which
> stream goes where? Sender or receiver?

I was thinking the sender would somehow indicate the spatial relationship
between the streams it sends.  Then a receiver can render it appropriately
on its display(s).

Mark

> examining
> description

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 2 Dec 2010 22:13:42 -0500

> It may be that we end up with the old "master-slave" pattern - so a room might agree to let an MCU control both decisions of where to display incoming video and where to send outgoing video - and i…

I think this is a little deeper than that. If the MCU (or whatever) is controlling this, then the source will think that the receiver is making the decisions, while the receiver will think that that the source is making the decisions, so that (to me) says that we have to accommodate both models. 

> Also, I think that as "video router" and/or peer to peer systems are deployed, we do not want to "bake in" an MCU as a required architectural element in the specification. 

Certainly not. 

Regards
Marshall

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 3 Dec 2010 09:06:38 +0100

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Duckworth, Mark
> Sent: 02 December 2010 21:37
> To: [email-masked]
[JRE] I see "far end camera control" as referring to the ability to control the angle and zoom of a particular camera - that would be out of scope. But controlling which of several cameras to receive during the course of a call should indeed be within the charter.

John

> > > other sites.  You kill the network, so some judicious 
> control of what
> > is

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 3 Dec 2010 08:15:11 -0500

> [JRE] I see "far end camera control" as referring to the ability to control the angle and zoom of a particular camera - that would be out of scope. But controlling which of several cameras to recei…

That is my take too.

Marshall

---

From: Peter Musgrave <[email-masked]>
Date: Fri, 3 Dec 2010 08:17:23 -0500

WFM

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Fri, 3 Dec 2010 14:47:14 -0800

Hi Peter,

I agree with Marshall's later comment that "we have to accommodate both
models". 

It seems to me that different systems may work differently - some may
use a subscription model, others can have it pre-set that either the
sender (including MCU), or receiver (as in current Cisco system) may
decide how to do the layout. In the primary spec that we are developing,
we just need to be able to give coherent information for whichever
implementation policy is being followed. So, we have to check that our
descriptive model works for different implementation methods.

The second aspect of the charter is to see what extensions  in our
protocols may be necessary to be able to carry out one or another
policy.  But for the spec that we are doing to describe the
relationships between multiple streams so that they can be reasonably
rendered, I think we need to have that description work for any of the
composition models (meaning sender composed or receiver composed).

What do you think?

Allyn

 ----------
From: Peter Musgrave <[email-masked]>
Date: Thu, Dec 2, 2010 at 2:02 PM
To: "Mike Hammer (hmmr)" <[email-masked]>
Cc: [email-masked]

I think this is actually a different, very interesting issue. This
speaks to control DURING the call and the need for that to be dynamic. I
completely agree - but as understand the charter it's out of
scope...although I think it is inevitable that we will need to look at
what exists (BFCP, conference event etc.) and determine if it can be
used or start to define something new.

Peter

----------

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 03 Dec 2010 20:20:29 -0500

> IHMO an approach which lets B have control over what it gets is more powerful. e.g. If I have heard the pitch from the speaker 100 times, I want to watch the customer and see if they are engaged, r…

Interesting question.

Maybe good to start considering this in purely sip terms without regard 
to telepresence specifics.

In that context I guess A might be sending an offer with three video 
m-lines, and would not initially know whether the recipient could deal 
with them all or not. (I'll ignore the audio to keep things simpler.) In 
a pure sip context it also may not be apparent what the roles are for 
the three media streams.

Then B would have to answer. If it can't handle three video m-lines, 
then it will have to choose one, though it may not have any basis to 
pick one over another. Perhaps it would just take the first.

So then we end up with a session having one video stream, with B having 
chosen *which* of the streams it was, and with A choosing *what* is 
transmitted on that stream.

And of course the challenge of maitai is to improve on that situation.

A minimalist improvement is for A to label the offered streams with the 
roles they play, so that B has a basis for choosing.

For maitai it will be necessary to do more than the above, so there is 
need for more description. But ideally it will be "backward compatible" 
to the extent that a dumb device will be able to make *some* sense of 
what it is offered and end up doing something as reasonable as it makes 
sense for a dumb device to do in that context.

	Thanks,
	Paul

---

From: Peter Musgrave <[email-masked]>
Date: Sat, 4 Dec 2010 11:16:50 -0500

Hi Allyn, 

I agree. I always start with bright-eyed optimism that the world can be simplified, but then the real-world intrudes. 

A agree that we will need modes which are in the pattern of client, server and peer (or client-server). Then we need modal specification of who can dictate or request what of whom....

Some of the semantics of BFCP might apply here. I need to go re-re-read that. (I'm not saying we'll use BFCP - just that is a model of the type of interaction we may need)

This also means that a representation format needs to fully encompass cameras, displays and audio sources and that an endpoint needs to be able to fully interpret a far ends capabilities in this regards. I was looking for simplifications where this could be fairly abstract or simplified (i.e. I  only need to tell the far end about my displays, I'll decide what do do with my cameras) but this seems to not be practical. 

Regards, 

Peter

On 2010-12-03, at 5:47 PM, Allyn Romanow (allyn) wrote:

> Hi Peter,
>  
> I agree with Marshall’s later comment that “we have to accommodate both models”.
> It seems to me that different systems may work differently – some may use a subscription model, others can have it pre-set that either the sender (including MCU), or receiver (as in current Cisco system) may decide how to do the layout. In the primary spec that we are developing, we just need to be able to give coherent information for whichever implementation policy is being followed. So, we have to check that our descriptive model works for different implementation methods.
>  
> The second aspect of the charter is to see what extensions  in our protocols may be necessary to be able to carry out one or another policy.  But for the spec that we are doing to describe the relationships between multiple streams so that they can be reasonably rendered, I think we need to have that description work for any of the composition models (meaning sender composed or receiver composed).
>  
> What do you think?
>  
> Allyn
> 
>  ----------

---

From: Peter Musgrave <[email-masked]>
Date: Sat, 4 Dec 2010 11:30:44 -0500

Hi Paul, 

I too gravitate towards concrete examples...so I am happy to go down the rabbit hole.

<rabbit_hole>
One of Charles points was that an endpoint might not be able to select a specific video feed, but rather say something like "I only receive one stream, please use voice activity to decide which stream to send me".  So the answering EP might not know which of three streams this should be....

It's also starting to seem like either end potentially could be the one who makes the decision. 

In the SIP context you present, it is therefore not clear to me that the answering endpoint is always the one making the decision about which media stream to get (as would be "normal" in SIP) and the offeror can't make any decisions until it knows about the receivers configuration. 

In fact in the first exchange we also need to provide some information about media asymmetry (e.g. I may have three screens, but 4 cameras or three screens and two cameras). so my video m lines do not necessarily imply anything my ability to send that many videos....

I am not sure how best to do capability exchange. I suspect the description we end up with will be of non-trivial size. Perhaps it is a new Content body type - and these capabilities are exchanged instead of SDP in the first exchange? It would work - but it creeps me out - since I expect it breaks when going through most middle boxes. The other tools in the toolbox which could apply are SUBSCRIBE, NOTIFY, PUBLISH etc. Perhaps there is a role for them. If OPTIONS went end-to-end it could be a candidate...

Ok - out of the rabbit hole!
</rabbit_hole>

Let me ask more questions: 

Does the capability exchange need to be part of dialog establishment? 
Could it be done as a precursor exchange?
Could an endpoint reasonably cache capabilities of EPs it has prior contact with? [This may be important if our data representation requires a certain amount of "interpretation"] ?

Peter

On 2010-12-03, at 8:20 PM, Paul Kyzivat wrote:

> Maybe good to start considering this in purely sip terms without regard to telepresence specifics.
> 
> In that context I guess A might be sending an offer with three video m-lines, and would not initially know whether the recipient could deal with them all or not. (I'll ignore the audio to keep things simpler.) In a pure sip context it also may not be apparent what the roles are for the three media streams.
> 
> Then B would have to answer. If it can't handle three video m-lines, then it will have to choose one, though it may not have any basis to pick one over another. Perhaps it would just take the first.
> 
> So then we end up with a session having one video stream, with B having chosen *which* of the streams it was, and with A choosing *what* is transmitted on that stream.
> 
> A minimalist improvement is for A to label the offered streams with the roles they play, so that B has a basis for choosing.
> 
> For maitai it will be necessary to do more than the above, so there is need for more description. But ideally it will be "backward compatible" to the extent that a dumb device will be able to make *some* sense of what it is offered and end up doing something as reasonable as it makes sense for a dumb device to do in that context.
> 
> 	Thanks,
> 	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 04 Dec 2010 13:11:17 -0500

> In the SIP context you present, it is therefore not clear to me that the answering endpoint is always the one making the decision about which media stream to get (as would be "normal" in SIP) and t…

This depends a bit on how much you assume about the UAS. In the worst 
case, its just a generic sip UA that supports one audio, and maybe one 
video stream. If offered more than one of each, it will probably either 
reject the call or else just accept the first of each and refuse the rest.

For a caller with more than that to offer, it would make sense for *it* 
to put the audio and video stream that are most important *first*.

In that case, the UAS is in some sense making the choice, but in a 
particularly dumb way. Its really the UAC that is making the decision 
about what a dumb UAS will get.

After that would could imagine a hierarchy of increasingly intelligent 
UASs that we might be calling. I'm not certain if that is worthwhile or 
not. For instance, maybe a UAS that is capable of understanding role 
tags for the offered streams and at least making an explicit choice of 
which one(s) to accept, rather than just accepting the first one.

After the first O/A exchange, the UAC will hopefully understand 
something about the capabilities of the UAS. Depending on the perceived 
stupidity of the UAS, the UAC might decide it knows best, and manipulate 
what it decides to send over the limited streams the UAS accepted.

If the UAS is discovered to be more capable, then a more complex 
negotiation might be followed.

> I am not sure how best to do capability exchange. I suspect the description we end up with will be of non-trivial size. Perhaps it is a new Content body type - and these capabilities are exchanged…

I don't know about that yet. As you say, there are a multitude of 
possibilities.

> Could it be done as a precursor exchange?

IMO it would be best to at least *start* the capability exchange as part 
of dialog establishment. You may not know *anything* about the UAS prior 
to the initial INVITE, so it should be possible to start there and 
eventually end up where you need to be.

I know Jon Peterson has been floating some ideas about something like 
you are talking about. But that is a much bigger subject. I don't think 
it makes much sense to do such a thing as a telepresence-specific mechanism.

One possibility is to use presence to discover such capabilities. But I 
think this should be optional.

> Could an endpoint reasonably cache capabilities of EPs it has prior contact with? [This may be important if our data representation requires a certain amount of "interpretation"] ?

Well, perhaps. But it then should not be done by AOR, since there may be 
multiple UAs behind an AOR. Again, presence might be the approach to 
doing that.

	Thanks,
	Paul

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Sat, 4 Dec 2010 21:22:25 +0100

I'm not sure how BFCP will help you with this.

BFCP works round a model of a centralised request server, which grants floors either automatically or under chair control, or as some combination of the two.

Or are you saying that this will only work with a centralised conference server, which also contains the telepresence server.

Keith

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Peter Musgrave
Sent: Saturday, December 04, 2010 4:17 PM
To: Allyn Romanow (allyn)
Cc: [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

Hi Allyn,

Regards,

Peter

Hi Peter,

 ----------
From: Peter Musgrave <[email-masked]<mailto:[email-masked]>>
Date: Thu, Dec 2, 2010 at 2:02 PM
To: "Mike Hammer (hmmr)" <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>

Peter

----------

---

From: Peter Musgrave <[email-masked]>
Date: Sat, 4 Dec 2010 17:07:58 -0500

Hi Keith, 

What I vaguely recall from reading BFCP was that the modes were client, server or client-server. That was the part I was attempting to draw a parallel with...

I agree that centralization is not model that can be applied to all the TP use cases. 

Thx

Peter

---

From: Peter Musgrave <[email-masked]>
Date: Sat, 4 Dec 2010 17:16:39 -0500

> This depends a bit on how much you assume about the UAS. In the worst case, its just a generic sip UA that supports one audio, and maybe one video stream. If offered more than one of each, it will probably either reject the call or else just accept the first of each and refuse the rest.
> 
> For a caller with more than that to offer, it would make sense for *it* to put the audio and video stream that are most important *first*.
> 
This highlights a trap in my thinking. I had assumed that the calling party "expected" the receiver to be a TP system. As you point out - it is not reasonable to assume this. 

A TP could be calling a simple SIP UA (perhaps without even video) all the way up immersive room....

Apologies if I am stating the obvious. 

Peter

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Mon, 6 Dec 2010 09:00:22 -0600

What one is capable of sending or receiving is one thing.

What roles any given sender may play is another, and could be dynamic
during the session.

May be best to keep those orthogonal.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Peter Musgrave
Sent: Saturday, December 04, 2010 11:31 AM
To: Paul Kyzivat (pkyzivat)
Cc: [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

Hi Paul, 

I too gravitate towards concrete examples...so I am happy to go down the
rabbit hole.

<rabbit_hole>
One of Charles points was that an endpoint might not be able to select a
specific video feed, but rather say something like "I only receive one
stream, please use voice activity to decide which stream to send me".
So the answering EP might not know which of three streams this should
be....

It's also starting to seem like either end potentially could be the one
who makes the decision. 

In the SIP context you present, it is therefore not clear to me that the
answering endpoint is always the one making the decision about which
media stream to get (as would be "normal" in SIP) and the offeror can't
make any decisions until it knows about the receivers configuration. 

In fact in the first exchange we also need to provide some information
about media asymmetry (e.g. I may have three screens, but 4 cameras or
three screens and two cameras). so my video m lines do not necessarily
imply anything my ability to send that many videos....

I am not sure how best to do capability exchange. I suspect the
description we end up with will be of non-trivial size. Perhaps it is a
new Content body type - and these capabilities are exchanged instead of
SDP in the first exchange? It would work - but it creeps me out - since
I expect it breaks when going through most middle boxes. The other tools
in the toolbox which could apply are SUBSCRIBE, NOTIFY, PUBLISH etc.
Perhaps there is a role for them. If OPTIONS went end-to-end it could be
a candidate...

Does the capability exchange need to be part of dialog establishment? 
Could it be done as a precursor exchange?
Could an endpoint reasonably cache capabilities of EPs it has prior
contact with? [This may be important if our data representation requires
a certain amount of "interpretation"] ?

Peter

On 2010-12-03, at 8:20 PM, Paul Kyzivat wrote:

>> I have been pondering possible ways of representing a room
configuration - but I think I need to back up and ask a question about
basic interop architecture.
>> Does A offer all three streams to B and let B pick? (This leads in
the direction of dynamic changes based on B, which is powerful but leads
us into continuous control...)
> 
> So then we end up with a session having one video stream, with B
having chosen *which* of the streams it was, and with A choosing *what*
is transmitted on that stream.
> A minimalist improvement is for A to label the offered streams with
the roles they play, so that B has a basis for choosing.

---

From: <[email-masked]>
Date: Tue, 7 Dec 2010 09:49:18 +0100

> 
> In fact in the first exchange we also need to provide some 
> information about media asymmetry (e.g. I may have three 
> screens, but 4 cameras or three screens and two cameras). so 
> my video m lines do not necessarily imply anything my ability 
> to send that many videos....
> 

Why can't we use one m= line per unidirectional stream and use
a=sendonly/a=recvonly to discrimate between screens and cameras?

---

From: Peter Musgrave <[email-masked]>
Date: Tue, 7 Dec 2010 06:06:56 -0500

I think this causes confusion with hold/resume. I can have a camera and display but choose not to receive your video temporarily as I e.g. use the display to look at something else. 

Hence I don't see it a suitable for describing the static endpoint physical configuration. 

In principle a new SDP attribute could be used (e.g. a=device:camera) but this will cause all kinds of backwards compatibility issues with calls to devices which do not understand this. I suspect some use of the SDP grouping framework might come into play here...

Peter

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Tue, 7 Dec 2010 09:01:05 -0600

Better to use a new attribute than overload an existing attribute.

But, this also assumes we are using SDP and SIP to provide dynamic
controls during call.
Have we crossed that bridge yet?

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Peter Musgrave
Sent: Tuesday, December 07, 2010 6:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

I think this causes confusion with hold/resume. I can have a camera and
display but choose not to receive your video temporarily as I e.g. use
the display to look at something else. 

Hence I don't see it a suitable for describing the static endpoint
physical configuration. 

In principle a new SDP attribute could be used (e.g. a=device:camera)
but this will cause all kinds of backwards compatibility issues with
calls to devices which do not understand this. I suspect some use of the
SDP grouping framework might come into play here...

Peter

---

From: Peter Musgrave <[email-masked]>
Date: Tue, 7 Dec 2010 10:09:53 -0500

Nope. 

I think there are several bridges. The first is the initial call setup and mapping streams. The second is adding/removing/making changes as the meeting proceeds and the need for specific stream content changes...

Peter

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Tue, 7 Dec 2010 09:23:53 -0600

Might be nice to have some sort of use cases, requirements, architecture
documents first before diving into the protocols.

Any wind is a good wind, if you don't know what direction to go.  :)

Mike

-----Original Message-----
From: Peter Musgrave [mailto:[email-masked]] 
Sent: Tuesday, December 07, 2010 10:10 AM
To: Mike Hammer (hmmr)
Cc: [email-masked]; [email-masked]
Subject: Re: [maitai] Roles of Sender and Receiver

Nope. 

I think there are several bridges. The first is the initial call setup
and mapping streams. The second is adding/removing/making changes as the
meeting proceeds and the need for specific stream content changes...

Peter

Behalf

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 07 Dec 2010 10:30:17 -0500

IMO the initial call setup and o/a should do something reasonable if one 
of the two parties is a telepresence device and the other is an 
"ordinary" device. In particular, it ought to allow the establishment of 
a session at the level the "ordinary" device can sustain. And that 
should work regardless of which is the caller and which is the offerer.

If it happens that both parties are telepresence devices, then the 
initial call setup should either sort all the media out appropriately, 
or else enable mechanisms to permit it to be done subsequently. That is 
probably the meat of the maitai work.

	Thanks,
	Paul

On 12/7/2010 10:09 AM, Peter Musgrave wrote:
>> On 2010-12-07, at 3:49 AM,<[email-masked]>  wrote:
>>
>>>>

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Tue, 7 Dec 2010 16:41:46 +0100

I dont think we have, and if we go that way, that implies to me that BFCP should have also gone the same way - which it did not.

Keith 

> [mailto:[email-masked]] On Behalf Of Mike Hammer (hmmr)
> Sent: Tuesday, December 07, 2010 3:01 PM
> To: Peter Musgrave; [email-masked]
> Cc: [email-masked]
> But, this also assumes we are using SDP and SIP to provide 
> dynamic controls during call.
> [mailto:[email-masked]] On Behalf Of Peter Musgrave
> I think this causes confusion with hold/resume. I can have a 
> camera and display but choose not to receive your video 
> temporarily as I e.g. use the display to look at something else. 
> 
> In principle a new SDP attribute could be used (e.g. 
> a=device:camera) but this will cause all kinds of backwards 
> compatibility issues with calls to devices which do not 
> understand this. I suspect some use of the SDP grouping 
> framework might come into play here...
> 
> Peter
> 
> 
> screens, but 
> >> 4 cameras or three screens and two cameras). so my video m 
> lines do 
> >> not necessarily imply anything my ability to send that many 
> >> videos....
> >> 
> >

---

From: <[email-masked]>
Date: Tue, 7 Dec 2010 16:58:34 +0100

I see. Unless the initiator has means to get information on the capabilities of the remote endpoint prior to initiate a session, it looks like we will have to use RFC5939 and include information that can be undertsood by an "ordinary" device as part of the "actual configuration" and telepresence specific information as part of the "potential configurations"...

> -----Message d'origine-----
> De : [email-masked] [mailto:[email-masked]] 
> De la part de Paul Kyzivat
> Envoyé : mardi 7 décembre 2010 16:30
> À : [email-masked]
> Objet : Re: [maitai] Roles of Sender and Receiver
> 
> IMO the initial call setup and o/a should do something 
> reasonable if one of the two parties is a telepresence device 
> and the other is an "ordinary" device. In particular, it 
> ought to allow the establishment of a session at the level 
> the "ordinary" device can sustain. And that should work 
> regardless of which is the caller and which is the offerer.
> 
> If it happens that both parties are telepresence devices, 
> then the initial call setup should either sort all the media 
> out appropriately, or else enable mechanisms to permit it to 
> be done subsequently. That is probably the meat of the maitai work.
> 
> 	Thanks,
> 	Paul
> 
> > I think there are several bridges. The first is the initial 
> call setup and mapping streams. The second is 
> adding/removing/making changes as the meeting proceeds and 
> the need for specific stream content changes...
> >
> > Peter
> >
> >> I think this causes confusion with hold/resume. I can have 
> a camera 
> >> and display but choose not to receive your video temporarily as I 
> >> e.g. use the display to look at something else.
> >>
> a=device:camera) 
> >> calls to devices which do not understand this. I suspect 
> some use of 
> >> the SDP grouping framework might come into play here...
> >>
> >> Peter
> >>
> >>
> >> On 2010-12-07, at 3:49 
> AM,<[email-masked]>  wrote:
> >>
> >>>>
> screens, 
> >>>> but 4 cameras or three screens and two cameras). so my video m 
> >>>> lines do not necessarily imply anything my ability to send that 
> >>>> many videos....
> >>>>
> >>>

---

From: Mary Barnes <[email-masked]>
Date: Tue, 7 Dec 2010 10:07:54 -0600

Mike,

There is a problem statement and a use case document:
http://tools.ietf.org/id/draft-romanow-dispatch-telepresence-prob-statement-01.txt
http://tools.ietf.org/id/draft-romanow-dispatch-telepresence-use-cases-01.txt

Feedback on the use cases could be quite helpful at this stage rather than
diving into the protocol, as you suggest.  As well, starting to tease out
requirements would be quite helpful input into the WG when it is formed.
 The problem statement document is covered more or less by the charter,
although with a bit more detail that might fit into an Overview of an
architecture/framework document.

Regards,
Mary.

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 07 Dec 2010 11:38:59 -0500

> I see. Unless the initiator has means to get information on the capabilities of the remote endpoint prior to initiate a session, it looks like we will have to use RFC5939 and include information th…

I don't know that its *necessary* to use 5939, but that might be a good 
way to accomplish the result. While that is new and presumably not much 
implemented, there will be new implementation for maitai in any case, so 
it might be acceptable to build on it here.

	Thanks,
	Paul