Subject: [Moq] Request for Agenda time

From: Suhas Nandakumar <[email-masked]>
Date: Mon, 7 Mar 2022 13:22:13 -0800

Hi,

We would like to request agenda time to discuss the aspects of moq charter
and problem scope.

We are seeing a huge convergence in use cases that combine aspects of
realtime media and streaming media. We believe that the moq work should
focus on considering the best things from these domains to support these
upcoming unified use cases.

To that effect, we have submitted the following 2 drafts that discuss
Unified architecture/protocol for supporting low latency and interactive
media use cases.

High Level Arch Draft:
https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-arch/

Protocol Draft:
https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-proto/

Thanks
Suhas

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 8 Mar 2022 11:43:46 +0000

Hi Suahs,

The intention of the BOF from us chairs is to focus on understanding what are 
reasonable set of use cases and what considerations that drive that selection. 
This to enable a next step after the BOF of actually discussing a charter for 
a WG to take on a protocol work for a solution that meets these use cases and 
relevant considerations. Resulting in a work scope that is actually practical 
and can be completed in reasonable time frame and be built on QUIC features 
that exists. Enabling for the future extension into more demanding usages and 
taking advantage of new QUIC features that help resolve any short comings of 
QUIC.

>From that perspective we have developed this current draft agenda are: 
<https://github.com/afrind/moqbof/blob/main/meeting-materials/agenda.md> 
https://github.com/afrind/moqbof/blob/main/meeting-materials/agenda.md

Having read your architecture draft: 
https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-arch/ I have the 
impression that you are considering a wider set of use cases than the other 
proponents for the BOF so far. It would good if you could further clarify what 
usages you have considered for your proposed architecture and protocol 
solution? It would also be good if you could summarize what other 
considerations that you see important to consider in a future protocol work. I 
did note that some considerations where included in the draft, however a 
summary might make it easier for people and ensure that they are not missing 
important aspects from your work.

In regards to your request for agenda time there is no MOQ charter to discuss 
and there will not be one for this BOF. On the problem scope I think we can 
consider to try to fit you in if you can complement the other proponents to 
explain what additional use case you think should be tackled and what impact 
that has on scope of the problem space the future protocol work needs to meet.

So if you can answer the above questions quickly it will help me and my 
co-chair make decisions on the agenda. I am also convinced that it will result 
in good discussion prior to the BOF session.

Cheers

Magnus Westerlund

MOQ BOF Chair

---

From: Suhas Nandakumar <[email-masked]>
Date: Tue, 8 Mar 2022 22:03:46 -0800

Hi Magnus,

Just want to be clear that what we were asking was time to discuss the

scope of the work (which is largely the charter questions), so just want

to be clear we are on the same page with our original request.

That said, we have proposed this sort of scope since the start

(as part of side meetings), so it seems slightly unfair to characterize

that this is not in alignment with consensus for something where we are

beginning to gather the range of options and figure out if there is

any consensus.

So let me jump in trying to explain some key uses that drive this.

We are interested in media over QUIC uses cases that blur

the transition between real time communications and

streaming. Examples

- Large scale streaming with low latency. A large city

watching a soccer game and people with streaming hear

the cheers across the city well before seeing a goal scored.

- Watching a soccer game live, but at same time using your mobile

phone to watch a feed of camera over the goal that is "real time"

- Large company wide meetings or conferences where lots of people are

watching, but during the Q & A any of them can switch to become an

active participant.

- e-sports streaming where commentators and players can react

real time to fans comments vs have a significant delay from when

the fan makes a comment to when the fan hears or sees the

commentator's reaction to it.

- e-sports streamed to many viewers where a fan can be switched to a

live participant in the stream.

- Efficient scale out and caching where CDN style cache nodes can

be pushed to the edge to achieve low latencies. The complexity of

clients is one of the reasons WebRTC does not work for some of the

use cases. The complexity of caches is difficult with deploying

LL-DASH for some use cases.

Aspects of these use cases have been brought up before but the argument

we are making is that there is more to these cases than

purely a streaming variant and/or slightly faster

HLS (which is already being done).

Every use case can be solved. It is very difficult to discuss which set of
use

cases should be solved without some examination of the possible design
space.

Clearly a simpler solution that covered many use cases would be beneficial.

Hope this helps

Cheers

Suhas

On Tue, Mar 8, 2022 at 3:43 AM Magnus Westerlund <
[email-masked]> wrote:

> Hi Suahs,
>
>
>
> The intention of the BOF from us chairs is to focus on understanding what
> are reasonable set of use cases and what considerations that drive that
> selection. This to enable a next step after the BOF of actually discussing
> a charter for a WG to take on a protocol work for a solution that meets
> these use cases and relevant considerations. Resulting in a work scope that
> is actually practical and can be completed in reasonable time frame and be
> built on QUIC features that exists. Enabling for the future extension into
> more demanding usages and taking advantage of new QUIC features that help
> resolve any short comings of QUIC.
>
>
>
> https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-arch/ I have
> the impression that you are considering a wider set of use cases than the
> other proponents for the BOF so far. It would good if you could further
> clarify what usages you have considered for your proposed architecture and
> protocol solution? It would also be good if you could summarize what other
> considerations that you see important to consider in a future protocol
> work. I did note that some considerations where included in the draft,
> however a summary might make it easier for people and ensure that they are
> not missing important aspects from your work.
>
>
>
> In regards to your request for agenda time there is no MOQ charter to
> discuss and there will not be one for this BOF. On the problem scope I
> think we can consider to try to fit you in if you can complement the other
> proponents to explain what additional use case you think should be tackled
> and what impact that has on scope of the problem space the future protocol
> work needs to meet.
>
>
>
> co-chair make decisions on the agenda. I am also convinced that it will
> result in good discussion prior to the BOF session.
>
>
>
> Cheers
>
>
>
> Magnus Westerlund
>
> MOQ BOF Chair
>

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 8 Mar 2022 23:59:20 -0800

Suhas --

The QuicR draft makes some points that deserve attention, particularly with
respect to caching and HTTP/3.  In previous BOFs, the goal has been
expressed to enable RTC systems to build on HTTP/3, but subsequent work has
not really addressed this need, and now your draft seems to refute some of
the original arguments, so the draft covers fundamental questions that are
well worth discussing, particularly since "Media over QUIC" can encompass
such a wide variety of architectures.

With respect to the draft I did have a few questions.

1. The dependency on containerization. As you may know, low latency
rendering of containerized media on the Web depends on usage of the MSE API
in a non-standard "low latency" mode.  Also, MSE API does not currently
support workers (MSEv2 will fix this). For these reasons, many users of the
MSE API are migrating to the WebCodecs API, which offers low
latency without "tricks", supports workers, and does not require
containerization.  Instead, frames are transported in non-containerized
form, saving CPU cycles in containerization/decontainerization.  Given the
advantages of WebCodecs, I'm expecting that the only use cases that will
retain support for containerization will be those requiring content
protection or endpoints that only support MSE but not WebCodecs.  Note that
there is interest in supporting content protection without containerization
(see:   https://github.com/w3c/webcodecs/issues/41).

2. Latency and topology impact.  The use cases you mention differ
substantially in their latency requirements and topologies.  For example,
in the company meeting scenario the goal is to save bandwidth on the
corporate WAN link which could be deluged by thousands of participants
receiving individual streams. CDN caching does not help with this; an eCDN
is needed. Also, in the company meeting scenario the latency requirement is
not as stringent as say, sporting events or gambling.  In practice, this
means that company meeting scenarios are willing to trade latency for
quality to a degree that ultra-low latency scenarios may not find
desirable.

3. Bandwidth adjustment. It wasn't clear to me from the draft how you were
proposing to deal with bandwidth adjustment. This is a fundamental problem
because client-side adaptation, while avoiding difficult problems such as
encoder rate control (in response to feedback) or scalability limitations
such as encoder state resets (in response to PLI/FIR), implies a
considerable cost in terms of latency or missed video frames.  Techniques
such as scalable video coding are not a panacea here since they are not
capable of supporting the wide range of devices and formats required.
Skipping over missing video frames (as proposed in WARP) may not be the
most desirable approach for a sporting event where the missing video could
represent the winning play.

On Tue, Mar 8, 2022 at 10:04 PM Suhas Nandakumar <[email-masked]>
wrote:

> --
> Moq mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/moq
>

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 9 Mar 2022 08:12:12 +0000

Hi Suhas,

Thanks for summarizing the context.

I will discuss your agenda request with my co-chair later today. There was no 
intention to be unfair, I realized that we should have made more reach out a 
few weeks ago for catching additional contributions to the BOF, sorry about 
that.

Cheers

Magnus

From: Suhas Nandakumar <[email-masked]>
Sent: den 9 mars 2022 07:04
To: Magnus Westerlund <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [Moq] Request for Agenda time

Hi Magnus,

any consensus.

streaming. Examples

active participant.

Every use case can be solved. It is very difficult to discuss which set of use

cases should be solved without some examination of the possible design space.

Hope this helps

Cheers

Suhas

On Tue, Mar 8, 2022 at 3:43 AM Magnus Westerlund 
<[email-masked] <mailto:[email-masked]> > 
wrote:

Hi Suahs,

>From that perspective we have developed this current draft agenda are: 
<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-a209b4dc7d6a2c44&q=1&e=eb7f478e-e1f7-4a2b-ac84-87972fec254a&u=https%3A%2F%2Fgithub.com%2Fafrind%2Fmoqbof%2Fblob%2Fmain%2Fmeeting-materials%2Fagenda.md> 
https://github.com/afrind/moqbof/blob/main/meeting-materials/agenda.md

Cheers

Magnus Westerlund

MOQ BOF Chair

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 9 Mar 2022 15:55:09 -0700

changed subject ... inline ...

On 3/9/2022 12:59 AM, Bernard Aboba wrote:
> Suhas --
> 
> The QuicR draft makes some points that deserve attention, particularly 
> with respect to caching and HTTP/3.  In previous BOFs, the goal has been 
> expressed to enable RTC systems to build on HTTP/3, but subsequent work 
> has not really addressed this need, and now your draft seems to refute 
> some of the original arguments, so the draft covers fundamental 
> questions that are well worth discussing, particularly since "Media over 
> QUIC" can encompass such a wide variety of architectures.
> 

Nermeen Ishmail really clarified for my why we were having a hard time 
with mismatch to to HTTP. Fundamentally HTTP was designed to have 
stateless servers which we are trying to use as media relays or even as 
SFUs in some cases. A pub/sub architecture is does not have a stateless 
server - the server keep the subscription state (and it can be refreshed 
frequently to make high availability easier ). HLS and DASH use things 
like the manifest to help have this state but move it out of the web 
server but that indirection comes with some performance issue too which 
is why you don't see the low latency version at 100 ms of latency.

That said, this could all easily be mapped on to HTTP/3 - the current 
draft stays clear of it that is a good mapping or not but right now not 
seeing the value vs complexity it adds. I plan to learn more about 
exactly what changes might be needed to H3 and quic to make that mapping 
work well.

> rendering of containerized media on the Web depends on usage of the MSE 
> API in a non-standard "low latency" mode.  Also, MSE API does not 
> currently support workers (MSEv2 will fix this). For these reasons, many 
> users of the MSE API are migrating to the WebCodecs API, which offers 
> low latency without "tricks", supports workers, and does not require 
> form, saving CPU cycles in containerization/decontainerization.  Given 
> the advantages of WebCodecs, I'm expecting that the only use cases that 
> will retain support for containerization will be those requiring content 
> protection or endpoints that only support MSE but not WebCodecs.  Note 
> that there is interest in supporting content protection without 
> containerization (see: https://github.com/w3c/webcodecs/issues/41 
> <https://github.com/w3c/webcodecs/issues/41>).

I can imagine this work going either way - one way or another, you need 
some meta data about the video to know what to do with it. The simple 
observation here is that doing it multiple ways ( say SDP for 
interactive, and MP4 for non interactive ) seems sort of nuts as you 
have stuff that moves back and forth. When you back up and ask is there 
something simple that could cover both, the answer is clearly yes.

> substantially in their latency requirements and topologies.  For 
> example, in the company meeting scenario the goal is to save bandwidth 
> on the corporate WAN link which could be deluged by thousands of 
> participants receiving individual streams. CDN caching does not help 
> with this; an eCDN is needed. Also, in the company meeting scenario the 
> latency requirement is not as stringent as say, sporting events or 
> gambling.  In practice, this means that company meeting scenarios are 
> willing to trade latency for quality to a degree that ultra-low latency 
> scenarios may not find desirable.

I certainly consider enterprise CDN as a subset of CDN. Perhaps take CDN 
in the draft as the conceptual architecture of some of the traditional 
CDNs. I think the CDN market is going to change a bunch over the next 
couple of years. We are already seeing a push of cache nodes out into 
the 5G networks, into the enterprise networks. I think we will see it 
move into WiFi access points. Apple and others already use one device in 
your home as a local cache for OS updates to other devices.

This broad CDN space is changing and the protocols we use to access will 
change. All the major CDN vendors have limited system to provide compute 
as well as storage and I am sure that will evolve at the same time.

> 
> 3. Bandwidth adjustment. It wasn't clear to me from the draft how you 
> were proposing to deal with bandwidth adjustment. This is a fundamental 
> problem because client-side adaptation, while avoiding difficult 
> problems such as encoder rate control (in response to feedback) or 
> scalability limitations such as encoder state resets (in response to 
> PLI/FIR), implies a considerable cost in terms of latency or missed 
> video frames.  Techniques such as scalable video coding are not a 
> panacea here since they are not capable of supporting the wide range of 
> devices and formats required. Skipping over missing video frames (as 
> proposed in WARP) may not be the most desirable approach for a sporting 
> event where the missing video could represent the winning play.
> 

Drafts are lacking on details but I think the key design tools to use 
from the draft are:

1. Like Dash/HLS have media encoded in different format (be that 
simulcast or scalable media), the client can rapidly change what media 
encoding it is subscribed to change the bandwidth

2. When subscribing to media, there is a quick way to get all the frames 
in a "group" so you get the I Frame even if you are past the point in 
time when the I Frame was sent

3. The transport does the congestion control but sends up to the quicr 
and application layer hints about available bandwidth so that the 
application can do client side rate control.

4. Objects have a best before or expires or something to tell them not 
to keep getting re-transmitted after they are not needed

5. Objects have a priority that gets used by the sender (not the 
network) to order sequencing of packets. This can be used to try for 
higher res video while not messing up the low res video. Or layers in a 
scalable codec.

Overall the design for this is largely the approach from HLS or DASH but 
moving to work for real time.