Subject: [Wish] WHIP should also allow *media* to be sent over the HTTPS connection

From: Ross Finlayson <[email-masked]>
Date: Fri, 12 Mar 2021 17:40:44 -0800

One of the promises of WISH is that it might provide a way for relatively simple media sources (e.g. things like network cameras and real-time text (e.g., telemetry) feeds) to be more easily used within WebRTC applications.

During the WISH meeting earlier this week, a few speakers expressed concern about complexities of the proposed ‘injestion’ protocol (WHIP), in particular, the need to implement ICE.  While Cullen (for example) correctly pointed out that if the web server has a public IP address (as it usually will), then then relatively little ICE is required, this seemed to only partially alleviate the concerns.

Fortunately, for very simple media source endpoints, there is an easy solution already available to us: Have the media source negotiate COMEDIA transport (RFC 4145) with RTP/RTCP packets framed as defined in RFC 4571.  The media source (i.e., offerer) can ask for a "TCP/RTP/AVP”[*] stream, with "a=connection:existing”.  If the media server accepts this, then the media source can proceed to send its media over the existing HTTPS connection (that was used for negotiation), without requiring any ICE or STUN or TURN or DTLS, or any wrestling with NAT.  Nothing new needs to be standardized for this to work.

Needless to say, streaming RTP/RTCP over a reliable stream isn’t ideal, but it’s an easy solution, and perhaps appropriate for some simple, relatively low-bitrate media streams.

The WHIP document should, IMHO, explicitly note this possibility, by describing a 'WHIP-light’ subset of the protocol that very simple media sources might want to use (with caveats).

Of course, a media server should not be required to support WHIP-light (just as it should not be required to support any particular codec(s)), but having this possibility will likely cause WHIP to get more adoption.

	Ross.

[*] Yes, having “TCP” in the transport description might be a bit of an anachronism if the HTTPS stream ends up being sent over QUIC, but the meaning remains clear.

---

From: Sean DuBois <[email-masked]>
Date: Fri, 12 Mar 2021 18:54:53 -0800

Hi Ross,

RTP/RTCP over HTTP totally works. These are some of the issues I anticipate or
things that I really like about WebRTC...

# Needs to get into the browser
WebCodecs + WebTransport might solve this, but it hasn't been proven to
be feasible yet.

# WHIP needs to define more
WebRTC has some complicated features that I think WHIP will benefit from

* Simulcast
* Negotiation (Codec support, Multiplexing tracks)
* Congestion Control/Error Correction (When using QUIC Datagrams)

Simulcast is a big one. It is expensive to run transcoders, I would much
rather have publishers upload multiple quality levels.

# WHIPs narrow usage will mean not many implementations
I can capture my webcam/distribute a file today on these platforms or
using these languages. These aren't just bindings. I don't see WHIP ever
matching this because of its narrow use case.

* C++/Mobile (Android/iOS) [0]
* C/RTOS [1]
* Go [2]
* Rust [3]
* Typescript [4]
* Python [5]
* C# [6]

# ICE is good for the client
Network switching (Wifi -> 4G) is nice to have. You have 'IRL Streamers'
who will be using WHIP. I am not familiar with the latest in
TCP/HTTP, but this is a 'solved problem' with WebRTC at least.

[0] https://webrtc.googlesource.com/src/
[1] https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c
[2] https://github.com/pion/webrtc
[3] https://github.com/webrtc-rs/webrtc (Rust, Full implementation)
[4] https://github.com/shinyoshiaki/werift-webrtc
[5] https://github.com/aiortc/aiortc
[6] https://github.com/sipsorcery-org/sipsorcery

On Fri, Mar 12, 2021 at 05:40:44PM -0800, Ross Finlayson wrote:
> --
> Wish mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/wish

---

From: Alexandre GOUAILLARD <[email-masked]>
Date: Sat, 13 Mar 2021 10:41:50 +0700

Hi ross,

The Working group has been chartered to work on a pretty focussed problem:

"to specify a simple, extensible, HTTPS-based signaling protocol to
establish one-way WebRTC-based audiovisual sessions between broadcasting
tools and real-time media broadcast networks."

There are more concrete example of the use cases being addressed in the
full charter here:
https://datatracker.ietf.org/wg/wish/about/

Media management is entirely delegated to the RTCWEB specifications. In
particular, the use of ICE, and the use of DTLS-SRTP are mandatory. For
clarification, the use of JSEP, SDP, SDP O/A are also all mandatory.

One should see WHIP as the missing signalling transport protocol, a
companion to existing specification, in the very specific use case of one
to one, unidirectional sessions: duplex sessions, as needed for video
conferencing for example, are out of scope,

In your case, you could either use your solution at application level, or,
since it touches on media, bring your proposal to the RTCWEB working group
where it seems it could be more in scope. Eventually, only the chairs of
the RTCWEB group can rule on that last point.

   -
   Dr Alex, Chair.

---

From: Alexandre GOUAILLARD <[email-masked]>
Date: Sat, 13 Mar 2021 10:56:09 +0700

hi sean,

> WebRTC has some complicated features that I think WHIP will benefit from

As mentioned to Ross, WHIP is aimed at being a 'companion' signalling
protocol to RTCWEB, not a replacement. The goal is for people to get most
if not all the rtcweb features when using WHIP for the signalling.

> # WHIPs narrow usage will mean not many implementations

WHIP does not aim to be a companion for all webrtc use cases. While the
number of implementations of WHIP will be at least one order of magnitude
less than those of RTCWEB, the IETF decided the use case was important
enough to justify this work and to create a working group.

Here is the official main use case, with some words emphasized by myself:

""*At the same time, the use of WebRTC as a mechanism for large-scale media
broadcast is gaining popularity, with companies such as Millicast,
Caffeine, Janus/Meetecho, Evercast, Wowza, Liveswitch, Antmedia, and
Strivecast deploying WebRTC-based media distribution networks. Unlike more
vertically integrated uses of WebRTC, these networks would benefit
immensely from being able to re-use the several broadcasting tools that
have been developed over time (such as Wirecast, OBS, Stretchcast, NewBlue
Stream, XSplit, FFSplit, Lightstream, vMix, and a host of other
applications). *

*To date, these media distribution networks have employed their own
proprietary signaling protocols to establish the connection between
broadcasting tools and the network, generally requiring either bespoke
software or customized modifications to existing broadcasting tools.*

*With the large number of available tools and the growing number of
real-time media distribution networks, this ad-hoc approach to creating
custom protocols for establishing sessions clearly does not scale. The
real-time broadcasting ecosystem would benefit immensely from a single,
shared protocol to meet this goal.*"

It is likely that any one-way output from video conferencing feature to
connect to delivery networks (zoom RTMP out, SKype RTMP out, Jitsi RTMP
out, ...) would also benefit from this without a lot of modifications.

The role of the IETF is to provide specifications, and not implementations,
or unified test suites like e.g. W3C which has a specific mandate for at
least two browsers implementations, and a reasonable level of testing.

I hope this clarifies.

   -

---

From: Ross Finlayson <[email-masked]>
Date: Fri, 12 Mar 2021 19:58:50 -0800

> On Mar 12, 2021, at 7:41 PM, Alexandre GOUAILLARD <[email-masked]> wrote:
> 
> Media management is entirely delegated to the RTCWEB specifications. In particular, the use of ICE, and the use of DTLS-SRTP are mandatory. For clarification, the use of JSEP, SDP, SDP O/A are also all mandatory.

Alex,

I think you may be missing my point.  A WebRTC stream that happens to negotiate COMEDIA transport (with RFC 4571 framing of RTP/RTCP) is still, in my mind, a WebRTC stream.  It would be negotiated the same way as any other WebRTC stream (using JSEP, SDP, O/A).  But the stream itself wouldn’t use datagrams, and thus wouldn’t use ICE (because it wouldn’t need it).  That’s why I described this as being ‘WHIP-light’ - a subset of the WHIP protocol just for this simple class of streams.

Perhaps I have a more expansive view of WebRTC than everyone else, but I consider what I’m describing as being a WebRTC stream; albeit a special case.

	Ross.

---

From: Alexandre GOUAILLARD <[email-masked]>
Date: Sat, 13 Mar 2021 12:16:10 +0700

> I think you may be missing my point.  A WebRTC stream that happens to
> negotiate COMEDIA transport (with RFC 4571 framing of RTP/RTCP) is still,
> in my mind, a WebRTC stream.  It would be negotiated the same way as any
> other WebRTC stream (using JSEP, SDP, O/A).  But the stream itself wouldn’t
> use datagrams, and thus wouldn’t use ICE (because it wouldn’t need it).
> That’s why I described this as being ‘WHIP-light’ - a subset of the WHIP
> protocol just for this simple class of streams.
>
> Perhaps I have a more expansive view of WebRTC than everyone else, but I
> consider what I’m describing as being a WebRTC stream; albeit a special
> case.
>
>         Ross.
>
>
Ross,

- If you are speaking about Media (e.g. if your question is: "does this
constitute a legitimate webrtc stream"), then you need to bring the
discussion to the RTCWEB group.
- If your media is a legitimate WebRTC stream, then there is nothing
special to discuss about it in this working group either, since we will
handle all WebRTC streams which correspond to our sub use case (one way
only, from a WebRTC end-point to a server, ...).
- if you want to propose a use case or a feature not in scope for the WISH
working group, The right place is the DISPATCH mailing list.

I encourage you to read the following RFC and references therein:
section 2.2 of RFC 8825, which defines the different notions and entities,
to separate browsers from non browsers entities for example,
section 5    of RFC 8825, which mandates DTLS-SRTP,
section 7    of RFC 8825, which mandates, among others, ICE.
If your proposal does not support DTLS-SRTP, and ICE, then it is not a
WebRTC Endpoint (even if Non-Browser).

"
   WebRTC Browser (also called a "WebRTC User Agent" or "WebRTC UA"):

      Something that conforms to both the protocol specification and the
      JavaScript API cited above.

   WebRTC Non-Browser:  Something that conforms to the protocol
      specification but does not claim to implement the JavaScript API.
      This can also be called a "WebRTC device" or "WebRTC native
      application".

   WebRTC Endpoint:  Either a WebRTC browser or a WebRTC non-browser.
      It conforms to the protocol specification.

[...]

   All WebRTC browsers are WebRTC endpoints, so any requirement on a
   WebRTC endpoint also applies to a WebRTC browser.

*(NOTE for *@Sean DuBois <[email-masked]> *=> Note the distinction
between browser and non-browser from the start.*

*This section below is likely the most interesting part for you)*

   A WebRTC non-browser may be capable of hosting applications in a way
   that is similar to the way in which a browser can host JavaScript
   applications, typically by offering APIs in other languages.  For
   instance, it may be implemented as a library that offers a C++ API
   intended to be loaded into applications.  In this case, security
   considerations similar to those for JavaScript may be needed;
   however, since such APIs are not defined or referenced here, this
   document cannot give any specific rules for those interfaces."

"
      5 <https://tools.ietf.org/html/rfc8825#section-5>. Data Framing and
Securing

   The format for media transport is RTP [RFC3550
<https://tools.ietf.org/html/rfc3550>].  Implementation of
   the Secure Real-time Transport Protocol (SRTP) [RFC3711
<https://tools.ietf.org/html/rfc3711>] is *REQUIRED*
   for all implementations.

   The detailed considerations for usage of functions from RTP and SRTP
   are given in [RFC8834 <https://tools.ietf.org/html/rfc8834>].  The
security considerations for the WebRTC
   use case are provided in [RFC8826
<https://tools.ietf.org/html/rfc8826>], and the resulting security
   functions are described in [RFC8827 <https://tools.ietf.org/html/rfc8827>].

"

"
      7 <https://tools.ietf.org/html/rfc8825#section-7>. Connection
Management
      [...]

   WebRTC endpoints *MUST* implement those functions described in
   [RFC8829 <https://tools.ietf.org/html/rfc8829>] that relate to the
network layer (e.g., BUNDLE [RFC8843
<https://tools.ietf.org/html/rfc8843>],
   "rtcp-mux" [RFC5761 <https://tools.ietf.org/html/rfc5761>], and
Trickle ICE [RFC8838 <https://tools.ietf.org/html/rfc8838>]), but
these endpoints
   do not need to support the API functionality described in [RFC8829
<https://tools.ietf.org/html/rfc8829>].

"

   - Dr Alex, Chair.

---

From: Tim Panton <[email-masked]>
Date: Sun, 14 Mar 2021 13:57:01 +0000

Sent from my iPad

> On 13 Mar 2021, at 03:59, Ross Finlayson <[email-masked]> wrote:
> 
> 
> 

Would it run DTLS over comedia ?
If not how would the SRTP keys be generated ?
how would bandwidth estimation /congestion control work ?
Without ice how does continuing consent work?

Unless I am mistaken it will take quite a lot of work to engineer this into the webRTC stack on a browser.

I am also not convinced that the results are much different from piping webAudio into a websocket. 

T.

---

From: Ross Finlayson <[email-masked]>
Date: Sun, 14 Mar 2021 08:32:08 -0700

> On Mar 14, 2021, at 6:57 AM, Tim Panton <[email-masked]> wrote:
> 
> 

No

> If not how would the SRTP keys be generated ?

If the streaming is over a HTTPS connection, then you wouldn’t need SRTP.  I.e., you’d need to negotiate "TCP/RTP/AVP” rather than “RTP/SAVP”.  Again, this is perhaps a more expansive view of WebRTC than everyone else :-)

> how would bandwidth estimation /congestion control work ?

You’d use the HTTPS connection’s congestion control.

> Without ice how does continuing consent work?

The HTTPS connection staying up implies consent.

> Unless I am mistaken it will take quite a lot of work to engineer this into the webRTC stack on a browser.

Yes - but who says that the media source has to be a browser :-)

	Ross.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 14 Mar 2021 18:19:45 +0100

> into the webRTC stack on a browser.

I'm confused again, sorry if I'm being dense.  If the media source is
a browser, then it presumably already implements the full WebRTC stack?
Why is a reduced complexity protocol required in that case?

---

From: T H Panton <[email-masked]>
Date: Sun, 14 Mar 2021 18:36:40 +0000

> On 14 Mar 2021, at 15:32, Ross Finlayson <[email-masked]> wrote:
> 
> 
> 

Right, you have just eliminated all connection with webRTC.  
(I.e. it isn’t the rtcweb wire protocol and it isn’t from a browser).

Which makes your proposal something out of scope for this group as currently chartered. 
I’m not a process guru, but I think that means you have to go back to dispatch and charter a new group, 
or ask the AGs to re-charter this one. 

If you can find a way to do what you want an preserve webRTC/rtcweb compatibility with something like TURN over COMEDIA
then you might be able to slip the proposal in scope, but you’d need to show why that was better than TURN TLS.

Tim.

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Sun, 14 Mar 2021 20:36:55 +0100

> Why is a reduced complexity protocol required in that case?

browser may not be the primary target, but whip must be possible to be
implemented on them.

Best regards
Sergio