Subject: [Wish] WHIP enhancements

From: Adam Roach <[email-masked]>
Date: Tue, 23 Feb 2021 20:09:00 -0600

Hey, folks! Now that we have an approved working group, it’s time to
get this party started. IETF 110 is just around the corner, and it would
be great to have things to talk about during our meeting slot.

I've gone over the current proposal and compared it against some of
the things that were brought up at the DISPATCH discussion, as well
as some of the things that I know our own network either needs or
will need in the near future. I'm going to lay this out by first
describing requirements, and then sketch out a proposal that
attempts to address these requirements.

-----------------------------------------------------------------

Requirements

One of the things that came up in DISPATCH, which is definitely
something that we need, is session management -- or at least session
termination. This means that we’ll need some way to correlate a
session termination request with a session origination request.
Beyond this, we've found the ability to have signaling-level
heartbeats to be important for service separation, as well as a
small handful of other operations, like ICE restarts and trickling
of ICE candidates.

Although not strictly necessary, there is a strong desire to be able
to perform *some* of these operations (e.g., session termination,
ICE restart) from the ingest node towards the broadcasting tool.

The current document leaves open whether an ingest connection
supports different ordinalities of audio and video streams; and, if
they do, how they might be distinguished. We should be clear about
what is expected in an SDP offer regarding the number of each, and
perhaps add some signaling so that the tool and ingest node can
negotiate or signal what combinations of media are supported.

Finally, for future extensibility and feature experimentation, there
is a desire to be able to expand the protocol with new operations
and new protocol fields in a backwards-compatible way.

Those are the requirements I'm aware of that we’ll want to consider
in updating the current protocol proposal. I'm going to sketch out
how we might achieve them below. One focus of this proposal is
ensuring that the implementation complexity on the client can be
kept as low as possible, so as to encourage the widest range of
broadcasting tools possible to adopt this solution.

-----------------------------------------------------------------

Implementation Sketch

First, we would update the POST/200 transaction so that it carries
more information than a simple offer/answer. This could be a MIME
multipart body, but that seems probably more trouble than
representing the SDP alongside the other information we want to
send.

So for example, one way of doing this is having the POST operation
contain a JSON object indicating which server-oriented operations it
supports (if any), along with the SDP offer; something like:

POST /ingest/username HTTP/1.1
Content-Type: application/json
...

{
   'offer': 'v=0\r\n0=- 20150 0 IN IP6...',
   'operations': ['update', 'end', 'ice-restart']
}

The POST response from the server then contains an answer, along
with a list of operations that it can accept, plus the endpoints
that the client would use to perform those operations. Something
like:

HTTP/1.1 200 OK
Content-Type: application/json
...

{
   'answer': 'v=0\r\n0=- 20150 0 IN IP6...',
   'heartbeat': 'https://example.com/stage/adam/ea48c65c/heartbeat',
   'update': 'https://example.com/stage/adam/ea48c65c/update',
   'end': 'https://example.com/stage/adam/ea48c65c/terminate',
   'trickle': 'https://mg.example.com/stage/adam/ea48c65c/ice-candidate',
   'ice-restart': 'https://mg.example.com/stage/adam/ea48c65c/ice-restart'
}

You'll notice that the endpoints are (or at least can be) customized
for the session by including a unique identifier (ea48c65c in the
example above). The broadcast client doesn't have to worry about
this; all it needs to do is use the endpoints that it gets in
response to a session creation post to enact the corresponding
operations.

We can have a discussion about which of these operations
implementations are expected to support, and which are supplemental,
but I think we definitely need at least heartbeat and session
termination (which I call "end" above).

For each of these operations, we would define how the client
interacts with them to enact the operation, but the general pattern
would be as above: a POST message sent to the corresponding
endpoint, containing a JSON body with operation parameters; and the
HTTP response will contain a JSON body with response parameters.

So that gets us an extensible list of operations, and allows the
ingest node to embed a session identifier into the endpoints as
necessary.

On top of this we can add something kind of clever to the
"heartbeat" operation that allows two-way communication without
requiring broadcast tools to do anything (relatively) complex like
implementing web sockets, by sending such operations in the response
to heartbeats.  One approach would be to limit egress nodes to
sending commands towards the client once per heartbeat period (e.g.,
if the clients sent a heartbeat once every 15 seconds, then the
server might have to wait as long as 15 seconds to send a command).
Another approach would be to have the server wait for the heartbeat
interval before responding, thereby leaving the heartbeat
transaction open for the entire time. If we decided to go with this
latter behavior, the server could send a command to the client at
any time by responding to the pending heartbeat operation.

Finally, I think the extensibility story here naturally falls out of
the techniques described above: new transactions can be defined by
adding them to the initial message exchange; and new parameter can
be added to existing operations by adding them to the JSON objects
that are part of those operation's POST/200 transactions.

-----------------------------------------------------------------

I'd like to close on the rough shape of these issues, and then get
a draft pulled together in time for the WISH meeting at IETF 110.
Input on how we might want to do things differently than I propose
above would be very welcome.  Thanks!

/a

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Wed, 24 Feb 2021 09:14:27 +0100

Hi Adam,

The need for a way to explicitly terminate the session was already by
several people, so I think we should definitely include it.

https://github.com/murillo128/webrtc-http-ingest-protocol/issues/2

However, I would be against optionalities, as it makes the testing and
interoperability much more difficult.

In that regard, I am against including the trickle option as the server
should be accessible to the publisher, and the only case where it could
make sense is if the WHIP server is on a public server but the media
servers are behind NATed network without port forwarding.

Also, I am not sure what would be the benefits of the ice restart
(especially if it is optional) compared to just restarting the whole
publishing session again, which is something that the service has to
implementa anyway.

On a last note, I would prefer to make the process less json-like and more
http-like, reusing HTTP headers and methods as much as possible.

My initial thoughts about how to implement it were that the POST request to
start the publication returned a 201 created response with a Location
header indicating the URI to the entity representing the publication. And
then allow the following methods on the request

- GET for retrieving information about the state of the publication (TBD).
- OPTIONS for keep alive
- UPDATE for updating the session (we would need to specify what would we
actually allow to be updated)
- DELETE for terminating the session

Best regards
Sergio

El mié, 24 feb 2021 a las 3:09, Adam Roach (<[email-masked]>) escribió:

> --
> Wish mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/wish
>

---

From: Lorenzo Miniero <[email-masked]>
Date: Wed, 24 Feb 2021 09:42:25 +0100

Hi all,

I was one of the proponents for the "terminate the session" option
Sergio mentioned, so I'm definitely up for that. I'm not against Adam's
proposal with respect to the JSON usage and the different URLs, as it
seems quite flexible to me: whether it's different addresses, or HTTP
headers, seems mostly semantics to me, so I'm fine either way (even
though a JSON syntax would definitely allow for easier extensibility in
the future). I'm fine with trickling as well, personally, and so see
some value in allowing ICE restarts to happen too.

What I'm not sure about are Adam's suggestion to use heartbeats for
two-way communications. Adam, what kind of commands are you envisaging
in the server-to-client direction here? Are you mostly thinking about
asynchronous events of some nature, or something else too? With respect
to the approach, I understand the desire to avoid long polls, but
having some commands/events returned only in response to heartbeats
here may lead to implementations that will poll often and aggressively
to be sure they have timely information, and "forcing" implementations
to only send heartbears at less frequent intervals may on the other end
lead to suboptimal user experience (depending, again, on the nature of
the commands/events the server would mean to send back).

In general, though, I think the proposal looks interesting, and I'm
looking forward to the session in a couple of weeks.

Lorenzo

On Wed, 24 Feb 2021 09:14:27 +0100
Sergio Garcia Murillo <[email-masked]> wrote:

> In that regard, I am against including the trickle option as the
> server should be accessible to the publisher, and the only case where
> it could make sense is if the WHIP server is on a public server but
> the media servers are behind NATed network without port forwarding.
> 
> On a last note, I would prefer to make the process less json-like and
> more http-like, reusing HTTP headers and methods as much as possible.
> 
> My initial thoughts about how to implement it were that the POST
> request to start the publication returned a 201 created response with
> a Location header indicating the URI to the entity representing the
> publication. And then allow the following methods on the request
> 
> - UPDATE for updating the session (we would need to specify what
> would we actually allow to be updated)
> > get this party started. IETF 110 is just around the corner, and it
> > would be great to have things to talk about during our meeting slot.
> >
> >    'trickle':
> > 'https://mg.example.com/stage/adam/ea48c65c/ice-candidate',
> > 'ice-restart':
> > 'https://mg.example.com/stage/adam/ea48c65c/ice-restart' }
> >

-- 
I'm getting older but, unlike whisky, I'm not getting any better
https://twitter.com/elminiero

---

From: Adam Roach <[email-masked]>
Date: Wed, 24 Feb 2021 17:25:15 -0600

Thanks for the quick review of my proposal! Comments inline below.

> interoperability much more difficult.

The scheme I'm looking for here is one of a very small set of core 
functions, plus optional fields that are ignored if an implementation 
doesn't understand them. To test interop, all that would be necessary is 
to check the proper functioning of the core functions (along with, I 
suppose, cursory checks that unknown fields are correctly ignored).

> publishing session again, which is something that the service has to 

I'm definitely okay leaving both trickle ice and ice restart functions 
off for now, as long as we have some kind of extensibility mechanism. 
What I'm trying to avoid is some kind of future situation where RT-CDNs 
are hosted behind application load balancers, and the processing nodes 
themselves lack v4 unique addresses (and are therefore natted). I don't 
see it being an issue right now, but depending on how quickly we can 
rely on 100% IPv6 deployment to end users, it may become an issue in the 
future. I don't want to be painted into a corner when/if this happens.

> more http-like, reusing HTTP headers and methods as much as possible.

I'm certainly not caught up on JSON as a syntax here, although it does 
make things considerably easier in most modern development environments. 
I'm open to other options.

Regarding our use of HTTP, I'd very much like to make sure we stay 
strictly within the bounds of BCP 56 (and, after discussions with the 
Area Directors, I'm fairly certain that publication of the 
post-last-call 
https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis-09 will be 
happening in fairly short order, so that's the reference we should be 
using). So re-using existing HTTP header fields to mean exactly what 
they're defined to mean is perfectly fine; but for any new information, 
I think we need to stay with the recommendations of BCP 56bis §4.7.

> publication.

I don't have any objection in principle to using HTTP header fields for 
pointing to a created resource, as long as we can make this work for all 
of our use cases.

> And then allow the following methods on the request
>
> - UPDATE for updating the session (we would need to specify what would 
> we actually allow to be updated)

Here's where things get a bit dicey. The observation in BCP 56bis §5.4.2 
that OPTIONS is not widely supported is highly relevant: as CDN ingest 
nodes are frequently behind application load balancers, requiring the 
support of OPTIONS can lead to significant operational headaches if the 
load balancers don't support OPTIONS natively. There's also a bit of a 
philosophical mismatch in that "OPTIONS" doesn't *really* mean "PING," 
and if the notion is that we're using HTTP methods as our semantic 
operation point, we really want things that are good semantic matches.

UPDATE is even more problematic, as HTTP doesn't have an UPDATE method, 
and the process of adding new methods to HTTP is politically difficult 
in the best circumstances (e.g., creating a method that has broad 
applicability to all resource types -- see the difficulty and insanely 
long timeline getting PATCH defined for an example).

So while I'm not married to any of the aspects of my proposal, I think 
we need a more explicit way of indicating what operations we need than 
HTTP methods; and, at the same time, I'm seriously concerned about the 
future and the viability of a protocol that has no extension points. Can 
we work to find common ground on those two points?

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 24 Feb 2021 17:39:21 -0600

Thanks for the review! Comments inline.

> some value in allowing ICE restarts to happen too.

One of the things that I didn't point out (although the example does 
show it) is that the scheme I describe lets you easily split up the 
machines responsible for handling different operations. For example, 
under certain decomposition models, it makes sense to perform ICE 
operations directly towards a media node, while other session operations 
go towards a controller node. An operation-to-URL mapping makes this 
trivial and transparent to the broadcast client.

> asynchronous events of some nature, or something else too?

Mostly session termination and ICE restarts. But I also want to make 
sure it's extensible.

> With respect

Given a free hand, I actually prefer the long poll option. I included 
the description of both to demonstrate that, even if we don't want a 
long poll mechanism, we could still implement *something* that gives the 
server a means of telling the client to do things (like end a session).

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 24 Feb 2021 17:44:18 -0600

On 2/24/21 17:25, Adam Roach wrote:
> I'm seriously concerned about the future and the viability of a 
> protocol that has no extension points. Can we work to find common 
> ground on those two points? 

I don't want to over-do the point, but after I sent this, I was reminded 
that the need for extensibility is something that Jonathan Lennox also 
highlighted during the WHIP discussion at DISPATCH 109, so I suspect 
it's something that's going to be conspicuously absent if we don't 
address it.

/a

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Thu, 25 Feb 2021 09:52:58 +0100

Hi Adam,

Comments inline

> suppose, cursory checks that unknown fields are correctly ignored).

My main concern about this is that most people will only implement the core
package and don't do the optional part.
So I would prefer to have as less optionality as possible, especially if
the optional functionality is something that may be required by some people
in order to make it work.

> future. I don't want to be painted into a corner when/if this happens.

That may be an interesting use case to support in order to be future proof
and would be easy to implement server side (just ignore the client updated
candidates). So my proposal would be either to not support it, or include
it on the core package.

> I think we need to stay with the recommendations of BCP 56bis §4.7.

Definitely, we should follow BCP 56.

> My bad, too much time working on SIP. Obviously defining any new http
methods or headers is something we should not do.

Best regards
Sergio

---

From: Adam Roach <[email-masked]>
Date: Thu, 25 Feb 2021 11:29:44 -0600

On 2/25/21 02:52, Sergio Garcia Murillo wrote:
> Hi Adam,
>
> Comments inline
>
> El jue, 25 feb 2021 a las 0:25, Adam Roach (<[email-masked] 
> <mailto:[email-masked]>>) escribió:
>
> My main concern about this is that most people will only implement the 
> core package and don't do the optional part.
> So I would prefer to have as less optionality as possible, especially 
> if the optional functionality is something that may be required by 
> some people in order to make it work.

To be clear, what I'm looking for here primarily is well-defined 
extension points. I'm happy leaving the actual optional extensions for 
later. The principle I'd follow here is that whatever is part of the 
core functionality should always be sufficient to connect to an ingress 
gateway and begin a broadcast.

Anything optional would necessarily be additional functionality that 
would possibly enhance the experience, but which would not be required 
for interop of core functionality. You mention SIP in your message, and 
I think there's a good example to use as a mental model: while a device 
that implements just basic, core SIP for voice calls can dial into a 
conference bridge and send/receive media, RFC 4575 defined a way that 
devices can learn and present additional information about the 
conference state. It's not necessary to implement; it just makes things 
nicer.

> That may be an interesting use case to support in order to be future 
> proof and would be easy to implement server side (just ignore the 
> client updated candidates). So my proposal would be either to not 
> support it, or include it on the core package.

I'm just as happy including it as part of core functionality. Given that 
it's part of the core JSEP model, I would expect any WebRTC stack to 
include it, so it's unlikely to make things particularly complicated for 
implementations.

[snip]

> methods or headers is something we should not do.

With that in mind, I think there are really only two ways that we can 
indicate operations:

 1. Have an in-band indication of the operation type in the body of the
    HTTP request, or
 2. Use unique endpoints for each request type.

I think either one works, although I think the second one is a bit 
cleaner, as it allows for separation of interests (e.g., routing 
requests to the node that can most efficiently handle them); and, as a 
nice side effect, it makes it clear which operations the server 
supports. If we choose option 1, then we'll need some additional 
mechanism for the client to learn which operations the server supports.

/a