Subject: Re: [Masque] A proposal for CONNECT-UDP

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 May 2020 18:38:09 +1000

Hey David,

I like this, thanks. AIUI it's probably 80%+ of what's needed to meet the charter, if IP isn't in scope.

A few comments/questions:

* You say 'The CONNECT-UDP method is defined for all versions of HTTP.' Does that imply that h1 and h2 implementations are expected to fall back to using TCP as transport, and if so, is how that gets serialised on the wire a TBD?

* If the intent is to be able to fall back gracefully to using TCP, it might be good to look at making this just a new header negotiation on CONNECT, rather than a new method. That might get you backwards compatibility not only with existing proxies, but also transit through firewalls (which often block unrecognised methods).

* If, on the other hand, you want to have a strong signal when the end-to-end connection doesn't know about the mechanism (conveyed by 501 Not Implemented) and/or when a hop doesn't want to do it (conveyed by 405 Method Not Allowed), using a new method might be a good answer. 

* Just to get the bike sheds warmed up (it's getting to winter here), did you consider CONNECT-DATAGRAM or maybe CONNECT-DG? Seems like it might be a better level of abstraction to advertise.

Cheers,

> On 17 Apr 2020, at 1:01 pm, David Schinazi <[email-masked]> wrote:
> 
> Hi MASQUE enthusiasts,
> 
> The initial proposal for MASQUE was formulated as a
> framework. draft-schinazi-masque-protocol-01 goes into
> the details, but the general idea was to have a unified
> MASQUE negotiation step that would allow bootstrapping
> other MASQUE applications (or services). Martin Thomson
> pointed out that it may be simpler to build this if each
> application/service handled its own negotiation, and there
> seemed to be support for that idea. The current MASQUE
> proposed charter (see other threads on this list) adopted
> this methodology, discussing how to build these specific
> applications individually, starting with UDP proxying.
> To help folks reason about what that could look like, I
> wrote up a short draft that defines the HTTP CONNECT-UDP
> Method: draft-schinazi-masque-connect-udp
> I hope this helps, please let me know what you think.
> 
> Thanks,
> David Schinazi
> -- 
> Masque mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/masque

--
Mark Nottingham   https://www.mnot.net/

---

From: Mike Bishop <[email-masked]>
Date: Mon, 4 May 2020 21:11:58 +0000

The header-on-CONNECT spelling is interesting, particularly given the discussion in http-core that new methods MUST NOT use the authority-form, which CONNECT_* almost certainly would.  Of course, you still need to check that a server understands the header first, so it doesn't accept your CONNECT and then give you a TCP connection by mistake.

-----Original Message-----
From: Masque <[email-masked]> On Behalf Of Mark Nottingham
Sent: Friday, May 1, 2020 4:38 AM
To: David Schinazi <[email-masked]>
Cc: MASQUE <[email-masked]>
Subject: Re: [Masque] A proposal for CONNECT-UDP

Hey David,

Cheers,

> The initial proposal for MASQUE was formulated as a framework. 
> draft-schinazi-masque-protocol-01 goes into the details, but the 
> general idea was to have a unified MASQUE negotiation step that would 
> allow bootstrapping other MASQUE applications (or services). Martin 
> Thomson pointed out that it may be simpler to build this if each 
> application/service handled its own negotiation, and there seemed to 
> be support for that idea. The current MASQUE proposed charter (see 
> other threads on this list) adopted this methodology, discussing how 
> to build these specific applications individually, starting with UDP 
> proxying.
> To help folks reason about what that could look like, I wrote up a 
> short draft that defines the HTTP CONNECT-UDP
> Method: draft-schinazi-masque-connect-udp I hope this helps, please 
> let me know what you think.
> 
> Thanks,
> David Schinazi
> --
> Masque mailing list
> [email-masked]

---

From: Ben Schwartz <[email-masked]>
Date: Mon, 4 May 2020 17:34:43 -0400

I wonder if we could spell it as a new ":scheme" (e.g. "udp").  Then the
request would be malformed to "legacy" servers.

On Mon, May 4, 2020 at 5:12 PM Mike Bishop <[email-masked]> wrote:

> The header-on-CONNECT spelling is interesting, particularly given the
> discussion in http-core that new methods MUST NOT use the authority-form,
> which CONNECT_* almost certainly would.  Of course, you still need to check
> that a server understands the header first, so it doesn't accept your
> CONNECT and then give you a TCP connection by mistake.
>
> * You say 'The CONNECT-UDP method is defined for all versions of HTTP.'
> Does that imply that h1 and h2 implementations are expected to fall back to
> using TCP as transport, and if so, is how that gets serialised on the wire
> a TBD?
>
> * If the intent is to be able to fall back gracefully to using TCP, it
> might be good to look at making this just a new header negotiation on
> CONNECT, rather than a new method. That might get you backwards
> compatibility not only with existing proxies, but also transit through
> firewalls (which often block unrecognised methods).
>
> * If, on the other hand, you want to have a strong signal when the
> end-to-end connection doesn't know about the mechanism (conveyed by 501 Not
> Implemented) and/or when a hop doesn't want to do it (conveyed by 405
> Method Not Allowed), using a new method might be a good answer.
>
> * Just to get the bike sheds warmed up (it's getting to winter here), did
> you consider CONNECT-DATAGRAM or maybe CONNECT-DG? Seems like it might be a
> better level of abstraction to advertise.
>
> Cheers,
>
>

---

From: David Schinazi <[email-masked]>
Date: Mon, 4 May 2020 14:39:51 -0700

Thanks for looking at this folks!

Adding some thoughts on the points of discussion from above:

1) Fallback to TCP when the proxy doesn't support this new feature

I don't personally believe that falling back to TCP (as in what CONNECT
does) is useful.
In practice, if I'm trying to proxy a UDP-based protocol, if the proxy
doesn't support
UDP then it's not helpful for it to send my UDP packets in a TCP stream, as
odds are
high that the server won't understand them. So I think it's more useful to
have the proxy tell
the client that this feature isn't available, and the client can fallback
to a different protocol
over TCP. For example, if I'm trying to use CONNECT-UDP to speak HTTP/3, if
the proxy
doesn't support it I want to instead use CONNECT to speak HTTP/2. This was
the main
motivation for a separate HTTP method, to ensure the request fails if it
isn't supported.
But alternate options such as a custom :protocol header could work as well
if we can make
them work for HTTP/1.1 in a way that ensures legacy servers reject
the request.

2) Encoding the new CONNECT-UDP method over HTTP/2 or HTTP/1.1
If my network environment is setup to require everything to go through a
proxy,
then it's worthwhile to encode the UDP payloads over the request stream even
if it causes a performance cost. The current draft encodes the payloads
over the
request stream using a simple length-value encoding.

3) HTTP/1.1 specific text
I copy-pasted a chunk of the definition of CONNECT-UDP from the definition
of
CONNECT in HTTP/1.1. Happy to rephrase this.

4) Negotiating a feature across multiple HTTP hops
Question for the HTTP experts in the room: what's the best way to determine
whether every proxy hop along a path supports a feature? The current draft
has something that looks like a hop-by-hop header, but I'm happy to replace
that with the most recommended mechanism. Is there a way to ensure that
proxies that do not understand a given header or method don't forward the
request?

Thanks,
David

---

From: Martin Duke <[email-masked]>
Date: Mon, 4 May 2020 17:17:42 -0700

Possibly a dumb question, but there are a few possible meanings of
‘fallback’ in this use case and I want to make sure we’re talking about the
same thing:

1) the MASQUE server does not support HTTP/3 and/or DATAGRAM frames. Are we
expecting MASQUE to soldier on in this case using HTTP/2 or datagram-less
H3 (functionally the same thing?)

2) the MASQUE server is not able to open UDP sockets on the other side,
perhaps because it doesn’t support CONNECTU

3) the end server only provides the desired application service (eg, HTTP)
over TCP, so CONNECTU needs to fall back to regular CONNECT.

What situations would the WG address?

On Mon, May 4, 2020 at 2:41 PM David Schinazi <[email-masked]>
wrote:

> UDP then it's not helpful for it to send my UDP packets in a TCP stream,
> as odds are
> over TCP. For example, if I'm trying to use CONNECT-UDP to speak HTTP/3,
> if the proxy

---

From: David Schinazi <[email-masked]>
Date: Mon, 4 May 2020 17:27:50 -0700

Responses inline given my personal understanding/opinions:

On Mon, May 4, 2020 at 5:17 PM Martin Duke <[email-masked]> wrote:

> 1) the MASQUE server does not support HTTP/3 and/or DATAGRAM frames. Are
> we expecting MASQUE to soldier on in this case using HTTP/2 or
> datagram-less H3 (functionally the same thing?)
>

Yes. I think there's value in having CONNECT-UDP work in HTTP/2. That
allows clients to use proxied UDP on networks that block UDP and/or QUIC.

> perhaps because it doesn’t support CONNECTU

No. If the proxy doesn't support CONNECT-UDP, I very much do not want it to
transparently convert my UDP payloads into something over TCP. What would
the end server do with this TCP flow? My QUIC end-servers don't support
QUIC over TCP.

> over TCP, so CONNECTU needs to fall back to regular CONNECT.

No. I don't think there's a need for CONNECT-UDP at all in this use-case.
If the client wishes to proxy to a TCP service, it just uses regular
CONNECT. No need for CONNECT-UDP+fallback.

>> high that the server won't understand them. So I think it's more useful
>> to have the proxy tell
>> doesn't support it I want to instead use CONNECT to speak HTTP/2. This
>> was the main
>> But alternate options such as a custom :protocol header could work as
>> well if we can make
>> I copy-pasted a chunk of the definition of CONNECT-UDP from the
>> definition of
>>> * Just to get the bike sheds warmed up (it's getting to winter here),
>>> did you consider CONNECT-DATAGRAM or maybe CONNECT-DG? Seems like it might
>>> be a better level of abstraction to advertise.
>>>
>>> Cheers,
>>>
>>>

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 05 May 2020 10:28:15 +1000

I'm interested in the case where I can't get UDP datagrams from a client to the MASQUE server.  Using TCP on that leg is most interesting to me.

On Tue, May 5, 2020, at 10:17, Martin Duke wrote:
> ‘fallback’ in this use case and I want to make sure we’re talking about 
> the same thing:
> 
> 1) the MASQUE server does not support HTTP/3 and/or DATAGRAM frames. 
> Are we expecting MASQUE to soldier on in this case using HTTP/2 or 
> 3) the end server only provides the desired application service (eg, 
> HTTP) over TCP, so CONNECTU needs to fall back to regular CONNECT.
> 
> On Mon, May 4, 2020 at 2:41 PM David Schinazi <[email-masked] 
> <mailto:[email-masked]>> wrote:
> > I don't personally believe that falling back to TCP (as in what CONNECT does) is useful..
> > In practice, if I'm trying to proxy a UDP-based protocol, if the proxy doesn't support
> > UDP then it's not helpful for it to send my UDP packets in a TCP stream, as odds are
> > high that the server won't understand them. So I think it's more useful to have the proxy tell
> > the client that this feature isn't available, and the client can fallback to a different protocol
> > over TCP. For example, if I'm trying to use CONNECT-UDP to speak HTTP/3, if the proxy
> > doesn't support it I want to instead use CONNECT to speak HTTP/2. This was the main
> > motivation for a separate HTTP method, to ensure the request fails if it isn't supported.
> > But alternate options such as a custom :protocol header could work as well if we can make
> > them work for HTTP/1.1 in a way that ensures legacy servers reject the request.
> > 
> > If my network environment is setup to require everything to go through a proxy,
> > if it causes a performance cost. The current draft encodes the payloads over the
> > I copy-pasted a chunk of the definition of CONNECT-UDP from the definition of

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 4 May 2020 17:31:05 -0700

On Mon, May 4, 2020 at 2:41 PM David Schinazi <[email-masked]>
wrote:

> high that the server won't understand them.
>

Well, we have prior art for precisely this scenario, which is to say TURN
over TCP. The issue is less about what the proxy supports (though it's
surely vastly easier to  add UDP to a H2 proxy than it is to convert to
QUIC) but rather about what the client->proxy environment looks like, and
we know that in a significant number of cases, UDP cannot get through
firewalls whereas TCP can.

-Ekr

---

From: "Christopher Wood" <[email-masked]>
Date: Mon, 04 May 2020 17:35:09 -0700

On Mon, May 4, 2020, at 5:31 PM, Eric Rescorla wrote:
> 
> 
> On Mon, May 4, 2020 at 2:41 PM David Schinazi <[email-masked]> wrote:
> > I don't personally believe that falling back to TCP (as in what CONNECT does) is useful.
> Well, we have prior art for precisely this scenario, which is to say 
> TURN over TCP.. The issue is less about what the proxy supports (though 
> it's surely vastly easier to add UDP to a H2 proxy than it is to 
> convert to QUIC) but rather about what the client->proxy environment 
> looks like, and we know that in a significant number of cases, UDP 
> cannot get through firewalls whereas TCP can.

Indeed! We can clarify this as the motivation for TCP fallback. 

Best,
Chris

---

From: Martin Duke <[email-masked]>
Date: Mon, 4 May 2020 17:36:59 -0700

Alright, so the Proposed charter is not correct in saying that MASQUE
happens “inside an HTTP/3 connection”, at least without qualification that
support in other HTTP versions is in scope.

On Mon, May 4, 2020 at 5:29 PM Martin Thomson <[email-masked]> wrote:

> I'm interested in the case where I can't get UDP datagrams from a client
> to the MASQUE server.  Using TCP on that leg is most interesting to me.
>
> > >>  * You say 'The CONNECT-UDP method is defined for all versions of
> HTTP.' Does that imply that h1 and h2 implementations are expected to fall
> back to using TCP as transport, and if so, is how that gets serialised on
> the wire a TBD?
> > >>
> > >>  * If the intent is to be able to fall back gracefully to using TCP,
> it might be good to look at making this just a new header negotiation on
> > >>  * Just to get the bike sheds warmed up (it's getting to winter
> here), did you consider CONNECT-DATAGRAM or maybe CONNECT-DG? Seems like it
> might be a better level of abstraction to advertise.
> > >>
> > >>  Cheers,
> > >>
> > >>