Subject: extensible prioritization scheme review

From: Stefan Eissing <[email-masked]>
Date: Thu, 18 Nov 2021 12:20:45 +0100

Reading https://httpwg.org/http-extensions/less-h2/draft-ietf-httpbis-priority.html again, I have comments to the draft. Thank you for considering my thoughts.

Small correction:
7.1:
"When the PRIORITY_UPDATE frame applies to a request stream, clients SHOULD provide a Prioritized Stream ID that refers to a stream in the "open", "half-closed (local)", or "idle" state."

I believe this is "half-closed (remote)" instead?

The question below I find not answered, really.

14. "Why use an End-to-End Header Field?"

"The way a client processes a response is a property associated to that client generating that request. Not that of an intermediary. Therefore, it is an end-to-end property."
...
"Having the Priority header field defined as end-to-end is important for caching intermediaries."

I fail to see that. Given the complexities of HTTP Caching and "Vary" handling, this needs more detail.

Clients process responses differently. How can the processing properties of Google Chrome at a certain point in time versus the ones from Firefox at another point in time apply to a server resource? It also depends in which context the resource is loaded. If an image is embedded in one HTML page, its Priority: properties will vary from it being embedded in another HTML page.

That is why I have difficulties to apply `Priority:` to the server response meta data. If an intermediary (CDN) cache wants to persist response priorities, it is certainly free to do to so. But that seems outside HTTP.

Would it not be better if servers can send PRIORITY_UPDATE Frames to the client, if indeed server's changes to Priority should be made known to the client side of the connection?

My main issue is around the requirement that non-incremental responses should pre-empt any other. As in:

10.
"Therefore, non-incremental responses of the same urgency SHOULD be served in their entirety, one-by-one, based on the stream ID, which corresponds to the order in which clients make requests."
...
"It is RECOMMENDED that servers avoid such starvation where possible."

Slowl0ris comes again. If a server sticks to the "SHOULD" of serving responses to priority as described (or as I read it), it seems trivial for a malicious client to hog server resources:
- start several streams with small window sizes
- send a "Priority: u=1" request for a Websocket resource located at the server. That one will never terminate and block all responses that have already been scheduled.

I can understand clients saying "I need this CSS resource NOW! Stop wasting my bandwidth on other things!". But the solution of this spec to only send the response for the CSS resource assumes that it can make full use of the available bandwidth. This will not always be the case. A cached CSS resource might need revalidation and, while this is done, nothing else is sent.

Stating "It is RECOMMENDED that servers avoid such starvation where possible.", means that - in real deployments - server implementations *need* to ignore SHOULDs in this spec. This seems not a good approach to protocol design.

When HTTP makes a reboot of Priorities, we should be sure to address such scenarios or the outcome in the end will again be not-satisfactory to clients who need to shave off milliseconds on page paints.

Replacing H2's priority dependency tree with something simpler and defined for H2+H3 is very welcome. Thank you for the work put into this.

Kind Regards,
Stefan

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 19 Nov 2021 03:21:34 +0000

Hi Stefan,

Thanks for taking the time to review, replies in line:

On Thu, Nov 18, 2021 at 11:24 AM Stefan Eissing <
[email-masked]> wrote:

> Reading
> https://httpwg.org/http-extensions/less-h2/draft-ietf-httpbis-priority.html
> again, I have comments to the draft. Thank you for considering my thoughts.
>
>
> Small correction:
> 7.1:
> "When the PRIORITY_UPDATE frame applies to a request stream, clients
> SHOULD provide a Prioritized Stream ID that refers to a stream in the
> "open", "half-closed (local)", or "idle" state."
>

>From the client's perspective, I believe "half-closed (local)" is correct.
c.f. HTTP/2 [1]

"An endpoint sending an END_STREAM flag causes the stream state to become
"half-closed (local)"; an endpoint receiving an END_STREAM flag causes the
stream state to become "half-closed (remote)""

A client sending a PRIORITY_UPDATE for a half-closed (remote) isn't going
to see much value.

> "The way a client processes a response is a property associated to that
> client generating that request. Not that of an intermediary. Therefore, it
> is an end-to-end property."
> ...
> Clients process responses differently. How can the processing properties
> of Google Chrome at a certain point in time versus the ones from Firefox at
> another point in time apply to a server resource? It also depends in which
> context the resource is loaded. If an image is embedded in one HTML page,
> its Priority: properties will vary from it being embedded in another HTML
> page.
>
> That is why I have difficulties to apply `Priority:` to the server
> response meta data. If an intermediary (CDN) cache wants to persist
> response priorities, it is certainly free to do to so. But that seems
> outside HTTP.
>
> Would it not be better if servers can send PRIORITY_UPDATE Frames to the
> client, if indeed server's changes to Priority should be made known to the
> client side of the connection?
>

The intermediary in charge of the priority scheduling can use the priority
signal from the client, in addition to that from the server, in addition to
a bunch of other information available with the request (i.e. referrer,
fetch metadata etc). I am not a caching expert so maybe I'm overlooking the
concern you have, perhaps Kazuho can speak more to this point. However,
since the server's priority signals are never intended to be used by a
client endpoint I do not think allowing PRIORITY_UPDATE frames to be sent
by servers is a good design change to make.

> "Therefore, non-incremental responses of the same urgency SHOULD be served
> in their entirety, one-by-one, based on the stream ID, which corresponds to
> the order in which clients make requests."
> ...
> Slowl0ris comes again. If a server sticks to the "SHOULD" of serving
> responses to priority as described (or as I read it), it seems trivial for
> a malicious client to hog server resources:
> - send a "Priority: u=1" request for a Websocket resource located at the
> server. That one will never terminate and block all responses that have
> already been scheduled.
>
> I can understand clients saying "I need this CSS resource NOW! Stop
> wasting my bandwidth on other things!". But the solution of this spec to
> only send the response for the CSS resource assumes that it can make full
> use of the available bandwidth. This will not always be the case. A cached
> CSS resource might need revalidation and, while this is done, nothing else
> is sent.
>
> Stating "It is RECOMMENDED that servers avoid such starvation where
> possible.", means that - in real deployments - server implementations
> *need* to ignore SHOULDs in this spec. This seems not a good approach to
> protocol design.
>
> When HTTP makes a reboot of Priorities, we should be sure to address such
> scenarios or the outcome in the end will again be not-satisfactory to
> clients who need to shave off milliseconds on page paints.
>

This topic has been discussed a lot. I'd like to avoid retreading all of
that again (GitHub has a lot if you'd like to go spelunking). To keep it
succinct, the priority scheme is optimized for web page use case. The
recommendations about how servers schedule priority based on signals were
added to give clients some idea how their signals would be interpreted.
They are only hints, but knowing how the server is likely to react to those
hints is useful to clients so they can plan how to pick the values. To my
mind, server scheduling is ultimately going to need to be delegated to an
implementation detail; there's only so much we can say.

There are several scenarios where naively following the recommendations can
result in a bad situation for a server. Your websocket use case is a good
example. But that problem existed in the old RFC 7540 world too, just set
Exclusive on the never ending CONNECT stream. We can't spell out the entire
combination of ways that things can get hairy, the text warns of the
general problem manifest, and provides a recommendation to avoid it. I
don't see these things as conflicting as strongly as you suggest.

I don't follow your CSS example. Nowhere do we suggest that implementations
block other requests when there is nothing to send. If there is a high
priority request for a resource that is not available then a server that
doesn't utilise the available bandwidth for other resources is doing a bad
job. But that might depend on things like if a server uses a serialized /
HoL blocking interface to speak to the component where resources live.
Again I think that is an implementation point and we can't speak to it much.

I don't think we need to do anything more here but if you have a concrete
proposal for changes or additions then it would be good to review them.

Replacing H2's priority dependency tree with something simpler and defined
> for H2+H3 is very welcome. Thank you for the work put into this.
>

Thanks for taking the time to read and provide feedback!

> Stefan
Cheers,
Lucas

[1]
https://httpwg.org/http2-spec/draft-ietf-httpbis-http2bis.html#section-5.1-7.8.2

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 19 Nov 2021 10:19:20 +0100

Thanks Lucas, for answering my very late questions to the draft. (non-ironically.😌)

> Am 19.11.2021 um 04:21 schrieb Lucas Pardue <[email-masked]>:
> 
> Hi Stefan,
> 
> On Thu, Nov 18, 2021 at 11:24 AM Stefan Eissing <[email-masked]> wrote:
> From the client's perspective, I believe "half-closed (local)" is correct. c.f. HTTP/2 [1]

My mistake. I was in the server's pov in when reading this.

> The intermediary in charge of the priority scheduling can use the priority signal from the client, in addition to that from the server, in addition to a bunch of other information available with the request (i.e. referrer, fetch metadata etc). I am not a caching expert so maybe I'm overlooking the concern you have, perhaps Kazuho can speak more to this point. However, since the server's priority signals are never intended to be used by a client endpoint I do not think allowing PRIORITY_UPDATE frames to be sent by servers is a good design change to make.

That makes it more clear to me how response Priority headers are intended to be used. Thank you. I am not a caching expert either. 

Looking at the documentation of HAProxy, for an example, it says:

"The cache uses a hash of the host header and the URI as the key." <http://cbonte.github.io/haproxy-dconv/2.5/configuration.html#6>

Which I read as client sending 'Priority:' headers will break their caching. Which they may fix one day, but all deployed instances will deliver suboptimal performance, it seems. Also the use of 'Vary:' in responses seem to limit their caching.

Willy: anything you can add to this feature's impact?

> This topic has been discussed a lot. I'd like to avoid retreading all of that again (GitHub has a lot if you'd like to go spelunking). To keep it succinct, the priority scheme is optimized for web page use case. The recommendations about how servers schedule priority based on signals were added to give clients some idea how their signals would be interpreted. They are only hints, but knowing how the server is likely to react to those hints is useful to clients so they can plan how to pick the values. To my mind, server scheduling is ultimately going to need to be delegated to an implementation detail; there's only so much we can say.
> 
> There are several scenarios where naively following the recommendations can result in a bad situation for a server. Your websocket use case is a good example. But that problem existed in the old RFC 7540 world too, just set Exclusive on the never ending CONNECT stream. We can't spell out the entire combination of ways that things can get hairy, the text warns of the general problem manifest, and provides a recommendation to avoid it. I don't see these things as conflicting as strongly as you suggest.
> 
> I don't follow your CSS example. Nowhere do we suggest that implementations block other requests when there is nothing to send. If there is a high priority request for a resource that is not available then a server that doesn't utilise the available bandwidth for other resources is doing a bad job. But that might depend on things like if a server uses a serialized / HoL blocking interface to speak to the component where resources live. Again I think that is an implementation point and we can't speak to it much.
> 
> I don't think we need to do anything more here but if you have a concrete proposal for changes or additions then it would be good to review them.

I may have taken the SHOULD too restrictive. I'll open an issue to propose some addition.

> [1] https://httpwg.org/http2-spec/draft-ietf-httpbis-http2bis.html#section-5.1-7.8.2