Subject: Fwd: New Version Notification for draft-nottingham-bikeshed-length-00.txt

From: Mark Nottingham <[email-masked]>
Date: Wed, 18 Mar 2020 12:29:24 +1100

FYI, as discussed in <https://github.com/httpwg/http-core/issues/276>.

Prettier version at <https://mnot.github.io/I-D/bikeshed-length/>.

> Begin forwarded message:
> 
> From: [email-masked]
> Subject: New Version Notification for draft-nottingham-bikeshed-length-00.txt
> Date: 18 March 2020 at 12:27:39 pm AEDT
> To: "Mark Nottingham" <[email-masked]>
> 
> 
> A new version of I-D, draft-nottingham-bikeshed-length-00.txt
> has been successfully submitted by Mark Nottingham and posted to the
> IETF repository.
> 
> Name:		draft-nottingham-bikeshed-length
> Revision:	00
> Title:		Advisory Content-Length for HTTP
> Document date:	2020-03-18
> Group:		Individual Submission
> Pages:		6
> URL:            https://www.ietf.org/internet-drafts/draft-nottingham-bikeshed-length-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-nottingham-bikeshed-length/
> Htmlized:       https://tools.ietf.org/html/draft-nottingham-bikeshed-length-00
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-nottingham-bikeshed-length
> 
> 
> Abstract:
>   The HTTP Content-Length header field is overloaded with (at least)
>   two duties: message delimitation in HTTP/1, and metadata about the
>   length of an incoming request body to the software handling it.
> 
>   This causes confusion, and sometimes problems.  This document
>   proposes a new header to untangle these semantics (at least
>   partially).
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
> 
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 18 Mar 2020 05:04:57 +0100

Hi Mark,

> Prettier version at <https://mnot.github.io/I-D/bikeshed-length/>.

Interesting. I've also thought many times "too bad we don't have a distinct
header field for this one". I too think that something advisory may be
very useful, but I'm also seeing some misuse. Typically some recipients
might decide to allocate a large enough memory area to receive the
advertised contents, and this will be used to DoS some servers by sending
very large lengths (as we used to see with C-L in the past).

Despite this I still think that if we word it appropriately (i.e. "do not
trust it") and if we make sure it is never correlated to the effective
length, we could easily address potential issues and make it useful.

I think that it could be worded as suggesting that even the sender does
not necessarily precisely know the exact value to place there. For example
if a client needs to decompress a file before sending it, it could
arbitrarily advertise 10 times its compressed size. Similarly a client
uploading a log file could advertise the current size plus a small
margin, knowing that the file might continue to grow during the transfer.

If presented like this it should be clear that the recipient must not trust
it for anything but a progress bar or 413 and that could be very useful.

BTW I wouldn't say "it is not allowed to occur in trailer sections" since
it's not expected to have any particular meaning there, and adding a new
rule will only make its processing more complicated, I'd simply say that
it's pointless to place it there and that it may be ignored.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 18 Mar 2020 07:00:36 +0000

--------
In message <[email-masked]>, Mark Nottingham writes:

I think the proposed header-name is perfectly fine, and see no reason to change it :-)

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 18 Mar 2020 19:17:42 +1100

I think that the expectation that this ultimately be accurate makes it no different than Content-Length.  Then you might be allowed to be different, with some appropriately loose definition that allows being off either a proportion of the overall or a fixed amount. Then it could be used to (for example) indicate the length of something that is being decompressed/compressed on the fly.  In a world with midders, the estimate could even be updated over time and converge on the actual length.

I agree with Willy about special rules for trailers.

I have opinions about the name, but I will refrain because the author clearly asked for that kind of restraint.

On Wed, Mar 18, 2020, at 12:29, Mark Nottingham wrote:
> > *From: [email-masked]
> > *Subject: **New Version Notification for draft-nottingham-bikeshed-length-00.txt*
> > *Date: *18 March 2020 at 12:27:39 pm AEDT
> > *To: *"Mark Nottingham" <[email-masked]>
> > 
> >

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 18 Mar 2020 11:16:07 -0700

> On Mar 18, 2020, at 12:00 AM, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

I would prefer Bs-Length for the 33% reduction in bytes on wire.

....Roy

---

From: Cory Benfield <[email-masked]>
Date: Thu, 19 Mar 2020 13:02:19 +0000

As with Willy and Martin, I think it would be good for this draft to
explicitly allow fuzziness in the field value. Otherwise, the draft
seems reasonable, and I'd implement it.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 19 Mar 2020 15:53:32 +0000

--------
In message <CAH_hAJGU=[email-masked]>

>seems reasonable, and I'd implement it.

We could make it explictly fuzzy, by definting it as kilobytes rounded down ?

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 19 Mar 2020 17:11:30 +0100

> We could make it explictly fuzzy, by definting it as kilobytes rounded down ?

I think it's an excellent idea, which even goes in the direction of
reducing the on-wire bytes. And given todays connections speeds, if we
use it only for progress bars it could even represent megabytes rounded
to the nearest. Less than 0.5 will usually not take more than a few
seconds and not deserve showing an accurate progress bar.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 19 Mar 2020 16:25:03 +0000

--------

>seconds and not deserve showing an accurate progress bar.

I thought about that, but I know of applications where traffic is sorted
in "small", "medium" and "huge", where "small" is significantly less than
a megabyte, so I think the units should in the kilobytes rather than
megabytes range.  Just to throw a number out there:  Smaller than 32k.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 19 Mar 2020 17:28:51 +0100

> megabytes range.  Just to throw a number out there:  Smaller than 32k.

I'm fine with this anyway.

Willy

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 20 Mar 2020 08:34:08 +1100

Do we want approximation or a cap on size?

An upper bound is good for some of the use cases we see.  An estimate for the other cases.

At the extreme, you might say X +/- Y with Z confidence.  With fields for each.

That sounds like overengineering, but it doesn't have to be complicated in common usage.  You could say that Y and Z are implied or have defaults to simplify the common case.  From that a recipient can build to whatever assumption they like.  Find a value of f(Z) for X + Y*f(Z) that suits you.  Just want to show a progress meter, then f(Z) = 0 is probably fine.  Want to allocate space with a low risk of overflow, then you might want something like f(Z) = 2/Z.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Mar 2020 08:58:55 +1100

"Bikeshed" was put in the title with an eye towards the field name choice, but I can see now that *this* WG wants to do more.

I think in -01 I'll:
  - Make it an integer number of kbytes
  - Explicitly call it a best-effort estimate (and that might be a seed for a name?)
  - Explicitly prohibit it from being used for message framing (just in case some proxy decides to use it for forward Content-Length or something)

Anything else?

> On 20 Mar 2020, at 3:25 am, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 19 Mar 2020 23:32:31 +0100

> An upper bound is good for some of the use cases we see.  An estimate for the

In case of a stream it's not necessarily known in advance, think for
example of a postscript processor sending to a printer and used to
display the %complete on the printer's LCD. However if sent as a
rough estimate it can still be extremely wrong and suddenly catch
up when reachin the end. You don't mind if your printer's LCD
"%complete" is wrong as long as it continues to make progress.

Willy

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 20 Mar 2020 10:55:10 +1100

>   - Make it an integer number of kbytes

Sure, the numbers will be smaller, but why have the rounding at all?

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 20 Mar 2020 00:02:09 +0000

--------
In message <[email-masked]>, "Martin Tho
>Sure, the numbers will be smaller, but why have the rounding at all?

To make sure it is not precise enough for framing use.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Mar 2020 11:22:59 +1100

This

> On 20 Mar 2020, at 11:02 am, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 20 Mar 2020 11:42:07 +1100

> > To make sure it is not precise enough for framing use.

And so when someone rounds up and expects that to be ample space, what then?  I'm not sure that I agree that this is a valid threat model.

Seems like over-rotating to me.  If implementations routinely encounter values that don't match the actual length, then they will learn not to depend on the two being the same.

---

From: Ben Schwartz <[email-masked]>
Date: Thu, 19 Mar 2020 22:09:06 -0400

Could the draft endorse greasing instead?

On Thu, Mar 19, 2020 at 8:46 PM Martin Thomson <[email-masked]> wrote:

> And so when someone rounds up and expects that to be ample space, what
> then?  I'm not sure that I agree that this is a valid threat model.
>
> Seems like over-rotating to me.  If implementations routinely encounter
> values that don't match the actual length, then they will learn not to
> depend on the two being the same.
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 20 Mar 2020 07:46:18 +0100

On Fri, Mar 20, 2020 at 11:42:07AM +1100, Martin Thomson wrote:
> And so when someone rounds up and expects that to be ample space, what then?
> I'm not sure that I agree that this is a valid threat model.
> 
> values that don't match the actual length, then they will learn not to depend
> on the two being the same.

The request was to have a size indicator to provide a progress bar, which
this seems to address fine. If the requirement is to have something more
accurate, then of course it's not enough, but at the same time comes again
the risk that it's misused (e.g. to size a malloc()).

Willy