Subject: p6: maximum delta-seconds of 2147483648

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 12 Nov 2013 02:44:13 -0800

This question was raised as part of issue #512

  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/512

and discussed during the WG meeting in Vancouver.

> In Section 1.2.1:
> 
>> "If a cache receives a delta-seconds value larger than the largest positive integer it can represent, or if any of its subsequent calculations overflows, the cache MUST consider the value to be 2147483648 (2^31). A recipient parsing a delta-seconds value MUST use an arithmetic type of at least 31 bits of range, and a sender MUST NOT generate delta-seconds with a value greater than 2147483648."
>> 
> Shouldn't the largest value be 2147483647 (see MAX_INT)?

Much of the discussion was around why this particular number, and
whether it is a mistaken 32bit INT_MAX (off by one) or some form of NaN?

This number was apparently added to the description of the Age header
field by Jeff Mogul, around March 1996, as a substitute for infinity.
68 years is a pretty long time, so that's not surprising.  The odd
juxtaposition with the requirement on "31 bits of range" is what
makes people believe it is an error.

Mark says that the conclusion at the Vancouver meeting is that we
would explain this number and not change it in the protocol, out of
concern that changing it might cause an interop issue.  Hence, he
added a note of

   Note that this value (2147483648) might cause implementation issues on some
   platforms (in particular, those that represent MAX_INT as
   2<x:sup>31</x:sup>-1), and therefore ought to be handled carefullly.

IMO, the existing requirements are either not testable in the protocol
(the internal arithmetic type) or not implemented in practice.

I would like to replace them with

   A recipient parsing a delta-seconds value ought to use an
   arithmetic type of at least 32 bits of signed integer range.
   If a cache receives a delta-seconds value larger than the largest
   positive integer it can represent, or if any of its subsequent
   calculations overflows, the cache &MUST; consider the value to be
   either 2147483648 (2<x:sup>31</x:sup>) or the largest positive
   integer it can conveniently represent.

Note that this changes the "MUST use an arithmetic type of at least
31 bits of range" to an ought, and removes the "sender MUST NOT generate
delta-seconds with a value greater than 2147483648" requirement.
I did some more digging and have not been able to find a server that
actually implements the latter 2147483648 requirement.  The closest
I have come is this mention of IE9's treatment of received values:

http://blogs.msdn.com/b/ie/archive/2010/07/14/caching-improvements-in-internet-explorer-9.aspx

Apache httpd, nginx, and IIS allow any value to be sent that would
fit within a 64bit time_t.  I am not concerned about ever triggering
these requirements in real life, since 68 years is an absurdly long
period for a delta-seconds.

Would anyone be opposed to making these changes now in response to
the last call comment?

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Nov 2013 12:14:50 +0100

> Would anyone be opposed to making these changes now in response to

Sounds good to me.

Best regards, Julian

---

From: Eliot Lear <[email-masked]>
Date: Wed, 13 Nov 2013 08:59:22 -0800

Roy,

My only issue with your proposed change is that we don't know what we
don't know.  There are quite a lot of browsers in phones of all sorts
and maybe other things that we haven't really tested.  It's a bit of a
gamble to remove the MUST NOT, and I wonder if there really is any
reason to do so.  Can we simply declare 68 years to be BIGNUM for
purposes of caching?

Eliot

---

From: Roy T. Fielding <[email-masked]>
Date: Wed, 13 Nov 2013 10:43:34 -0800

> reason to do so.  Can we simply declare 68 years to be BIGNUM for

That's a good concern, but what does a requirement on senders do
for them?  The recipient parser is going to have to expect numbers
larger than 2^31, because they can't trust senders regardless, and
the remaining requirement (to treat overflow as either 2147483648
or INT_MAX) is intended to allow any device to handle this situation
properly.  That is, unlike the existing text, which requires overflow
be handled as 2147483648 even if the device is limited to 2^31-1.

BTW, the reason for 2^31 seems to be because the value is unsigned.
The concern was for overflow after arithmetic operations on Age, so
maybe I should point that out in the spec.

Yes, it is fair to say that 68 years is far beyond the realm for
delta-seconds, since it is relative to the time of receipt.
I don't think there is any confusion about 68 years being too long
for caching (most limit to one year or 10 years, regardless).

I am concerned that we have a MUST NOT in the spec that most people
simply ignore.  Some of these implementations probably assumed
that a time_t is always 32 bits, back in 1995-97; I haven't
found a sender of delta-seconds that checks for >= 2147483648.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Nov 2013 13:31:55 +0100

> found a sender of delta-seconds that checks for >= 2147483648.

Agreed.

This is the only current issue that holds up draft -25. If no further 
information comes up, I'll apply this change (unless Roy beats me to 
it), and submit -25 over the weekend.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Nov 2013 22:18:23 +0100

> it), and submit -25 over the weekend.

Proposed patch: 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/512/512.diff>

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 14 Nov 2013 22:26:21 +0100

Hi Julian,

> <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/512/512.diff>

 "A recipient parsing a delta-seconds value ought to use an arithmetic type
  of at least 32 bits of signed integer range."
                         ^^^^^^
You meant unsigned, right ? Because the value 2147483648 requires at least
32 bits unsigned or 33 bits signed, but 32 bits signed never satisfies this
requirement.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 15 Nov 2013 12:05:04 +0800

I’m OK with this patch, as long as Willy’s question is answered.

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Nov 2013 08:20:08 +0100

>    arithmetic type of at least 32 bits of signed integer range.

That is consistent with the previous requirement of "MUST use an 
arithmetic type of at least 31 bits of range", no?

> ...

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 15 Nov 2013 08:29:16 +0100

Hi Julian,

> arithmetic type of at least 31 bits of range", no?

Yes but it's still confusing. I read it as "declare a variable delta_seconds
of type signed int" and later "check if that variable equals or is greater
than 2147483648", which value cannot be represented with this type. So this
is confusing for an implementer. Also, I know many developers who don't know
the exact limits of the types they manipulate, especially in the +/-1 at the
end.

Also quite often, the integer parsing functions will return undefined values
for the inputs they can't parse, which is worse. Here I'm pretty sure we'll
see this :

   int delta_seconds;

   ...
   delta_seconds = atoi(age);

   if (delta_seconds >= 2147483648)
       delta_seconds = INT_MAX;

But this will never match and the atoi() will return crap, sometimes -1, or 0
or whatever without the developer even being aware of this.

Is there a reason for this type to be signed ?

Best regards,
Willy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 14 Nov 2013 23:36:37 -0800

On Nov 14, 2013, at 11:20 PM, Julian Reschke wrote:

> That is consistent with the previous requirement of "MUST use an arithmetic type of at least 31 bits of range", no?

Yes, but I feel for Willy's desire to get rid of the apparent
contradiction altogether ... but not that much.

I suggest we use the original 31 bits of range, remind folks that
it is unsigned, and explain that the 2147483648 can be used as a
canned numeric string on overflow even if they happen to be forced
to use a signed 32bit storage.  *sigh*

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Nov 2013 08:40:34 +0100

> Is there a reason for this type to be signed ?

a) It's been implicitly like that before.

b) Java, for instance, doesn't have unsigned integers.

Of course we can add even more prose or even pseudo-code, and also 
warnings about broken language libraries. But does this edge case 
*really* require this? Can we please

1) focus on whether this is an improvement over RFC 2616, addressing te 
LC comment, and/or

2) propose concrete changes?

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 15 Nov 2013 09:16:56 +0100

(replying to your two e-mails at once, I'm in a hurry and late)

> to use a signed 32bit storage.  *sigh*

Seems reasonable eventhough not looking nice.

> a) It's been implicitly like that before.

OK

> b) Java, for instance, doesn't have unsigned integers.

OK I didn't know.

> *really* require this?

In fact it's not that much of an edge case if we suggest that a
specific value has a special meaning, because that specific value
will be hard-coded in some senders, making it less of an edge case
and causing interop issues with the receivers which will read -1 or
0 there.

> Can we please

Probably that Roy's right. After all, if this is a specific value
then it makes sense to warn the developer about the fact he might
need to read the man or use strcmp(!).

Best regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 15 Nov 2013 16:22:55 +0800

Needless to say, I’m +1 on this.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Nov 2013 09:38:04 +0100

> need to read the man or use strcmp(!).

But this is true for parsing Content-Length as well, for instance. Do we 
need to state it there as well?

Best regards, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 15 Nov 2013 00:41:41 -0800

This is what I am about to commit (and am happy to let Julian make any
tweaks later if more discussion calls for it) ...

1.2.1.  Delta Seconds

   The delta-seconds rule specifies a non-negative integer, representing
   time in seconds.

     delta-seconds  = 1*DIGIT

   A recipient parsing a delta-seconds value and converting it to binary
   form ought to use an arithmetic type of at least 31 bits of non-
   negative integer range.  If a cache receives a delta-seconds value
   greater than the greatest integer it can represent, or if any of its
   subsequent calculations overflows, the cache MUST consider the value
   to be either 2147483648 (2^31) or the greatest positive integer it can
   conveniently represent.

      Note: The value 2147483648 is here for historical reasons,
      effectively represents infinity (over 68 years), and does not need
      to be stored in binary form; an implementation could produce it as
      a canned string if any overflow occurs, even if the calculations
      are performed with an arithmetic type incapable of directly
      representing that number.  What matters here is that an overflow
      be detected and not treated as a negative value in later
      calculations.

Cool?

....Roy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 15 Nov 2013 00:52:52 -0800

On Nov 15, 2013, at 12:38 AM, Julian Reschke wrote:
> But this is true for parsing Content-Length as well, for instance. Do we need to state it there as well?

Oh, hell no ... we are only doing this for backwards compatibility
with a silly magic number that should never have been discussed in
the first place.  Protocols should never tell people how to program.

The sole purpose of the original text was to avoid overflows being
treated as negative numbers during Age calculations, and a sane
description would have simply said that (as did Jeff's original draft).
A specific number was suggested by committee, over-specification
commenced, and then it was rearranged into a generic section for
delta-seconds. Murphy's law at work.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Nov 2013 13:49:39 +0100

>        be detected and not treated as a negative value in later

Yep (<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/2494>).

I believe we can now proceed to produce a new set of drafts -- I'll take 
care of that over the weekend.

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 15 Nov 2013 14:27:31 +0100

>      be detected and not treated as a negative value in later

I http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/1270.html
raised this issue back in March and the text above is fine with me.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 16 Nov 2013 08:45:02 +0100

>       be detected and not treated as a negative value in later

Yes, that perfectly fits the issue in my opinion.

Thanks Roy!

willy

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 16 Nov 2013 08:50:53 +0100

> delta-seconds. Murphy's law at work.

Which is why I don't like the fact that we don't suggest at least a
little bit of typing in the specs. I faced the issue several times
in the past. Content-length being to short as a 32-bit quantity for
people retrieving a DVD image.... Then my chunk parser that was
optimized for dealing with up to 7 hex digits once faced a case of
an application buffering and sending chunks larger than 256 MB!

It's probably too late for this standard to recommend anything, but
it would be nice in the future to recommend sizes to meet to ensure
interoperability over the internet.

Willy