Subject: [Tzdist-bis] comments on tzdist-tzif-08

From: Paul Eggert <[email-masked]>
Date: Fri, 29 Jun 2018 00:18:02 -0700

There seems to be a reasonable amount of repetition in the specs for the 
application/tzif and application/tzif-leap media types, e.g., the "Security 
considerations" sections are the same. Can't this be factored out, so that 
application/tzif-leap says "I'm like application/tzif, except for this one 
little thing"?

The wording about 32-bit data being a subsequence of the 64-bit data and footer 
is now repeated three times in the text. Is this repetition needed? If it is, 
the wording should be identical and should mention the footer, as in the 
attached proposed patch. If not, more surgery would be needed.

The new constraint that application/tzif must describe files with zero leapcnt 
is worded backwards. For example, it says that a non-leap TZif file MUST use 
application/tzif. But other MIME types like application/octect-stream are also 
allowed for random binary data and would work with a non-leap TZif file. The 
wording should instead say that if the MIME type is application/tzif, then the 
file MUST be a TZif file without leap seconds.

Also, it seems to me that application/tzif-leap should not require leap seconds, 
it should only recommend them; that is, one can imagine passing along 
non-leap-seconds files as part of a test that code supporting leap seconds can 
deal with files without leap seconds (maybe the ITU will retroactively abolish 
leap seconds! I wouldn't put it past them).

Also, the wording should clearly say whether it's talking about 32- vs 64-bit 
leapcnt; they need not agree, and 2038 they are currently planned to disagree 
soon after January 2038 (though we don't know the details yet).

Proposed patch attached, for all but the repetition issues.
--- draft-murchison-tzdist-tzif-08.xml	2018-06-28 21:24:43.309524691 -0700
+++ draft-murchison-tzdist-tzif-08-1.xml	2018-06-28 23:58:19.380768144 -0700
@@ -260,7 +260,7 @@
	
	      <t>The sequence of time changes defined by the 32-bit header and
	      data block SHOULD be a contiguous subsequence of the time
-	      changes defined by the 64-bit header, data block, and footer.
+	      changes defined by the 64-bit header and data block and by the footer.
	      When reading a version 2 or 3 file, implementations SHOULD
	      ignore the 32-bit header and data block except for the purpose
	      of skipping over them.</t>
@@ -504,14 +504,14 @@
	              strictly ascending order.
	              The number of records is specified by the
	              'leapcnt' field in the header.
-	              A file that has a 'leapcnt' of zero (0) MUST use the
-	              MIME media type
-	              <xref target="tzif">"application/tzif"</xref>
-	              if exchanged via a MIME-compatible protocol.
-	              A file that has a non-zero 'leapcnt' MUST use the
-	              MIME media type
-	              <xref target="tzif-leap">"application/tzif-leap"</xref>
-	              if exchanged via a MIME-compatible protocol.
+		      When exchanged via a MIME-compatible protocol,
+		      a file that uses the MIME media type
+		      <xref target="tzif">"application/tzif"</xref>
+		      MUST have a zero 'leapcnt' in the 32- and 64-bit data,
+		      and a file that uses the MIME media type
+		      <xref target="tzif-leap">"application/tzif-leap"</xref>
+		      SHOULD have a non-zero 'leapcnt' in the 64-bit data
+		      (or in the 32-bit data, in a version 1 file).
	              Each record has one of the following structures:
		      <list style='hanging'>
		        <t hangText="32-bit Data Block:">
@@ -856,7 +856,8 @@
	            Implementations SHOULD generate version 2 or 3 files.
	            The sequence of time changes defined by the 32-bit header
	            and data block SHOULD be a contiguous subsequence of the
-	            time changes defined by the 64-bit header and data block.
+		    time changes defined by the 64-bit header and data block
+		    and by the footer.
	          </t>
	          <t hangText="Published specification:">
	            This specification.
@@ -923,14 +924,16 @@
	            externally.
	          </t>
	          <t hangText="Interoperability considerations:">
-	            This media type indicates that leap second records MUST be
-	            included in the TZif data; 'leapcnt' in the header(s) MUST
+		    This media type indicates that leap second records SHOULD be
+		    included in the TZif data; 'leapcnt' in the 64-bit header
+		    (or in the 32-bit header, if a version 1 file) SHOULD
	            be non-zero.
	            <vspace/><vspace/>
	            Implementations SHOULD generate version 2 or 3 files.
	            The sequence of time changes defined by the 32-bit header
	            and data block SHOULD be a contiguous subsequence of the
-	            time changes defined by the 64-bit header and data block.
+		    time changes defined by the 64-bit header and data block
+		    and by the footer.
	          </t>
	          <t hangText="Published specification:">
	            This specification.

---

From: Ken Murchison <[email-masked]>
Date: Fri, 29 Jun 2018 09:46:42 -0400

On 06/29/2018 03:18 AM, Paul Eggert wrote:
> There seems to be a reasonable amount of repetition in the specs for 
> the application/tzif and application/tzif-leap media types, e.g., the 
> "Security considerations" sections are the same. Can't this be 
> factored out, so that application/tzif-leap says "I'm like 
> application/tzif, except for this one little thing"?

I don't know if one MIME type registration form can reference another. 
That's a question for Eliot and/or Ned.

I also was reluctant to repeat the entire declaration, which is one 
reason why I suggested a mandatory parameter on application/tzif rather 
than a separate MIME type.

> The wording about 32-bit data being a subsequence of the 64-bit data 
> and footer is now repeated three times in the text. Is this repetition 
> needed? If it is, the wording should be identical and should mention 
> the footer, as in the attached proposed patch. If not, more surgery 
> would be needed.

Do you have any suggestions on consolidation?

> The new constraint that application/tzif must describe files with zero 
> leapcnt is worded backwards. For example, it says that a non-leap TZif 
> file MUST use application/tzif. But other MIME types like 
> application/octect-stream are also allowed for random binary data and 
> would work with a non-leap TZif file. The wording should instead say 
> that if the MIME type is application/tzif, then the file MUST be a 
> TZif file without leap seconds.

Agreed.

> Also, it seems to me that application/tzif-leap should not require 
> leap seconds, it should only recommend them; that is, one can imagine 
> passing along non-leap-seconds files as part of a test that code 
> supporting leap seconds can deal with files without leap seconds 
> (maybe the ITU will retroactively abolish leap seconds! I wouldn't put 
> it past them).

I disagree on this point.  If I'm a TZDIST client wanting TZif data with 
leap seconds, the only way for me to request that is via 
Accept:application/tzif-leap.  If the server can choose to omit leap 
seconds when specifically requested, then what is the point of a 
separate MIME type?  If the server doesn't advertise 
application/tzif-leap as a supported type, then the client knows that it 
can't get TZif files with leap seconds.

> Also, the wording should clearly say whether it's talking about 32- vs 
> 64-bit leapcnt; they need not agree, and 2038 they are currently 
> planned to disagree soon after January 2038 (though we don't know the 
> details yet).
>

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Eliot Lear <[email-masked]>
Date: Fri, 29 Jun 2018 16:51:49 +0200

> That's a question for Eliot and/or Ned.

We're officious, but we're not THAT officious.  Use "Same as ..."

---

From: Ken Murchison <[email-masked]>
Date: Fri, 29 Jun 2018 11:00:05 -0400

> We're officious, but we're not THAT officious.  Use "Same as ..."

For security and interop, etc?

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Eliot Lear <[email-masked]>
Date: Fri, 29 Jun 2018 17:03:21 +0200

> For security and interop, etc?

As long as they're the same, why not?  If IANA complains, cut and paste
is easy.

Eliot

---

From: Ken Murchison <[email-masked]>
Date: Fri, 29 Jun 2018 12:22:27 -0400

On 06/29/2018 09:46 AM, Ken Murchison wrote:
>> and footer is now repeated three times in the text. Is this 
>> repetition needed? If it is, the wording should be identical and 
>> should mention the footer, as in the attached proposed patch. If not, 
>> more surgery would be needed.
>

Actually, I wonder if the "contiguous" text is even needed in the MIME 
declarations, as its already discussed in the text.

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Eliot Lear <[email-masked]>
Date: Fri, 29 Jun 2018 20:52:13 +0200

> declarations, as its already discussed in the text.

A reference is fine.

---

From: Paul Eggert <[email-masked]>
Date: Fri, 29 Jun 2018 13:43:35 -0700

> Do you have any suggestions on consolidation?

I gave it a shot in the attached patch.

> I disagree on this point.  If I'm a TZDIST client wanting TZif data 
> with leap seconds, the only way for me to request that is via 
> separate MIME type?

OK, fair enough. The attached patch takes the approach you suggest.

While drafting it, I noticed several more issues and the patch has 
proposed fixes for them too. Here are the issues:

* A missing ":" after "Unix Leap Time".

* The spec nowhere says that the TZ string uses ASCII. Although POSIX 
allows TZ strings to be EBCDIC (or any other encoding that has the ASCII 
characters), in practice as far as I know they are always ASCII in TZif 
files, and the spec should address this issue.

* Similarly, the spec nowhere says that time zone abbreviations must be 
in ASCII when in the character array. Here, the reference implementation 
allows any non-NUL bytes. However, it'd be bad practice to depart from 
the POSIX rules for time zone abbreviations as a lot of readers assume 
ASCII, so the patch uses a SHOULD for this, and also replaces 
"character" with "octet" to allow (though recommend against) non-ASCII 
characters (presumably UTF-8?).

* POSIX requires time zone abbreviations to be at least 3 and at most 6 
characters; the spec should mention this.

* POSIX requires that time zone abbreviations must contain only ASCII 
alphanumerics, or '-', or '+'. The spec should mention this too.

* The interoperability sections seem to just repeat the SHOULDs of the 
main text. Better would be an interoperability section that talks about 
practical problems, without any MUSTs or SHOULDs (which should be part 
of the main spec), and the MIME interoperability paragraphs can mostly 
just refer to this new section. A draft of this is in the patch. (Whew, 
that took more time to write than I thought it would! There are a lot of 
interoperability issues. I've probably forgotten some. On the other 
hand, maybe I've gone overboard in describing so many glitches.)

This patch is complicated enough that I'm repeating it in two ways in 
attachments. The first attachment is a patch relative to my previous 
proposal. The second is a patch relative to tzdist-tzif-08.
--- draft-murchison-tzdist-tzif-08-1.xml	2018-06-28 23:58:19.380768144 -0700
+++ draft-murchison-tzdist-tzif-08-2.xml	2018-06-29 13:42:56.906520753 -0700
@@ -189,7 +189,7 @@
	          leap seconds.  As an extension to POSIX, negative values
	          represent times before the POSIX Epoch, using UT.
	        </t>
-		<t hangText="UNIX Leap Time">
+		<t hangText="UNIX Leap Time:">
	          UNIX time plus all preceding leap second corrections.
		  For example, if the first leap second record in a TZif file
	          occurs at 1972-06-30 23:59:60 UTC, the UNIX leap time for
@@ -321,7 +321,9 @@
	                  The TZ string in the
			  <xref target='footer'>footer</xref>, if
	                  nonempty, MUST strictly adhere to the
-			  POSIX requirements for the TZ environment variable.
+			  POSIX requirements for the TZ environment variable,
+			  and MUST encode the POSIX portable character set as
+			  ASCII.
		        </t>
		        <t hangText="'3' (0x33)">
			  Version 3 - The file MUST contain the 32-bit
@@ -330,6 +332,7 @@
	                  The TZ string in the
			  <xref target='footer'>footer</xref>, if
			  nonempty, MUST conform to POSIX requirements
+			  with ASCII encoding,
			  except that it MAY use the TZ string extensions
	                  <xref target='tzstrext'>described below</xref>.
	                  <vspace/><vspace/>
@@ -482,16 +485,19 @@
	                </t>
		        <t hangText="(desig)idx:">
	                  A one-octet unsigned integer specifying an index into
-	                  the series of time zone designation characters,
+			  the series of time zone designation octets,
	                  thereby selecting a particular designation string.
-	                  Each index MUST be in the range [0, 'charcnt').
+			  Each index MUST be in the range [0, 'charcnt'), and
+			  MUST index a NUL-terminated (0x00) sequence of
+			  octets that SHOULD consist of three (3) to six (6)
+			  ASCII characters that are '-', '+' or alphanumeric.
	                </t>
	              </list>
	            </t>
		    <t hangText="time zone designations:">
-	              A series of ASCII characters constituting an array of
+		      A series of octets constituting an array of
	              NUL-terminated (0x00) time zone designation strings.
-	              The total number of characters is specified by the
+		      The total number of octets is specified by the
	              'charcnt' field in the header.
	              Note that two designations MAY overlap if one is a
	              suffix of the other.
@@ -510,7 +516,7 @@
		      MUST have a zero 'leapcnt' in the 32- and 64-bit data,
		      and a file that uses the MIME media type
		      <xref target="tzif-leap">"application/tzif-leap"</xref>
-		      SHOULD have a non-zero 'leapcnt' in the 64-bit data
+		      MUST have a non-zero 'leapcnt' in the 64-bit data
		      (or in the 32-bit data, in a version 1 file).
	              Each record has one of the following structures:
		      <list style='hanging'>
@@ -627,7 +633,7 @@
	        <t>In order to eliminate unused space in a TZif file, every
		nonzero local time type index SHOULD appear at least once in the
		transition type array.
-	        Likewise, every character in the time zone designations array
+		Likewise, every octet in the time zone designations array
	        SHOULD be used by at least one time type record.</t>
	      </section>  <!-- data -->
	
@@ -656,7 +662,7 @@
		      The string is either empty or uses the expanded format
	              of the "TZ" environment variable as defined in Section 8
	              of the "Base Definitions" Volume of <xref
-		      target="POSIX"/>, possibly with extensions
+		      target="POSIX"/> with ASCII encoding, possibly with extensions
		      <xref target='tzstrext'>described below</xref> in
		      version 3 files.
		      If the string is empty, the corresponding information is
@@ -667,7 +673,8 @@
	              evaluating the TZ string at the time of the last
	              transition should yield the same time type as the time
	              type specified in the last transition.
-	              Note that the string MUST NOT be NUL-terminated and
+		      The string MUST NOT contain NUL octets or be
+		      NUL-terminated, and
	              SHOULD NOT begin with the ':' (colon) character.
	            </t>
		  </list>
@@ -717,6 +724,88 @@
	
	    </section>  <!-- tzif -->
	
+	    <section anchor='interop'
+		     title='Interoperability Considerations'>
+	
+	      <t>When new versions of the TZif format have been defined,
+	      they were compatible with older readers as much as is practical,
+	      so that a reader can successfully use a TZif file of a
+	      later version than what the reader was designed for.
+	      However, this compatibility is not perfect.
+	      For example, a reader that groks only version 2
+	      format might mishandle timestamps after a version 3
+	      file's last transition, because it cannot parse the
+	      version 3 POSIX-style TZ string.
+	      Writers of TZif files can attempt to work around
+	      these issues; for example, a version 3 writer
+	      can output more transitions than necessary,
+	      so that only far-future timestamps are mishandled by
+	      version 2 readers.</t>
+	
+	      <t>Interoperability problems with TZif include the following:
+	      <list style='symbols'>
+		<t>Some readers examine only 32-bit data.</t>
+	
+		<t>Some readers ignore the footer, and instead
+		predict future timestamps from the
+		time type of the last transition.</t>
+	
+		<t>Some readers do not use time type 0 for timestamps
+		before the first transition, in that they
+		infer a time type using a heuristic that does not always
+		select time type 0.</t>
+	
+		<t>Some readers mishandle timestamps before the first transition
+		that has a timestamp not less than -2**31.
+		Readers that support only 32-bit timestamps are likely to be more
+		prone to this problem.</t>
+	
+		<t>Some readers mishandle timestamps before the first transition
+		that has a nonnegative timestamp.
+		Readers that do not support negative timestamps are likely
+		to be more prone to this problem.</t>
+	
+		<t>Some readers mishandle a transition if its
+		timestamp has the minimum possible signed 32- or 64-bit value.</t>
+	
+		<t>Some readers mishandle POSIX-style TZ strings that
+		contain '&lt;' or '&gt;'.</t>
+	
+		<t>Many readers mishandle time zone abbreviations that
+		contain non-ASCII characters.</t>
+	
+		<t>Some readers may mishandle time zone abbreviations that
+		contain fewer than 3 or more than 6 characters, or that
+		contain ASCII characters other than alphanumerics, '-',
+		and '+'.</t>
+	
+		<t>Some readers mishandle time zone abbreviations that
+		contain '+' or '-'.</t>
+	
+		<t>Some readers mishandle UT offsets that are out
+		of the traditional range of -12 through +12 hours, and
+		so do not support locations like Kiritimati that are
+		outside this range.</t>
+	
+		<t>Some readers mishandle UT offsets in the range [-3599, -1]
+		seconds from UT, because they integer-divide the offset by 3600
+		to get 0 and then display the hour part as "+00".</t>
+	
+		<t>Some readers mishandle UT offsets that are not
+		a multiple of one hour, or of 15 minutes, or of 1 minute.</t>
+	
+		<t>Some readers mishandle TZif files that specify
+		daylight-saving time UT offsets that are less than the
+		UT offsets for the corresponding standard time.
+		These readers do not support locations like Ireland,
+		which uses the equivalent of the POSIX TZ string
+		"IST-1GMT0,M10.5.0,M3.5.0/1", observing standard time
+		(IST, +01) in summer and daylight saving time (GMT, +00)
+		in winter.</t>
+	      </list>
+	    </t>
+	    </section>  <!-- interop -->
+	
	    <section anchor='tzdist'
	             title='Use with the Time Zone Data Distribution Service'>
	      <t>The <xref target="RFC7808">Time Zone Data Distribution
@@ -849,15 +938,12 @@
	            externally.
	          </t>
	          <t hangText="Interoperability considerations:">
-	            This media type indicates that leap second records MUST
+		    In addition to the <xref target='interop'>general TZif
+		    interoperability considerations</xref>,
+		    this media type indicates that leap second records MUST
	            NOT be included in the TZif data; 'leapcnt' in the
	            header(s) MUST be zero (0).
	            <vspace/><vspace/>
-	            Implementations SHOULD generate version 2 or 3 files.
-	            The sequence of time changes defined by the 32-bit header
-	            and data block SHOULD be a contiguous subsequence of the
-		    time changes defined by the 64-bit header and data block
-		    and by the footer.
	          </t>
	          <t hangText="Published specification:">
	            This specification.
@@ -908,62 +994,26 @@
	          <t hangText="Optional parameters:">none</t>
	          <t hangText="Encoding considerations:">Binary</t>
	          <t hangText="Security considerations:">
-	            The Time Zone Information Format contains no executable
-	            code and the format does not define any extensibility
-	            areas that could be used to store such code.
-	            <vspace/><vspace/>
-	            TZif contains counted arrays of data elements.  All counts
-	            should be checked when processing TZif objects to guard
-	            against references past the end of the object.
-	            <vspace/><vspace/>
-	            TZif provides no confidentiality or integrity protection.
-	            Time zone information is normally public and does not call
-	            for confidentiality protection.  Since time zone
-	            information is used in many critical applications,
-	            integrity protection may be required, and must be provided
-	            externally.
+		    See "application/tzif".
	          </t>
	          <t hangText="Interoperability considerations:">
-		    This media type indicates that leap second records SHOULD be
-		    included in the TZif data; 'leapcnt' in the 64-bit header
-		    (or in the 32-bit header, if a version 1 file) SHOULD
-	            be non-zero.
-	            <vspace/><vspace/>
-	            Implementations SHOULD generate version 2 or 3 files.
-	            The sequence of time changes defined by the 32-bit header
-	            and data block SHOULD be a contiguous subsequence of the
-		    time changes defined by the 64-bit header and data block
-		    and by the footer.
+		    Same as "application/tzif", except that leap second
+		    records MUST be included in the TZif data; 'leapcnt' in
+		    the 64-bit header (or in the 32-bit header, if a version 1
+		    file) MUST be non-zero.
	          </t>
	          <t hangText="Published specification:">
	            This specification.
	          </t>
	          <t hangText="Applications that use this media type:">
-	            This media type is designed for widespread use by
-	            applications that need to use or exchange time zone
-	            information, such as the
-	            <eref
-			target='http://man7.org/linux/man-pages/man8/zic.8.html'>
-	              Time Zone Information Compiler (zic)
-	            </eref>
-	            and the
-	            <eref target='https://www.gnu.org/software/libc/'>
-	              GNU C Library</eref>.
-	            The <xref target='RFC7808'>Time Zone Distribution
-	            Service</xref> can directly use this media type.
+		    See "application/tzif".
	          </t>
	          <t hangText="Fragment identifier considerations:">N/A</t>
	          <t hangText="Additional information:">
-		    <list style='hanging'>
-	              <t hangText="Magic number(s):">
-	              The first 4 octets are 0x54, 0x5A, 0x69, 0x66</t>
-	              <t hangText="File extensions(s):">N/A</t>
-	              <t hangText="Macintosh file type code(s):">N/A</t>
-	            </list>
+		    See "application/tzif".
	          </t>
-	          <t hangText="Person &amp; email address to contact for further
-	                       information:">
-	            Time Zone Database mailing list &lt;[email-masked]&gt;
+	          <t hangText="Person &amp; email address to contact for further information:">
+		    See "application/tzif".
	          </t>
	          <t hangText="Intended usage:">COMMON</t>
	          <t hangText="Restrictions on usage:">N/A</t>
--- draft-murchison-tzdist-tzif-08.xml	2018-06-28 21:24:43.309524691 -0700
+++ draft-murchison-tzdist-tzif-08-2.xml	2018-06-29 13:42:56.906520753 -0700
@@ -189,7 +189,7 @@
	          leap seconds.  As an extension to POSIX, negative values
	          represent times before the POSIX Epoch, using UT.
	        </t>
-		<t hangText="UNIX Leap Time">
+		<t hangText="UNIX Leap Time:">
	          UNIX time plus all preceding leap second corrections.
		  For example, if the first leap second record in a TZif file
	          occurs at 1972-06-30 23:59:60 UTC, the UNIX leap time for
@@ -260,7 +260,7 @@
	
	      <t>The sequence of time changes defined by the 32-bit header and
	      data block SHOULD be a contiguous subsequence of the time
-	      changes defined by the 64-bit header, data block, and footer.
+	      changes defined by the 64-bit header and data block and by the footer.
	      When reading a version 2 or 3 file, implementations SHOULD
	      ignore the 32-bit header and data block except for the purpose
	      of skipping over them.</t>
@@ -321,7 +321,9 @@
	                  The TZ string in the
			  <xref target='footer'>footer</xref>, if
	                  nonempty, MUST strictly adhere to the
-			  POSIX requirements for the TZ environment variable.
+			  POSIX requirements for the TZ environment variable,
+			  and MUST encode the POSIX portable character set as
+			  ASCII.
		        </t>
		        <t hangText="'3' (0x33)">
			  Version 3 - The file MUST contain the 32-bit
@@ -330,6 +332,7 @@
	                  The TZ string in the
			  <xref target='footer'>footer</xref>, if
			  nonempty, MUST conform to POSIX requirements
+			  with ASCII encoding,
			  except that it MAY use the TZ string extensions
	                  <xref target='tzstrext'>described below</xref>.
	                  <vspace/><vspace/>
@@ -482,16 +485,19 @@
	                </t>
		        <t hangText="(desig)idx:">
	                  A one-octet unsigned integer specifying an index into
-	                  the series of time zone designation characters,
+			  the series of time zone designation octets,
	                  thereby selecting a particular designation string.
-	                  Each index MUST be in the range [0, 'charcnt').
+			  Each index MUST be in the range [0, 'charcnt'), and
+			  MUST index a NUL-terminated (0x00) sequence of
+			  octets that SHOULD consist of three (3) to six (6)
+			  ASCII characters that are '-', '+' or alphanumeric.
	                </t>
	              </list>
	            </t>
		    <t hangText="time zone designations:">
-	              A series of ASCII characters constituting an array of
+		      A series of octets constituting an array of
	              NUL-terminated (0x00) time zone designation strings.
-	              The total number of characters is specified by the
+		      The total number of octets is specified by the
	              'charcnt' field in the header.
	              Note that two designations MAY overlap if one is a
	              suffix of the other.
@@ -504,14 +510,14 @@
	              strictly ascending order.
	              The number of records is specified by the
	              'leapcnt' field in the header.
-	              A file that has a 'leapcnt' of zero (0) MUST use the
-	              MIME media type
-	              <xref target="tzif">"application/tzif"</xref>
-	              if exchanged via a MIME-compatible protocol.
-	              A file that has a non-zero 'leapcnt' MUST use the
-	              MIME media type
-	              <xref target="tzif-leap">"application/tzif-leap"</xref>
-	              if exchanged via a MIME-compatible protocol.
+		      When exchanged via a MIME-compatible protocol,
+		      a file that uses the MIME media type
+		      <xref target="tzif">"application/tzif"</xref>
+		      MUST have a zero 'leapcnt' in the 32- and 64-bit data,
+		      and a file that uses the MIME media type
+		      <xref target="tzif-leap">"application/tzif-leap"</xref>
+		      MUST have a non-zero 'leapcnt' in the 64-bit data
+		      (or in the 32-bit data, in a version 1 file).
	              Each record has one of the following structures:
		      <list style='hanging'>
		        <t hangText="32-bit Data Block:">
@@ -627,7 +633,7 @@
	        <t>In order to eliminate unused space in a TZif file, every
		nonzero local time type index SHOULD appear at least once in the
		transition type array.
-	        Likewise, every character in the time zone designations array
+		Likewise, every octet in the time zone designations array
	        SHOULD be used by at least one time type record.</t>
	      </section>  <!-- data -->
	
@@ -656,7 +662,7 @@
		      The string is either empty or uses the expanded format
	              of the "TZ" environment variable as defined in Section 8
	              of the "Base Definitions" Volume of <xref
-		      target="POSIX"/>, possibly with extensions
+		      target="POSIX"/> with ASCII encoding, possibly with extensions
		      <xref target='tzstrext'>described below</xref> in
		      version 3 files.
		      If the string is empty, the corresponding information is
@@ -667,7 +673,8 @@
	              evaluating the TZ string at the time of the last
	              transition should yield the same time type as the time
	              type specified in the last transition.
-	              Note that the string MUST NOT be NUL-terminated and
+		      The string MUST NOT contain NUL octets or be
+		      NUL-terminated, and
	              SHOULD NOT begin with the ':' (colon) character.
	            </t>
		  </list>
@@ -717,6 +724,88 @@
	
	    </section>  <!-- tzif -->
	
+	    <section anchor='interop'
+		     title='Interoperability Considerations'>
+	
+	      <t>When new versions of the TZif format have been defined,
+	      they were compatible with older readers as much as is practical,
+	      so that a reader can successfully use a TZif file of a
+	      later version than what the reader was designed for.
+	      However, this compatibility is not perfect.
+	      For example, a reader that groks only version 2
+	      format might mishandle timestamps after a version 3
+	      file's last transition, because it cannot parse the
+	      version 3 POSIX-style TZ string.
+	      Writers of TZif files can attempt to work around
+	      these issues; for example, a version 3 writer
+	      can output more transitions than necessary,
+	      so that only far-future timestamps are mishandled by
+	      version 2 readers.</t>
+	
+	      <t>Interoperability problems with TZif include the following:
+	      <list style='symbols'>
+		<t>Some readers examine only 32-bit data.</t>
+	
+		<t>Some readers ignore the footer, and instead
+		predict future timestamps from the
+		time type of the last transition.</t>
+	
+		<t>Some readers do not use time type 0 for timestamps
+		before the first transition, in that they
+		infer a time type using a heuristic that does not always
+		select time type 0.</t>
+	
+		<t>Some readers mishandle timestamps before the first transition
+		that has a timestamp not less than -2**31.
+		Readers that support only 32-bit timestamps are likely to be more
+		prone to this problem.</t>
+	
+		<t>Some readers mishandle timestamps before the first transition
+		that has a nonnegative timestamp.
+		Readers that do not support negative timestamps are likely
+		to be more prone to this problem.</t>
+	
+		<t>Some readers mishandle a transition if its
+		timestamp has the minimum possible signed 32- or 64-bit value.</t>
+	
+		<t>Some readers mishandle POSIX-style TZ strings that
+		contain '&lt;' or '&gt;'.</t>
+	
+		<t>Many readers mishandle time zone abbreviations that
+		contain non-ASCII characters.</t>
+	
+		<t>Some readers may mishandle time zone abbreviations that
+		contain fewer than 3 or more than 6 characters, or that
+		contain ASCII characters other than alphanumerics, '-',
+		and '+'.</t>
+	
+		<t>Some readers mishandle time zone abbreviations that
+		contain '+' or '-'.</t>
+	
+		<t>Some readers mishandle UT offsets that are out
+		of the traditional range of -12 through +12 hours, and
+		so do not support locations like Kiritimati that are
+		outside this range.</t>
+	
+		<t>Some readers mishandle UT offsets in the range [-3599, -1]
+		seconds from UT, because they integer-divide the offset by 3600
+		to get 0 and then display the hour part as "+00".</t>
+	
+		<t>Some readers mishandle UT offsets that are not
+		a multiple of one hour, or of 15 minutes, or of 1 minute.</t>
+	
+		<t>Some readers mishandle TZif files that specify
+		daylight-saving time UT offsets that are less than the
+		UT offsets for the corresponding standard time.
+		These readers do not support locations like Ireland,
+		which uses the equivalent of the POSIX TZ string
+		"IST-1GMT0,M10.5.0,M3.5.0/1", observing standard time
+		(IST, +01) in summer and daylight saving time (GMT, +00)
+		in winter.</t>
+	      </list>
+	    </t>
+	    </section>  <!-- interop -->
+	
	    <section anchor='tzdist'
	             title='Use with the Time Zone Data Distribution Service'>
	      <t>The <xref target="RFC7808">Time Zone Data Distribution
@@ -849,14 +938,12 @@
	            externally.
	          </t>
	          <t hangText="Interoperability considerations:">
-	            This media type indicates that leap second records MUST
+		    In addition to the <xref target='interop'>general TZif
+		    interoperability considerations</xref>,
+		    this media type indicates that leap second records MUST
	            NOT be included in the TZif data; 'leapcnt' in the
	            header(s) MUST be zero (0).
	            <vspace/><vspace/>
-	            Implementations SHOULD generate version 2 or 3 files.
-	            The sequence of time changes defined by the 32-bit header
-	            and data block SHOULD be a contiguous subsequence of the
-	            time changes defined by the 64-bit header and data block.
	          </t>
	          <t hangText="Published specification:">
	            This specification.
@@ -907,60 +994,26 @@
	          <t hangText="Optional parameters:">none</t>
	          <t hangText="Encoding considerations:">Binary</t>
	          <t hangText="Security considerations:">
-	            The Time Zone Information Format contains no executable
-	            code and the format does not define any extensibility
-	            areas that could be used to store such code.
-	            <vspace/><vspace/>
-	            TZif contains counted arrays of data elements.  All counts
-	            should be checked when processing TZif objects to guard
-	            against references past the end of the object.
-	            <vspace/><vspace/>
-	            TZif provides no confidentiality or integrity protection.
-	            Time zone information is normally public and does not call
-	            for confidentiality protection.  Since time zone
-	            information is used in many critical applications,
-	            integrity protection may be required, and must be provided
-	            externally.
+		    See "application/tzif".
	          </t>
	          <t hangText="Interoperability considerations:">
-	            This media type indicates that leap second records MUST be
-	            included in the TZif data; 'leapcnt' in the header(s) MUST
-	            be non-zero.
-	            <vspace/><vspace/>
-	            Implementations SHOULD generate version 2 or 3 files.
-	            The sequence of time changes defined by the 32-bit header
-	            and data block SHOULD be a contiguous subsequence of the
-	            time changes defined by the 64-bit header and data block.
+		    Same as "application/tzif", except that leap second
+		    records MUST be included in the TZif data; 'leapcnt' in
+		    the 64-bit header (or in the 32-bit header, if a version 1
+		    file) MUST be non-zero.
	          </t>
	          <t hangText="Published specification:">
	            This specification.
	          </t>
	          <t hangText="Applications that use this media type:">
-	            This media type is designed for widespread use by
-	            applications that need to use or exchange time zone
-	            information, such as the
-	            <eref
-			target='http://man7.org/linux/man-pages/man8/zic.8.html'>
-	              Time Zone Information Compiler (zic)
-	            </eref>
-	            and the
-	            <eref target='https://www.gnu.org/software/libc/'>
-	              GNU C Library</eref>.
-	            The <xref target='RFC7808'>Time Zone Distribution
-	            Service</xref> can directly use this media type.
+		    See "application/tzif".
	          </t>
	          <t hangText="Fragment identifier considerations:">N/A</t>
	          <t hangText="Additional information:">
-		    <list style='hanging'>
-	              <t hangText="Magic number(s):">
-	              The first 4 octets are 0x54, 0x5A, 0x69, 0x66</t>
-	              <t hangText="File extensions(s):">N/A</t>
-	              <t hangText="Macintosh file type code(s):">N/A</t>
-	            </list>
+		    See "application/tzif".
	          </t>
-	          <t hangText="Person &amp; email address to contact for further
-	                       information:">
-	            Time Zone Database mailing list &lt;[email-masked]&gt;
+	          <t hangText="Person &amp; email address to contact for further information:">
+		    See "application/tzif".
	          </t>
	          <t hangText="Intended usage:">COMMON</t>
	          <t hangText="Restrictions on usage:">N/A</t>

---

From: Ken Murchison <[email-masked]>
Date: Fri, 29 Jun 2018 16:49:51 -0400

Funny that you suggest an interop section.  I have been compiling one 
for the 30min prior to your email.  I will probably hand-merge your 
patch to fit into my new layout.

On 06/29/2018 04:43 PM, Paul Eggert wrote:
> allows TZ strings to be EBCDIC (or any other encoding that has the 
> ASCII characters), in practice as far as I know they are always ASCII 
> in TZif files, and the spec should address this issue.
>
> * Similarly, the spec nowhere says that time zone abbreviations must 
> be in ASCII when in the character array. Here, the reference 
> implementation allows any non-NUL bytes. However, it'd be bad practice 
> to depart from the POSIX rules for time zone abbreviations as a lot of 
> readers assume ASCII, so the patch uses a SHOULD for this, and also 
> replaces "character" with "octet" to allow (though recommend against) 
> non-ASCII characters (presumably UTF-8?).
>
> * POSIX requires time zone abbreviations to be at least 3 and at most 
> 6 characters; the spec should mention this.
>
> main text. Better would be an interoperability section that talks 
> about practical problems, without any MUSTs or SHOULDs (which should 
> be part of the main spec), and the MIME interoperability paragraphs 
> can mostly just refer to this new section. A draft of this is in the 
> patch. (Whew, that took more time to write than I thought it would! 
> There are a lot of interoperability issues. I've probably forgotten 
> some. On the other hand, maybe I've gone overboard in describing so 
> many glitches.)
>

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Ken Murchison <[email-masked]>
Date: Sat, 30 Jun 2018 13:45:53 -0400

I just posted version -09.  It includes most of your editorial changes 
and my previously started interop section.  I felt it was best to 
consolidate all of the best practices in one area rather than having 
them scattered throughout the text.  This also makes it easy to 
reference just the one section.

Issues/questions:

- Since we now use the term 'octets' for the contents of time zone 
designations, 'charcnt' seems out of place.  Should we change to 'octetcnt'?

- Your interop section seems to describe mostly bugs in readers. I'm not 
sure we need/want a list of where non-compliant applications fail 
(unless we put it in an appendix?)  Do we have best practices to avoid 
some of these things?

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Sat, 30 Jun 2018 17:43:00 -0700

Ken Murchison wrote:

> - Since we now use the term 'octets' for the contents of time zone designations, 
> 'charcnt' seems out of place.  Should we change to 'octetcnt'?

In practice they're single-byte ASCII characters, which are both octets and 
characters. Though I don't feel about it strongly, I'd leave it alone since it's 
been called 'charcnt' since the 1980s.

> - Your interop section seems to describe mostly bugs in readers. I'm not sure we 
> need/want a list of where non-compliant applications fail (unless we put it in 
> an appendix?)  Do we have best practices to avoid some of these things?

An appendix or other such section would be fine. By my count only five of the 
bulleted issues are clear reader bugs; the other eight issues describe what 
readers following older, vaguer specs will do when given files that meet this 
newer spec. The first three bullet points are in the latter category, as are the 
points about '<' and '>', the two non-ASCII issues, weird time zone 
abbreviations, and negative DST.

The idea is to document common problems in implementing and upgrading the spec, 
to help TZif writers output files that avoid common pitfalls in older or buggy 
TZif readers (this is the "be conservative in what you do" part of Postel's 
law), and to help future spec-writers (assuming there are any) see what sort of 
things they are up against when they're defining version 4, 5, etc.