Subject: Re: xsd:dateTime vs. RFC 3339

From: Robert Sayre <[email-masked]>
Date: Fri, 04 Feb 2005 11:37:11 -0500

Norman Walsh wrote:
> The 05 draft of the Atom format says:
>
> I know we're writing an IETF document, but I think there's going to be
> a lot of off-the-shelf XML software that understands xsd:dateTimes and
> I think it would be a lot better if we defined Date Constructs in
> terms of W3C XML Schema Part 2 than RFC 3339.
> 
> I propose that we change the spec to do so.
> 

I agree. I was just writing a protocol implementation in Ruby On Rails 
(CRUDs very fast, btw). When I got to the part on date formats, I used 
xsd:dateTime code that was already done, figuring that's what everyone 
else will do.

Robert Sayre

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 17:55:12 +0100

> xsd:dateTime code that was already done, figuring that's what everyone 

But in that case, we'll need to profile xsd:dateTime. For instance, that 
one allows timestamps without timezone (with a distinct meaning!).

Best regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Norman Walsh <[email-masked]>
Date: Fri, 04 Feb 2005 12:22:55 -0500

/ Julian Reschke <[email-masked]> was heard to say:
| Robert Sayre wrote:
[...]
|> I agree. I was just writing a protocol implementation in Ruby On
|> Rails (CRUDs very fast, btw). When I got to the part on date
|> formats, I used xsd:dateTime code that was already done, figuring
|> that's what everyone else will do.
|
| But in that case, we'll need to profile xsd:dateTime. For instance,
| that one allows timestamps without timezone (with a distinct meaning!).

Or not. I propose that we say that Date Constructs MUST be valid
xsd:dateTime values and SHOULD include an explicit time zone. Heck, I
could even live with SHOULD be in UTC. If pressed, I could live with
MUST be in UTC.

But the evidence suggests that software is, in practice, going to use
libraries that recognize and process xsd:dateTime values, so defining
it in some other way is just going to lead to deviation from the spec.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | He that shuns trifles must shun the
http://nwalsh.com/            | world.--George Chapman

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 18:21:48 +0100

Norman Walsh wrote:
>       3.3  Date Constructs
>       A Date construct is an element whose content MUST conform to the
>       date-time BNF rule in [RFC3339].
> 
> I'm actually using xsd:dateTime in the RELAX NG grammar and I went off
> to look at RFC 3339 thinking I might write a regex to do that instead,
> since that's what the spec says. Then I got to wondering just how similar
> the two definitions actually are. In some private correspondence, Paul
> Byron made the following observations:
> 
> 1. RFC 3339 allows lower- and upper-case 'T' and 'Z', while
>    xsd:dateTime requires upper-case only. There is a note in sect 5.6
>    of RFC 3339 which says:
> 
>       This date/time format may be used in some environments or
>       contexts that distinguish between the upper- and lower-case
>       letters 'A'-'Z' and 'a'-'z' (e.g. XML). Specifications that use
>       this format in such environments MAY further limit the date/time
>       syntax so that the letters 'T' and 'Z' used in the date/time
>       syntax must always be upper case. Applications that generate
>       this format SHOULD use upper case letters.

I don't see this as a problem. We can either allow both (default), or 
explicitly restrict it.

> 2. RFC 3339 allows the replacement of the 'T' with a space which
>    xsd:dateTime does not. [Note: ISO 8601 doesn't either, hence RFC
>    3339 isn't actually a "profile of ISO 8601" as it claims].

As far as I can tell, ISO 8601 allows this.

> 3. RFC 3339 does not allow the hour 24 but xsd:dateTime does (in
>    xsd:dateTime [as in ISO 8601] 00 & 24 both represent midnight, but
>    24 means the end of one day while 00 means the start of the next)

That looks like a useful restriction to me...

> 4. RFC 3339 gives a different semantic to a timezone offset of -00:00 from
>    +00:00 and Z which xsd:dateTime (and to the best of my knowledge
>    ISO 8601) doesn't. Sect 4.3 of RFC 3339 reads:
> 
>      If the time in UTC is known, but the offset to local time is
>      unknown, this can be represented with an offset of "-00:00". This
>      differs semantically from an offset of "Z" or "+00:00", which
>      imply that UTC is the preferred reference point for the specified
>      time. RFC2822 [IMAIL-UPDATE] describes a similar convention for
>      email.

I'm not sure how this is relevant. All three representations (Z, +00:00, 
-00:00) refer to the same instant in time.

> He goes on to suggest that there may be other differences, but these
> are the ones he noticed. That's enough for me: they are different.
 >

So what do you do with something like

	2005-02-04T17:20:00

?

-> Neither xsd:dateTime nor RFC3339 are perfect for our needs, so we may 
have to profile one of them. In which case I'd prefer to stick with RFC3339.

Best regards, Julian

---

From: Norman Walsh <[email-masked]>
Date: Fri, 04 Feb 2005 11:18:17 -0500

3.3  Date Constructs

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | It is important what a man still plans
http://nwalsh.com/            | at the end. It shows the measure of
                              | injustice in his death.--Elias Canetti

---

From: Norman Walsh <[email-masked]>
Date: Fri, 04 Feb 2005 12:29:44 -0500

/ Julian Reschke <[email-masked]> was heard to say:
[...]
| So what do you do with something like
|
| 	2005-02-04T17:20:00

Dunno.

| -> Neither xsd:dateTime nor RFC3339 are perfect for our needs, so we
| may have to profile one of them. In which case I'd prefer to stick
| with RFC3339.

*shrug* I've said why I think xsd:dateTime is a better choice, but
you're right, we just have to pick one.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | A new scientific truth does not triumph
http://nwalsh.com/            | by convincing its opponents and making
                              | them see the light, but rather because
                              | its opponents eventually die, and a new
                              | generation grows up that is familiar
                              | with it (Planck 1949)

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 18:36:41 +0100

Julian Reschke wrote:
> ...

And anyway, what RFC3339 says is:

"   date-time       = full-date "T" full-time

       NOTE: Per [ABNF] and ISO8601, the "T" and "Z" characters in this
       syntax may alternatively be lower case "t" or "z" respectively.

       This date/time format may be used in some environments or contexts
       that distinguish between the upper- and lower-case letters 'A'-'Z'
       and 'a'-'z' (e.g. XML).  Specifications that use this format in
       such environments MAY further limit the date/time syntax so that
       the letters 'T' and 'Z' used in the date/time syntax must always
       be upper case.  Applications that generate this format SHOULD use
       upper case letters.

       NOTE: ISO 8601 defines date and time separated by "T".
       Applications using this syntax may choose, for the sake of
       readability, to specify a full-date and full-time separated by
       (say) a space character."

So as long as we normatively refer to the "date-time" production, only 
"T" is allowed.

In general, I find RFC3339 a lot easier to read than xsd:dateTime, and 
for the date representations we actually want to support, they define 
the same meaning.

Thus -1 on changing it, and +1 on possibly profiling it further.

Best regards, Julian

---

From: Walter Underwood <[email-masked]>
Date: Fri, 04 Feb 2005 09:39:13 -0800

> terms of W3C XML Schema Part 2 than RFC 3339.

Strongly agree.

Also, we have an unresolved issue with historic Livejournal entries,
which do not have timezones. XML Schema explains exactly how to 
handle those. We can have a SHOULD for timezone info, with an explanation
of what you lose without that.

wunder
--
Walter Underwood
Principal Architect, Verity

---

From: Norman Walsh <[email-masked]>
Date: Fri, 04 Feb 2005 12:44:46 -0500

Perhaps a compromise? Date Construct values MUST be valid xsd:dateTime
values and MUST also match the following regular expression:

  [0-9]{8}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?(Z|[\+\-][0-9]{2}:[0-9]{2})

(Expressed for my own testing purposes in XML Schema pattern syntax.)

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Some disguised deceits counterfeit
http://nwalsh.com/            | truth so perfectly that not to be taken
                              | in by them would be an error of
                              | judgment.--La Rochefoucauld

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 18:46:33 +0100

> which do not have timezones. XML Schema explains exactly how to 

So what does it recommend?

> of what you lose without that.

So what is a recipient of a timestampe without TZ info supposed to do 
with it? Throw it away? Default to UTC?

-1

---

From: Walter Underwood <[email-masked]>
Date: Fri, 04 Feb 2005 10:05:50 -0800

>> of what you lose without that.

Treating the datetime value as if it has an uncertainty equal to the
maximum possible timezone offset. The other advantage of use XML Schema
is that is defines how to order timestamps, which is the main thing
we want to do with them.

I think the section is pretty clear, and I'm picky about specs:

  <http://www.w3.org/TR/xmlschema-2/#dateTime>

wunder

---

From: Tim Bray <[email-masked]>
Date: Fri, 04 Feb 2005 10:36:52 -0800

> xsd:dateTime code that was already done, figuring that's what everyone 

For what it's worth, in Perl and C I use Posix strftime with this as 
the argument:

  "%Y-%m-%dT%H:%M:%S-08:00"

(the -08 bit because I'm in Pacific time, adjust as appropriate).
  -Tim

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 20:33:58 +0100

> (Expressed for my own testing purposes in XML Schema pattern syntax.)

Sounds good to me, except that I would keep the reference to RFC3339 
(semantics are the same, but I think RFC3339 is more readable).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Feb 2005 20:37:42 +0100

> maximum possible timezone offset. The other advantage of use XML Schema

Just to clarify: do you see that as as MUST or MAY level requirement? 
I'm pretty sure that most recipients that actually parse the date will 
only persist a timestamp (such as ms since the epoch), and not bother 
with any addutional information.

>   <http://www.w3.org/TR/xmlschema-2/#dateTime>

As far as I am concerned, the problem with xsd:dateTime is that it 
includes features (such as the aforementioned TZ uncertainty) that most 
recipients aren't prepared to handle. So, we need to

1) either subset it so that that syntax is not allowed, or

2) define what recipients should do with these timestamps.

I'm -1 on option 2).

Julian

---

From: Robert Sayre <[email-masked]>
Date: Fri, 04 Feb 2005 16:18:52 -0500

Tim Bray wrote:
> For what it's worth, in Perl and C I use Posix strftime with this as the 
> argument:
> 

I was parsing, but one of the bad things about Ruby On Rails is that you 
don't need to learn Ruby to use it. I have three working apps done and I 
think I've written like 150 lines of Ruby in my life. So, I just tried 
the standard library's Date.parse() function, which applies all sorts of 
grubby heuristics. Worked fine. That's probably similar to what most 
people in scripting land will do, so I guess the difference between XSD 
and RFC 3339 isn't a big deal from the that point of view.

Robert Sayre

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 05 Feb 2005 20:46:42 +0100

> terms of W3C XML Schema Part 2 than RFC 3339.

Now I agree as well.

-- 
Asbjørn Ulsberg     -=|=-    http://virtuelvis.com/quark/
«He's a loathsome offensive brute, yet I can't look away»

---

From: Tim Bray <[email-masked]>
Date: Sat, 05 Feb 2005 12:10:13 -0800

>> terms of W3C XML Schema Part 2 than RFC 3339.

I tend to agree as well; in this case, the fact that this is an XML 
vocabulary is probably more relevant than the fact that it's an IETF 
RFC.  So can we please have a Pace to call out to XSD?  I'm pretty sure 
that implementors would just use the libraries and The Right Thing 
Would Happen. -Tim

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Feb 2005 21:43:15 +0100

Tim Bray wrote:
> that implementors would just use the libraries and The Right Thing Would 
> Happen. -Tim

Risking that I sound like a broken report...: xsd:dateTime allows a 
superset of what we can represent in RFC3339 (I'm talking about 
semantics, not syntax here). So if we *don't* profile it, the spec will 
need to explain how to obtain an ordering from a set of atom:updated 
timestamps where some are lacking the timezone information.

Best regards, Julian

---

From: Norman Walsh <[email-masked]>
Date: Sat, 05 Feb 2005 15:43:03 -0500

/ Tim Bray <[email-masked]> was heard to say:
| I tend to agree as well; in this case, the fact that this is an XML
| vocabulary is probably more relevant than the fact that it's an IETF
| RFC.  So can we please have a Pace to call out to XSD?  I'm pretty

Done. PaceDatesXSD

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Man's great misfortune is that he has
http://nwalsh.com/            | no organ, no kind of eyelid or brake,
                              | to mask or block a thought, or all
                              | thought, when he wants to.--Paul Valéry

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 05 Feb 2005 21:52:34 +0100

>that implementors would just use the libraries and The Right Thing 

That depends on whether we include restrictions that such libraries
would ignore...
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Robert Sayre <[email-masked]>
Date: Sat, 05 Feb 2005 15:54:55 -0500

> timestamps where some are lacking the timezone information.

I would tend to agree. Can we have that regex go in the Pace itself? 
That would make the proposal pretty much editorial, since the set of 
allowed timestamps would be the same (right?).

Robert Sayre

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Feb 2005 22:07:02 +0100

> allowed timestamps would be the same (right?).

As long as the set of allowed timestamps and their semantics is the 
same, that's fine with me. I still have doubts why this is better than 
sticking with the RFC version, though :-)

Julian

---

From: Norman Walsh <[email-masked]>
Date: Sat, 05 Feb 2005 16:19:20 -0500

/ Robert Sayre <[email-masked]> was heard to say:
| I would tend to agree. Can we have that regex go in the Pace itself?

The regex is in the pace.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Life is a great bundle of little
http://nwalsh.com/            | things.--Oliver Wendell Holmes

---

From: Robert Sayre <[email-masked]>
Date: Sat, 05 Feb 2005 16:36:54 -0500

> The regex is in the pace.

I took the liberty of adding it to the proposal section.

Robert Sayre

---

From: Sam Ruby <[email-masked]>
Date: Sat, 05 Feb 2005 17:38:53 -0500

-1

See http://www.imc.org/atom-syntax/mail-archive/msg06611.html

(Note: Scott is the IETF Director responsible for advising our workgroup)

Having written the datetime support for Apache Axis (a webservice 
implementation based on XSD schema and having hosted a number of SOAP 
interop facilities, I am +1 on the regular expression limitation, 
believe that all dates that that conform to this limitation are valid 
RFC 3339 and xsd:dateTime values, and believe that it will interop with 
all of the existing XSD implementations.

- Sam Ruby

---

From: "Scott Hollenbeck" <[email-masked]>
Date: Sat, 5 Feb 2005 19:43:04 -0500

> Having written the datetime support for Apache Axis (a 
> webservice implementation based on XSD schema and having 
> hosted a number of SOAP interop facilities, I am +1 on the 
> regular expression limitation, believe that all dates that 
> that conform to this limitation are valid RFC 3339 and 
> xsd:dateTime values, and believe that it will interop with 

I'm not sure that I understand why some folks seem to think that this has to
be an either-or situation.  As Sam noted, it's quite possible to specify
date-time syntax that conform to both RFC 3339 and XML Schema Part 2.
Here's what I did in RFC 3731:

2.4.  Dates and Times

   Date and time attribute values MUST be represented in Universal
   Coordinated Time (UTC) using the Gregorian calendar.  The extended
   date-time form using upper case "T" and "Z" characters defined in
   [RFC3339] MUST be used to represent date-time values as XML Schema
   does not support truncated date-time forms or lower case "T" and "Z"
   characters.

I then used the xsd:dateTime data type in my schema (with an appropriate
reference to Part 2), and all is well.  The normative text eliminates some
of the 3339 possibilities that don't work or play well with XML Schema.

-Scott-

---

From: Eric Scheid <[email-masked]>
Date: Sun, 06 Feb 2005 15:01:15 +1100

On 6/2/05 9:38 AM, "Sam Ruby" <[email-masked]> wrote:

> I am +1 on the regular expression limitation,

where "this" is the regex? that regex allows this totally bogus value

    2005-13-34T25:65:96+00:65

there are test cases in the validator to check this, but those test cases
will no longer be supported by the spec and will thus be a bug in the
validator :-(

e.

---

From: Norman Walsh <[email-masked]>
Date: Mon, 07 Feb 2005 08:38:55 -0500

/ Eric Scheid <[email-masked]> was heard to say:
| On 6/2/05 9:38 AM, "Sam Ruby" <[email-masked]> wrote:
|
|> I am +1 on the regular expression limitation,
|> believe that all dates that that conform to this limitation are valid
|> RFC 3339 and xsd:dateTime values, and believe that it will interop with
|> all of the existing XSD implementations.
|
| where "this" is the regex? that regex allows this totally bogus value
|
|     2005-13-34T25:65:96+00:65
|
| there are test cases in the validator to check this, but those test cases
| will no longer be supported by the spec and will thus be a bug in the
| validator :-(

I think the proposed constraint was (RFC 3339 or xsd:dateTime) and
matching the regular expression.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | The only thing that makes life possible
http://nwalsh.com/            | is permanent, intolerable uncertainty;
                              | not knowing what comes next.--Ursula K.
                              | LeGuin

---

From: Sam Ruby <[email-masked]>
Date: Mon, 07 Feb 2005 10:17:39 -0500

> matching the regular expression.

Let's take credit for all four requirements: regular expression, RFC 
3339, xsd:dateTime, and w3cdtf.

I don't know of any valid tool produced dates which would not conform to 
all four, so the requirement on the producer side should not be onerous.

This would enable the widest range of readers.

- Sam Ruby