Subject: [Geojson] Timestamps for feature collections

From: Sean Gillies <[email-masked]>
Date: Thu, 3 Dec 2015 15:15:48 -0700

In working through open issues in the draft-geojson repo, I've come to
https://github.com/geojson/draft-geojson/issues/80. I'd like to remind all
that, as I wrote there

> This issue is not about representing trajectories of features or paths in
> time. It's about letting providers specify that coordinates in a collection
> were measured at a particular time so that future users can account for WGS
> 84 revisions and drift between, say, Ordnance Survey National Grid and WGS.
>
Having said that, I'd like to get consensus on adding or not adding a
recommendation for an optional single timestamp to collections and on
specifying the interpretation of measurement time when no timestamp is
provided, using filesystem or transport protocol timestamps.
-- 
Sean Gillies

---

From: Allan Doyle <[email-masked]>
Date: Thu, 3 Dec 2015 20:20:34 -0500

I don’t think I have heartburn about adding a note to encourage time, but I would not make any mention of what to assume if there is no time stamp. There’s no way to say anything meaningful about timestamps gleaned from outside sources like filesystems or transports.

> On Dec 3, 2015, at 5:15 PM, Sean Gillies <[email-masked]> wrote:
> 
> In working through open issues in the draft-geojson repo, I've come to https://github.com/geojson/draft-geojson/issues/80 <https://github.com/geojson/draft-geojson/issues/80>. I'd like to remind all that, as I wrote there
> This issue is not about representing trajectories of features or paths in time. It's about letting providers specify that coordinates in a collection were measured at a particular time so that future users can account for WGS 84 revisions and drift between, say, Ordnance Survey National Grid and WGS.
> 
> Having said that, I'd like to get consensus on adding or not adding a recommendation for an optional single timestamp to collections and on specifying the interpretation of measurement time when no timestamp is provided, using filesystem or transport protocol timestamps.
> 
> -- 
> Sean Gillies
> _______________________________________________
> GeoJSON mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/geojson

---

From: Martin Thomson <[email-masked]>
Date: Fri, 4 Dec 2015 15:44:16 +1100

> provided, using filesystem or transport protocol timestamps.

Having a timestamp in the format avoids a lot issues: timestamps on
files are fabulously unreliable.

I don't think that you need to encourage its use.  Different
applications will make their own assessment about the value of the
parameter.

When you say collections, do you mean geometry, feature or both?  And
is there any good reason to limit this to collections?

---

From: Calvin Metcalf <[email-masked]>
Date: Fri, 04 Dec 2015 11:08:43 +0000

Is there anything specifying how the timestamp is formated? If timestamp
means something specific in this context then we don't need to worry about
it otherwise we should probably reference an rfc.

---

From: Erik Wilde <[email-masked]>
Date: Fri, 4 Dec 2015 16:29:24 +0100

On 2015-12-04 12:08, Calvin Metcalf wrote:
> means something specific in this context then we don't need to worry
> about it otherwise we should probably reference an rfc.

+1. one possibility would be to use this as an example to describe how 
extensions can/should be used. this would mean:

- put the timestamp in a place where the extension model says that 
extensions should be expected.

- define the extension in the same way as regular core properties, i.e. 
in terms of the acceptable values, and what they mean.

-- 
erik wilde | mailto:[email-masked] |
            | http://dret.net/netdret    |
            | http://twitter.com/dret    |

---

From: Martin Thomson <[email-masked]>
Date: Sat, 5 Dec 2015 06:29:12 +1100

Or, it could be a core property and we can run that exercise on paper to
validate the extensions guidance.

Btw, RFC 3339 should do here.

---

From: Erik Wilde <[email-masked]>
Date: Fri, 4 Dec 2015 20:40:26 +0100

On 2015-12-04 20:29, Martin Thomson wrote:
> On Dec 5, 2015 2:29 AM, "Erik Wilde" <[email-masked]
> <mailto:[email-masked]>> wrote:
>  > +1. one possibility would be to use this as an example to describe
> how extensions can/should be used. this would mean:
>  > - define the extension in the same way as regular core properties,
> i.e. in terms of the acceptable values, and what they mean.

sure, that would work as well, as long as it is kept optional (so that 
it does not introduce any backwards compatibility issues).

---

From: Martin Thomson <[email-masked]>
Date: Sat, 5 Dec 2015 20:27:04 +1100

On 5 December 2015 at 06:40, Erik Wilde <[email-masked]> wrote:
> sure, that would work as well, as long as it is kept optional (so that it
> does not introduce any backwards compatibility issues).

No question about that.  Not everyone will want to provide a timestamp anyway.

---

From: Alexandre Petrescu <[email-masked]>
Date: Sat, 5 Dec 2015 13:13:29 +0100

One _would_ care about the effects of how a timestamp is formatted, 
maybe the allowed "resolution" (every second?  10Hz? sampling resolution 
vs recording resolution vs transmission message frequency) and a maximum 
lifespan (until year 3000 for example).

Protocols use various formatting of a timestamp (e.g. NTP, PMIP) for 
particular cases of application use.

Since this is WGS84 exclusively, and seems to be in the context of 
consumer electronics equipped with GPS receivers, one would consider the 
WGS84 timestamp format with a max resolution of 10Hz to live until year 
2050.

Alex

Le 04/12/2015 12:08, Calvin Metcalf a écrit :
> On Thu, Dec 3, 2015 at 11:44 PM Martin Thomson <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     On 4 December 2015 at 09:15, Sean Gillies <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     [email-masked] <mailto:[email-masked]>

---

From: Erik Wilde <[email-masked]>
Date: Sat, 5 Dec 2015 14:03:19 +0100

> WGS84 timestamp format with a max resolution of 10Hz to live until year

i'd be in favor of using something a bit more webby, 
https://tools.ietf.org/html/rfc3339 probably would be a good choice. it 
does allow fractional seconds, if such a resolution is necessary.

---

From: Carl Reed <[email-masked]>
Date: Sat, 5 Dec 2015 14:00:41 -0700

Alex

Time stamps, time, time duration, etc is a non-trivial topic. Suddenly many
other requirements related to "time" become important. If you want an
example of what I am suggesting check out OGC/ISO Observations and
Measurements as well as the more recent JSON encoding for time elements
related to an observation (soon to be published as it was just approved
last week for release).

Therefore, if just a time stamp is important, I would keep that use
optional and very simple. Other IETF RFCs express time. Those of wide
importance to Next Gen 911 in the States would be a good reference for an
identical (or harmonized) time stamp definition. For example:

<timestamp>2003-06-22T20:57:29Z</timestamp>

Sorry about the XML :-) Or possibly time as expressed in HTML5.

Cheers

Carl

On Sat, Dec 5, 2015 at 5:13 AM, Alexandre Petrescu <
[email-masked]> wrote:

> One _would_ care about the effects of how a timestamp is formatted, maybe
> the allowed "resolution" (every second?  10Hz? sampling resolution vs
> recording resolution vs transmission message frequency) and a maximum

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Sean Gillies <[email-masked]>
Date: Mon, 7 Dec 2015 23:34:46 -0700

Erik,

> does allow fractional seconds, if such a resolution is necessary.

Yes, (as Martin also pointed out) RFC 3339 is the right fit.

There's precedence at
http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON#Structure. The
OSM Overpass API is a popular and very useful service. It's timestamp
property is underspecified

timestamp: the timestamp as given by the data source

but in the example, you can see RFC 3339

    "timestamp": "2013-04-25T08:50:03Z"

I feel like one timestamp – not necessarily named "timestamp" and since
there are a bunch of different common metadata timestamps in use (see
Dublin Core, Atom, etc) a generic "timestamp" seems wrong to me – for a
feature collection addresses a big issue: data archival. GeoJSON files, if
saved for a number of years, will need coordinate adjustment to be used
accurately. It seems to me that collections of features are going to be the
kind of GeoJSON that gets archived. Single features or geometries may pass
in messages, but I don't see them being saved for long. I also don't see a
use case for serving collections of features that have different coordinate
(and attribute) timestamps: implementers will need to reconcile aggregated
features to a uniform coordinate basis if they want to provide any value to
end users.

Could we call this timestamp "created" (as per Dublin Core) and have its
semantics be that all the attributes and coordinates of the features in the
collection be accurate as of that time?

-- 
Sean Gillies

---

From: Allan Doyle <[email-masked]>
Date: Tue, 8 Dec 2015 10:24:46 -0500

> On Dec 8, 2015, at 1:34 AM, Sean Gillies <[email-masked]> wrote:
> 
> Erik,
> 
> On Sat, Dec 5, 2015 at 6:03 AM, Erik Wilde <[email-masked] <mailto:[email-masked]>> wrote:
> i'd be in favor of using something a bit more webby, https://tools.ietf.org/html/rfc3339 <https://tools.ietf.org/html/rfc3339> probably would be a good choice. it does allow fractional seconds, if such a resolution is necessary.
> 

I agree RFC 3399 is good.

> 
> There's precedence at http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON#Structure <http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON#Structure>. The OSM Overpass API is a popular and very useful service. It's timestamp property is underspecified
> 
> I feel like one timestamp – not necessarily named "timestamp" and since there are a bunch of different common metadata timestamps in use (see Dublin Core, Atom, etc) a generic "timestamp" seems wrong to me – for a feature collection addresses a big issue: data archival. GeoJSON files, if saved for a number of years, will need coordinate adjustment to be used accurately. It seems to me that collections of features are going to be the kind of GeoJSON that gets archived. Single features or geometries may pass in messages, but I don't see them being saved for long. I also don't see a use case for serving collections of features that have different coordinate (and attribute) timestamps: implementers will need to reconcile aggregated features to a uniform coordinate basis if they want to provide any value to end users.
> 
> Could we call this timestamp "created" (as per Dublin Core) and have its semantics be that all the attributes and coordinates of the features in the collection be accurate as of that time?

But what about people who want to leave “created” as when the feature was created and “modified” for the most recent time of accurateness?

I think if we’re going to tell people that any time values SHOULD be specified per RFC 3339, I’m ok with that. If we’re going start adding semantics to timestamp labels, just winging it with a few quick labels would not be giving the topic its due. There are probably differences in communities about what to call timestamps.

	Allan

---

From: Sean Gillies <[email-masked]>
Date: Tue, 8 Dec 2015 10:43:37 -0700

On Tue, Dec 8, 2015 at 8:24 AM, Allan Doyle <[email-masked]>
wrote:

> But what about people who want to leave “created” as when the feature was
> created and “modified” for the most recent time of accurateness?
>
> I think if we’re going to tell people that any time values SHOULD be
> specified per RFC 3339, I’m ok with that. If we’re going start adding
> semantics to timestamp labels, just winging it with a few quick labels
> would not be giving the topic its due. There are probably differences in
> communities about what to call timestamps.
>

Thanks, Allan. I'm going to go with "modified" in a PR that I'll link from
https://github.com/geojson/draft-geojson/issues/80.

-- 
Sean Gillies

---

From: Martin Thomson <[email-masked]>
Date: Wed, 9 Dec 2015 07:45:05 +1100

> https://github.com/geojson/draft-geojson/issues/80.

A caution: "modified" might be misinterpreted to suggest a new
timestamp be applied when any modification is made, even if the
modification does not do anything to tweak coordinates.  That would
affect the archival property Sean describes.

Is "created" so bad here?  An old object that is translated for
contemporary use can be recreated.  Going with "modified" is fine if
you think that you can clearly articulate the prerequisites for
updating the timestamp.

---

From: Sean Gillies <[email-masked]>
Date: Tue, 8 Dec 2015 14:07:07 -0700

> you think that you can clearly articulate the prerequisites for

Good point. Allan, let's give "created" a try?

-- 
Sean Gillies

---

From: Allan Doyle <[email-masked]>
Date: Tue, 8 Dec 2015 18:11:09 -0500

> 
> On Dec 8, 2015, at 4:07 PM, Sean Gillies <[email-masked]> wrote:
> 
> On Tue, Dec 8, 2015 at 1:45 PM, Martin Thomson <[email-masked] <mailto:[email-masked]>> wrote:
> On 9 December 2015 at 04:43, Sean Gillies <[email-masked] <mailto:[email-masked]>> wrote:
> > https://github.com/geojson/draft-geojson/issues/80 <https://github.com/geojson/draft-geojson/issues/80>.
> 
> 

This is illustrative. It doesn’t seem right to simply toss in a name, attach semantics and say it should be used. Time is a huge topic.

My expectation would be that a given community could add their own vocabulary into GeoJSON, not that we should define an incomplete set of the more obvious terms and attach semantics that are easily misinterpreted.

I have a lot of respect for the earnest effort Sean and others are putting in to the extension mechanism. But I also hope that the working group is able to feel justified if the work results in a clean, minimal spec.

As was mentioned on this list several years ago, if you want a full-featured spec, there’s GML. 

> Good point. Allan, let's give "created" a try?

I’m 0 to -1 this. I’m not sure I buy the premise that GeoJSON should try to be an archival format. If someone wants to use it that way, let them figure out how to do it. I’m certain it won’t be by adding a single timestamp called “created”.

	Allan

---

From: Carl Reed <[email-masked]>
Date: Tue, 8 Dec 2015 16:25:36 -0700

All

"created" as an element name is fine with me. However, a couple of
questions/thoughts:

Is this the time the original data was created or the time the data was
created through some process or the time the GeoJSON data was requested or
the time the GeoJSON package was created or the time . . . You catch my
drift. Applications receiving a GeoJSON package might require such clarity
in order to initiate the proper processing for modelling sequence. This is
especially true for content from some sensor sequence and given that most
location enabled data now comes from sensors, this (at least I think) is an
important consideration. By the way, the same questions apply to
time_stamp, shich is the element name used in KML for a time_instant.

I am not trying to be difficult but for simplicity and clarity, a clear
definition of "created" needs to be provided.

Further, in the sensor world and process engineering world (which includes
geospatial process) - and why Java has such a construct - there is the
concept of time instant.

Time Instant is a zero-dimensional geometric primitive that represents an
identifiable position in time.  In a process stream, TM_Instant describes
the time when the result became available, typically when the procedure
associated with creating the observation (data) was completed. Given that

>From JAVA - An instantaneous point on the time-line. Probably the best
place to start with the java.time package is the Instant class. An Instant
represents a point in time (similar to java.util.Date) with nanoseconds
precision (unlike the old Date which has milliseconds precision).

Regards

Carl

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: <[email-masked]>
Date: Tue, 8 Dec 2015 23:54:37 +0000

Allan has a good point here. An arbitrary small subset of temporal properties with ambiguous semantics is unlikely to be helpful. If we could tie down the semantics then better. But I would suggest at least two should be included, so as to raise a flag to users that there are choices to be made. If there is just one, it will be used by everyone for everything, and therefore close to useless.

Simon J D Cox
Research Scientist
Environmental Information Infrastructures
Land and Water
CSIRO

E [email-masked] T +61 3 9545 2365 M +61 403 302 672
Physical: Reception Central, Bayview Avenue, Clayton, Vic 3168
Deliveries: Gate 3, Normanby Road, Clayton, Vic 3168
Postal: Private Bag 10, Clayton South, Vic 3169
http://people.csiro.au/Simon-Cox
http://orcid.org/0000-0002-3884-3420
http://researchgate.net/profile/Simon_Cox3

________________________________
From: GeoJSON on behalf of Allan Doyle
Sent: Tuesday, 8 December 2015 11:11:09 PM
To: [email-masked]
Subject: Re: [Geojson] Timestamps for feature collections

On Dec 8, 2015, at 4:07 PM, Sean Gillies <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Dec 8, 2015 at 1:45 PM, Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

Allan

---

From: Sean Gillies <[email-masked]>
Date: Tue, 8 Dec 2015 17:27:27 -0700

wrote:
> This is illustrative. It doesn’t seem right to simply toss in a name,
attach semantics and say it should be used. Time is a huge topic.
>
> My expectation would be that a given community could add their own
vocabulary into GeoJSON, not that we should define an incomplete set of the
more obvious terms and attach semantics that are easily misinterpreted.
>
> I have a lot of respect for the earnest effort Sean and others are
putting in to the extension mechanism. But I also hope that the working
group is able to feel justified if the work results in a clean, minimal
spec.
> I’m 0 to -1 this. I’m not sure I buy the premise that GeoJSON should try
to be an archival format. If someone wants to use it that way, let them
figure out how to do it. I’m certain it won’t be by adding a single
timestamp called “created”.

I agree that a lot more is required of an archival format. What I'm
proposing is the bare minimum that we can and should do to keep an
incidentally archived GeoJSON file viable for a few years.

The enormity of time is irrelevant here. This timestamp is about
interoperability and very basic provenance. I expect push back on any spec
change, but let's not bring out the slippery slope argument against
everything.

---

From: Allan Doyle <[email-masked]>
Date: Tue, 8 Dec 2015 19:40:42 -0500

> On Dec 8, 2015, at 7:27 PM, Sean Gillies <[email-masked]> wrote:
> 
> On Dec 8, 2015 4:11 PM, "Allan Doyle" <[email-masked] <mailto:[email-masked]>> wrote:
> >>
> >>
> >> On Dec 8, 2015, at 4:07 PM, Sean Gillies <[email-masked] <mailto:[email-masked]>> wrote:
> >>
> I agree that a lot more is required of an archival format. What I'm proposing is the bare minimum that we can and should do to keep an incidentally archived GeoJSON file viable for a few years.
> 
> The enormity of time is irrelevant here. This timestamp is about interoperability and very basic provenance. I expect push back on any spec change, but let's not bring out the slippery slope argument against everything.
> 
> 
If you provide the illusion of provenance, people will use it without thinking. No sense in making the serious people wonder what we were thinking while the dilettantes feel they are doing the right thing by adding a single timestamp.

	Allan

---

From: Howard Butler <[email-masked]>
Date: Tue, 8 Dec 2015 20:34:12 -0600

> On Dec 8, 2015, at 12:34 AM, Sean Gillies <[email-masked]> wrote:
> 
> I feel like one timestamp – not necessarily named "timestamp" and since there are a bunch of different common metadata timestamps in use (see Dublin Core, Atom, etc) a generic "timestamp" seems wrong to me – for a feature collection addresses a big issue: data archival. GeoJSON files, if saved for a number of years, will need coordinate adjustment to be used accurately. It seems to me that collections of features are going to be the kind of GeoJSON that gets archived. 

I don't see the geodesists clamoring for a GeoJSON that support time-precise coordinate precision, and using GeoJSON for that purpose seems to me to be using the wrong tool. Additionally, I think the spec is talking out both sides of its mouth to say "WGS84 most of the time is good enough for general interop, but be sure to precisely define its time epoch". 

Are future implementations supposed to make sure they apply local time corrections to data if there's timestamps? I know the language says MAY, but having only the timestamp as described in the pull request [1] is insufficient to make the transformation. An implementation that *doesn't* do it is seen as incomplete in some way. We need a lot more description than a point in time, and I think including this language sets people up for failure and frustration. 

> Could we call this timestamp "created" (as per Dublin Core) and have its semantics be that all the attributes and coordinates of the features in the collection be accurate as of that time?

Accurate to what? The most commonly used time epoch at that time? The latest one given that time? The high-precision local one? The less precise but globally available one? It's the same swamp as SRS, but worse due to the fact that there aren't as many conventions as SRSs to paper over the challenge of naming, specifying, and transforming between everything. 

If the semantics of "created" or "timestamp" or whatever really are "when this file was created" and not trying to say something about the relative time position of the coordinates, it is maybe useful sugar. However, it is scope creep to tell future GeoJSON implementors that they MAY be responsible for weakly specified coordinate time corrections, but not give them all the supporting information they need to actually do it. 

Howard

[1] https://github.com/geojson/draft-geojson/commit/2a7165d0bf844de9b412cc0827a181ebd02b1b8b#diff-1261ada59649c3fc43efdedc43f6173bR280

---

From: David Neufeld - NOAA Affiliate <[email-masked]>
Date: Tue, 8 Dec 2015 16:35:05 -1000

Some good definitions of terms below regarding created vs. modified,
these have been fairly well accepted among netcdf communities.  Added
bonus is that there are mappings of these terms to ISO 19115.

date_created - The date on which this version of the data was created.
(Modification of values implies a new version, hence this would be
assigned the date of the most recent values modification.) Metadata
changes are not considered when assigning the date_created. The ISO
8601:2004 extended date format is recommended, as described in the
Attribute Content Guidance section.

date_modified - The date on which the data was last modified. Note
that this applies just to the data, not the metadata. The ISO
8601:2004 extended date format is recommended, as described in the
Attributes Content Guidance section.

See:
http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_1-3#Recommended

Dave

-- 
David Neufeld
Senior Associate Scientist
CIRES
Software Engineering Support Branch
Data Stewardship Division
NOAA National Centers for Environmental Information (NCEI)
Ph. 303-497-6507

---

From: Martin Daly <[email-masked]>
Date: Wed, 9 Dec 2015 09:54:03 +0000

> The enormity of time is irrelevant here. This timestamp is about interoperability and very basic provenance.
> I expect push back on any spec change, but let's not bring out the slippery slope argument against everything.

I agree that time doesn't have special enormity here, but disagree in that I fail to see how that makes it more special than the other 14 entries in the Dublin Core Metadata Element Set.

It's all (ie some words about how to use *all* of the DC elements, including the fact that the cardinality is "0..*" for them all) or nothing for me. You won't be surprised to hear that I favour the latter over the former.

Martin

---

From: Sean Gillies <[email-masked]>
Date: Wed, 16 Dec 2015 16:33:10 -0700

Howard and all,

Sorry for the delay in responding. I was overwhelmed by other duties last
week, but have recovered.

On Tue, Dec 8, 2015 at 7:34 PM, Howard Butler <[email-masked]> wrote:

> kind of GeoJSON that gets archived.
>
> I don't see the geodesists clamoring for a GeoJSON that support
> time-precise coordinate precision, and using GeoJSON for that purpose seems
> to me to be using the wrong tool. Additionally, I think the spec is talking
> out both sides of its mouth to say "WGS84 most of the time is good enough
> for general interop, but be sure to precisely define its time epoch".
>

Yes, I think this is true.

>
> Are future implementations supposed to make sure they apply local time
> corrections to data if there's timestamps? I know the language says MAY,
> but having only the timestamp as described in the pull request [1] is
> insufficient to make the transformation. An implementation that *doesn't*
> do it is seen as incomplete in some way. We need a lot more description
> than a point in time, and I think including this language sets people up
> for failure and frustration.
>
> Accurate to what? The most commonly used time epoch at that time? The
> latest one given that time? The high-precision local one? The less precise
> but globally available one? It's the same swamp as SRS, but worse due to
> the fact that there aren't as many conventions as SRSs to paper over the
> challenge of naming, specifying, and transforming between everything.
>
> If the semantics of "created" or "timestamp" or whatever really are "when
> this file was created" and not trying to say something about the relative
> time position of the coordinates, it is maybe useful sugar. However, it is
> scope creep to tell future GeoJSON implementors that they MAY be
> responsible for weakly specified coordinate time corrections, but not give
> them all the supporting information they need to actually do it.
>

I don't see how this is scope creep. But after doing a bunch of reading on
epochs I do see how a single timestamp doesn't help those who are looking
for centimeter accuracy and those who are satisfied with less accuracy are
not asking for this feature.

-- 
Sean Gillies