Subject: Re: [Tzdist] RESTful URL design

From: Daniel Migault <[email-masked]>
Date: Fri, 10 Oct 2014 13:22:04 +0200

Dear Friends and Colleagues,

It looks to me that remarks from Stephen should be considered, and at least
discussed. One of the reason is that it may turn out implementation to be
much easier to implement, and more inter-operable. It may change a bit the
current architecture of the service, but I do not believe that will be hard
to overcome.

I would be happy you share your opinion/suggestion with the WG this week.

BR,
Daniel

On Mon, Sep 29, 2014 at 11:57 PM, Stephen Colebourne <[email-masked]>
wrote:

> Daniel, Cyrus,
> I do not overly care whether the phrase "RESTful" has meaning or not
> as far as the spec is concerned. However, I do believe that there is
> enough best practice out there to clearly say what is good and what is
> not good in this field, focussed around the use of the features of
> HTTP, including
> - using GET/POST/PUT/DELETE (not relevant here)
> - using HTTP headers for formats and languages where appropriate
> - actions as resources (based on GET/POST/PUT/DELETE principles)
> - queries for searches
>
> Here is a high up Google link that I think captures the design
> principles pretty well
> http://codeplanet.io/principles-good-restful-api-design/
>
> It is with this best practice that I find "?action=" to be just plain
> wrong, especially when there is a clear and simple set of URL based
> resources that can expose the functionality. (ie. this is not a
> particularly difficult space in terms of resource design)
>
> I'm happy to explain the detail of my original post, however I hope it
> is fairly self-explanatory.
>
> Stephen
>
>
> On 29 September 2014 20:29, Daniel Migault <[email-masked]> wrote:
> > Hi,
> >
> > I agree with Cyrus, that the debate should not be whether we meet a
> RESTful
> > architecture or not. Given that, we would like to get feed back from the
> WG
> > on the URL format, we should adopt.
> >
> > BR,
> > Daniel
> >
> > On Mon, Sep 29, 2014 at 7:32 PM, Tobias Conradi <[email-masked]>
> > wrote:
> >>
> >> On Mon, Sep 29, 2014 at 6:07 PM, Cyrus Daboo <[email-masked]> wrote:
> >> > I don't want to get into an argument about best practices for web api
> >> > design
> >> > because it is trivial to find counter-arguments for many design
> choices,
> >> > and
> >> > I also don't consider myself an expert in that area either - at this
> >> > point I
> >> > would much rather defer to others on this issue so I do want to hear
> >> > from
> >> > the REST of the WG ...
> >>
> >> 1)
> >> Stephen didn't address the issues pointed out in the first reply to
> >> his proposal.
> >>
> >> http://www.ietf.org/mail-archive/web/tzdist/current/msg00674.html
> >>
> >> So these are either considered not to be issues or the proposal needs
> >> fixing.
> >>
> >> Also the reasoning that avoiding different parameter orders which may
> >> prevent caching are solved by his proposal has been shown to be
> >> flawed.
> >>
> >> 2)
> >> a) So there is a proposal using
> >>
> >> http://example.com/tzdds/?param1=value1&param2=value2
> >>
> >> Generation well supported by HTML forms.
> >>
> >> b)
> >> and Stephen's that replaces some similar to this:
> >>
> >> http://example.com/tzdds/param1/value1?param2=value2
> >>
> >> Then TZD servers need URLs parsing. For humans it might be hard to see
> >> whether a string that could be either is a parameter name or a value
> >>
> >> e.g.:
> >> http://example.com/tzdds/publisher/atzdb/version/201/id/de/lang/id/
> >>
> >> How would values that contain a "/" be written in such a path?
> >>
> >>
> >> --
> >> Tobias Conradi
> >> Rheinsberger Str. 18
> >> 10115 Berlin
> >> Germany
> >>
> >> http://tobiasconradi.com
> >>
> >> _______________________________________________
> >> Tzdist mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/tzdist
> >
> >
> >
> >
> > --
> > Daniel Migault
> > Orange Labs -- Security
> > +33 6 70 72 69 58
>
> _______________________________________________
> Tzdist mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tzdist
>

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Tobias Conradi <[email-masked]>
Date: Fri, 10 Oct 2014 17:23:07 +0200

On Fri, Oct 10, 2014 at 1:22 PM, Daniel Migault <[email-masked]> wrote:
> It looks to me that remarks from Stephen should be considered
They have been considered, flaws have been found, he didn't bother to clarify.

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Paul Eggert <[email-masked]>
Date: Fri, 10 Oct 2014 08:48:17 -0700

> I would be happy you share your opinion/suggestion with the WG this week.

I agree with Stephen Colebourne's suggestions about preferring URL segments to 
query parameters for arguments that are required and hierarchical.  His redesign 
along those lines looks good to me, though I admit I have not reviewed its 
details carefully (just as I have not carefully reviewed the second half of the 
draft).  It all seemed so obvious that I didn't feel the need to comment until 
now.  As I recall, Tobias Conradi had some quibbles (e.g., whether to use "--" 
to designate a time range) but nothing that affected the soundness of the basic 
idea.

---

From: Tobias Conradi <[email-masked]>
Date: Fri, 10 Oct 2014 18:20:31 +0200

On Fri, Oct 10, 2014 at 5:48 PM, Paul Eggert <[email-masked]> wrote:
> I agree with Stephen Colebourne's suggestions about preferring URL segments
> to query parameters for arguments that are required and hierarchical.
I searched
https://tools.ietf.org/html/draft-ietf-tzdist-service-01
for the string "hier" - it does not exist.

I searched Stephen Colebourne first email in this thread
http://www.ietf.org/mail-archive/web/tzdist/current/msg00672.html
for the string "hier" - it does not exist.

> As I recall, Tobias Conradi had some quibbles
> (e.g., whether to use "--" to designate a time range) but nothing that
> affected the soundness of the basic idea.
If the basic idea is to design parameters (plural name vs singular
name) and values (time range vs two time points) different, depending
on whether they are part of "required and hierarchical" increases the
inconsistencies in the document.

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Stephen Colebourne <[email-masked]>
Date: Fri, 10 Oct 2014 17:36:47 +0100

To expand a little on why I ended up with the specific proposal I did.

The basic principle I adopted was to have a "list" URL and an "item"
URL at each level. To give an example away from tzdb, you might have

GET /flights
GET /flights/BA123

The "list" URL provides access to all the data about its concept. In
database terms it can be thought of as "SELECT * FROM flights".

The "item" URL provides access to a single identifier element within
the list. In database terms it can be thought of as "SELECT * FROM
flights WHERE id = BA123" where there is guaranteed to be one result
(or zero if not found).

This approach allows further expansion when searches are considered,
and this is where ? queries apply.

GET /flights?airline=BA

This is a filter that applies to the whole list of flights. In
database terms it can be thought of as "SELECT * FROM flights WHERE
airline = BA" where there can be many results, but generally less than
the complete set of flights.

When mapping tzdb to this approach, I used "ids" instead of "flights"
and the actual id "America/New_York" as the id. Because TZDB ids have
a / in their name, I replaced it with %2F in the URL. This isn't
absolutely required, but makes it easier to parse other URLs such as
the /
transitions subresource.

The details of the format of the truncate parameter looked wrong to
me, which is why I mentioned it. Having truncateStart and truncateEnd
offers more power and seems more useful than combining them in one
parameter, whether that is ISO-8601 formatted or not.

As I've mentioned separately, there is a case to have the provider and
version also in the URL:

GET /publishers/{publisher}/versions/{version}/ids/{id}

where the {version} "latest" corresponds to whatever the latest data is.

Such an approach adds more complexity, but it does model the domain.
One possibility is to say that each of these are equivalent:

GET /ids/{id}
GET /publishers/IANA/ids/{id}
GET /publishers/IANA/versions/latest/ids/{id}

The notion of multiple URLs pointing to the same actual data is common
in this space, and useful here.

As an option to the original proposal, here is a variation for the
distinction between rules and expanded transitions:

original proposal:
GET /ids/{id}
GET /ids/{id}/transitions

alternate proposal:
GET /rules/{id}
GET /transitions/{id}

Either of these works for me, the first gives a more canonical URL for
the id, the latter is perhaps simpler to understand (and does not
require encoding the / in America/New_York).

Stephen

On 10 October 2014 16:48, Paul Eggert <[email-masked]> wrote:
> to query parameters for arguments that are required and hierarchical.  His
> redesign along those lines looks good to me, though I admit I have not
> reviewed its details carefully (just as I have not carefully reviewed the
> second half of the draft).  It all seemed so obvious that I didn't feel the
> need to comment until now.  As I recall, Tobias Conradi had some quibbles

---

From: Tobias Conradi <[email-masked]>
Date: Fri, 10 Oct 2014 19:11:07 +0200

> flights WHERE id = BA123" where there is guaranteed to be one result
That is a list too. Both queries go to the same table and selected are
all fields ("*").

> and this is where ? queries apply.
Why treat "airline" different from "id"?

GET /flights?id=BA123
GET /flights/BA123 (Stephen)

GET /flights?airline=BA (Stephen)
GET /flights/BA

> the complete set of flights.
Same for id=BA123

> a / in their name,
AFAIK that is only true for the IANA TZDB

> GET /publishers/{publisher}/versions/{version}/ids/{id}
Again, why plural, when in the SQL query and in the tzdist-service-01
it would be "id=<id>"?

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Lester Caine <[email-masked]>
Date: Fri, 10 Oct 2014 19:59:41 +0100

> That is a list too. Both queries go to the same table and selected are

The difference is in what is returned ...
Returning a single packet of data with all of the flight details is
different to producing a list of flights meeting the filter requirement.
Yes you could return ALL of the data full packets, but that can become
un-managable, so the default would be a list of id's only.

Filter can be a combination of elements such as BA flights to Europe
between X and Y which is not something one would encode into the URL.

> GET /flights/BA123 (Stephen)
In most systems these would be equivalent ...

> GET /flights?airline=BA (Stephen)
That is more difficult ... one may not know what the 'BA' refers to and
have to find what type of id it is. Especially for filter elemennts like
from and to dates.

>> the complete set of flights.
See above ...
id returns a detailed data set for a flight, airline just returnes a
list of id's

> AFAIK that is only true for the IANA TZDB
Spaces in the names are just as bad, but personally t think the / is a
problem in general.

> Again, why plural, when in the SQL query and in the tzdist-service-01
Personally ...

GET /publishers returns list of available publishers ( so plural )
But rather than GET /publishers/{publisher} I'd go with GET /{publisher}

Then same argument for versions with latest as another keyword

Then ids pulls a list of supported ids.
But at this level one can add a filter for '?continent=xxx' to limit the
list.

So we get
GET /{publisher}/{version}/{id} for a full set of data and  ...
GET /{publisher}/{version}/{id}?fromdate=xxx&todate= for an expanded
data set
I would go as far as
GET /{publisher}/{version}/{id}?timestamp=xxxyyy would return the
correct offset for that UTC time which is why providing the from and to
dates as URL elements would not work.

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

---

From: Tobias Conradi <[email-masked]>
Date: Fri, 10 Oct 2014 21:34:27 +0200

> The difference is in what is returned ...
As stated
>> selected are
>> all fields ("*")

> have to find what type of id it is. Especially for filter elemennts like
One may also not know what BA123 stands for. That is why in SQL and in
the ?-URLs one mentions the field name.

> id returns a detailed data set for a flight, airline just returnes a
See above, (selected are all fields ("*")).

> Spaces in the names are just as bad, but personally t think the / is a
Yes, "_" also seems only to exist in TZIDs of the IANA TZDB. Same for
"." and "-".

There was a proposal to at least disallow the latter two

Tobias Conradi 2012-05-24 10:45:49 UTC
Proposal to deprecate "." and "-" in zone names
http://mm.icann.org/pipermail/tz/2012-May/017936.html

but not even that minimal change has been implemented.

> Then ids pulls a list of supported ids.
That would make it consistent with SQL and the ?-URL parameters.

> dates as URL elements would not work.
Why not?

GET /{publisher}/{version}/{id}/20140101T012345

GET /{publisher}/{version}/{id}/2012-01-01--2014-05-06

But returning different types of data for same base
(/{publisher}/{version}/{id}/) looks like creating future problems.

Some people may expect to get the same data for

GET /{publisher}/{version}/{id}/20140101T012345 (~range of length 1s)

GET /{publisher}/{version}/{id}/20140101T012345--GET
/{publisher}/{version}/{id}/20140101T012346  (~range of length 2s)

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Tobias Conradi <[email-masked]>
Date: Fri, 10 Oct 2014 21:36:08 +0200

> /{publisher}/{version}/{id}/20140101T012346  (~range of length 2s)

Correcting my comment:

GET /{publisher}/{version}/{id}/20140101T012345--20140101T012346
(~range of length 2s)

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 24 Oct 2014 08:35:56 +0100

> But rather than GET /publishers/{publisher} I'd go with GET /{publisher}

NO!

If you do that, then you need to maintain a list of strings (like
"publishers") that can't be the name of a publisher, with all the pain that
that implies. Better to make it consistent whether the first component is
a command or not.

So either require the /publishers/, or have a rule that publisher names
must contain a hyphen and command names must not, or something like that.

-- 
Clive D.W. Feather          | If you lie to the compiler,
Email: [email-masked]     | it will get its revenge.
Web: http://www.davros.org  |   - Henry Spencer
Mobile: +44 7973 377646

---

From: Lester Caine <[email-masked]>
Date: Fri, 24 Oct 2014 09:23:27 +0100

> must contain a hyphen and command names must not, or something like that.

Isn't that the debate? Although I may be missing a command.

GET /?action=get&publisher=x&version=y&id=z&truncate=a,b

vs

GET /get/x/y/z/?truncate=a,b

Required values form the url while optional elements remain as variable
data.

But I don't see that the cammands are needed if names like 'default' and
'latest' provide the only special cases.

So accessing a single source server
GET /latest/  works for me to get either a 'no change' via ETag or a
list of changed TZID's
while
GET /Ver2013d/ gives the list of ID's for that version.
No need for commands as such.

What messes this up a bit IS that current tz id's have slashes in, so
have to be the last url element?

---

From: Stephen Colebourne <[email-masked]>
Date: Fri, 24 Oct 2014 11:43:33 +0100

> Isn't that the debate? Although I may be missing a command.

So, as Clive says, it is important to have what are essentially "known
keywords" in the URL, such as "zones" or "ids" or "publishers". One
purpose of the keyword URL segment is to clearly identify what the
next segment means, and that next segment is generally the identifier
of a single entity of the type described by the keyword.

Thus a good format is:
GET /versions/{versionId}/zones/{zoneId}
and a not so good one is:
GET /{versionId}/{zoneId}

> Required values form the url while optional elements remain as variable

My analysis suggested that only the ID is really required with the
other parts as search criteria on lists or filtering of objects.

> 'latest' provide the only special cases.

Its about future change. While you can define some keywords like
'default' in spec v1, you cannot add any more keywords in spec v2
because they might clash with a pre-existing versionId or zoneId.

> have to be the last url element?

This is the biggest choice in URL design in this case. Do we insist on
the ID being at the end, or do we URL encode slashed in IDs?

Stephen