Subject: [Trans] Directory instead of .well-known for URL structure

From: Jacob Hoffman-Andrews <[email-masked]>
Date: Thu, 20 Jun 2019 15:33:26 -0700

The latest draft adopts a /.well-known/ path for CT as a way to get 
around BCP 190 (URI Design and Ownership: 
https://tools.ietf.org/html/bcp190#section-3).

Personally I think BCP 190 makes it needlessly painful to specify 
HTTP-based APIs using techniques that are very common among 
practitioners. However, given that it is still considered best practice 
for IETF documents, I propose that CT should use a different workaround, 
one used very successfully by ACME: Directory URLs.

In ACME, a CA defines one URL, a Directory, which provides a JSON 
dictionary showing where to find the other URLs. For instance:

$ curl https://acme-v01.api.letsencrypt.org/directory
{
   "fUKGu3pAntw": 
"https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
   "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
   "meta": {
     "caaIdentities": [
       "letsencrypt.org"
     ],
     "terms-of-service": 
"https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
     "website": "https://letsencrypt.org"
   },
   "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
   "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
   "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
   "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}

To use an ACME CA, rather than configuring a base URL, an ACME client 
configures a directory URL. All other URLs can be found by looking up 
fields in the directory.

Adding a directory to rfc6962-bis provides the most minimally invasive 
way to satisfy BCP 190. Well-known URLs aren't really meant for this 
type of application. To quote RFC 5785 
(https://tools.ietf.org/html/rfc5785#section-1.1):

    1.1.  Appropriate Use of Well-Known URIs

        There are a number of possible ways that applications could use
    Well-
        known URIs.  However, in keeping with the Architecture of the World-
        Wide Web [W3C.REC-webarch-20041215], *well-known URIs are not
    intended**
    **   for general information retrieval or establishment of large URI**
    **   namespaces* on the Web.  Rather, they are designed to facilitate
        discovery of information on a site when it isn't practical to use
        other mechanisms; for example, when discovering policy that needs to
        be evaluated before a resource is accessed, or when using multiple
        round-trips is judged detrimental to performance.

        As such, the well-known URI space was created with the expectation
        that it will be used to make site-wide policy information and other
        metadata available directly (if sufficiently concise), or provide
        references to other URIs that provide such metadata.

(emphasis mine)

Beyond that, requiring log operators to organize their log under 
/.well-known/ would pose unnecessary difficulties. Service providers 
intentionally restrict the /.well-known/ URL space. Depending on 
deployment model, log operators might have to bypass such restrictions 
or get special exemptions.

Thanks,
Jacob

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 24 Jun 2019 09:28:26 +1000

I agree with Jacob here. As I have expressed in the past, I believe that this is a better design than the well-known prefix.

On Fri, Jun 21, 2019, at 08:33, Jacob Hoffman-Andrews wrote:
>    "revoke-cert": 
> "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
> }
> 
> _______________________________________________
> Trans mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/trans
>

---

From: Melinda Shore <[email-masked]>
Date: Sun, 23 Jun 2019 16:05:27 -0800

On 6/23/19 3:28 PM, Martin Thomson wrote:
> I agree with Jacob here. As I have expressed in the past, I believe
> that this is a better design than the well-known prefix.

>> practitioners. However, given that it is still considered best
>> practice for IETF documents, I propose that CT should use a
>> different workaround, one used very successfully by ACME: Directory
>> URLs.

I have a fairly profound dislike for BCP 190, to be honest,
and am in agreement with the proposal.

Melinda

-- 
Melinda Shore
[email-masked]

Software longa, hardware brevis

---

From: Rob Stradling <[email-masked]>
Date: Mon, 24 Jun 2019 14:15:47 +0000

This sounds unanimous.  :-)

I'll go ahead and make this change, and I'll cancel the registration 
process for "ct" as a .well-known URI suffix.

-- 
Rob Stradling
Senior Research & Development Scientist
Email: [email-masked]
Bradford, UK
Office: +441274024707
Sectigo Limited

This message and any files associated with it may contain legally 
privileged, confidential, or proprietary information. If you are not the 
intended recipient, you are not permitted to use, copy, or forward it, 
in whole or in part without the express consent of the sender. Please 
notify the sender by reply email, disregard the foregoing messages, and 
delete it immediately.

---

From: "Manger, James" <[email-masked]>
Date: Tue, 25 Jun 2019 00:42:56 +0000

Would it be sufficient to provide a URL template for each log, instead of a directory URL?
Possible text:

  The address of a log is defined by a URL template [RFC6570] that MUST include
  the following variables: action, first, second, hash, start, end.

  'action' identifies the client message, such as 'submit-entry' or 'get-sth'.
  'first' and 'second' are tree sizes. 'hash' is a base64-encoded v2 leaf hash.
  'start' and 'end' are 0-based entry indicies.

  Example: https://log.example.net/ct/v2/red/{action}{?first}{?second}{?hash}{?start}{?end}

--
James Manger

-----Original Message-----
From: Trans <[email-masked]> On Behalf Of Rob Stradling
Sent: Tuesday, 25 June 2019 12:16 AM
To: Melinda Shore <[email-masked]>; [email-masked]
Subject: Re: [Trans] Directory instead of .well-known for URL structure

I'll go ahead and make this change, and I'll cancel the registration process for "ct" as a .well-known URI suffix.

On 24/06/2019 01:05, Melinda Shore wrote:
> I have a fairly profound dislike for BCP 190, to be honest, and am in 
> agreement with the proposal.
> 
> Melinda
> 

This message and any files associated with it may contain legally privileged, confidential, or proprietary information. If you are not the intended recipient, you are not permitted to use, copy, or forward it, in whole or in part without the express consent of the sender. Please notify the sender by reply email, disregard the foregoing messages, and delete it immediately.
_______________________________________________
Trans mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/trans

---

From: Eran Messeri <[email-masked]>
Date: Tue, 25 Jun 2019 11:12:04 +0100

Can you clarify if you mean a single template to cover all possible actions
or separate one for each action?

The former would, I think, be tricky to define as different actions have
different parameters.
The latter is what we had, and is not in compliance with BCP190.

Either way, the issue here is compliance with BCP190, so if your proposal
is in compliance with BCP190 it's worth highlighting why (as it's not clear
to me personally).

Thanks,
Eran

On Tue, Jun 25, 2019 at 1:45 AM Manger, James <
[email-masked]> wrote:

>   Example:
> https://log.example.net/ct/v2/red/{action}{?first}{?second}{?hash}{?start}{?end}
>
> --
> James Manger
>
> intended recipient, you are not permitted to use, copy, or forward it, in
> whole or in part without the express consent of the sender. Please notify
> the sender by reply email, disregard the foregoing messages, and delete it
> immediately.

---

From: "Manger, James" <[email-masked]>
Date: Wed, 26 Jun 2019 03:00:57 +0000

I meant a single template to cover all possible actions.

It is easy to define a single template for the current API for a log that uses the query string for the parameters (the “obvious” design), as the example shows:
https://log.example.net/ct/v2/red/{action}{?first}{?second}{?hash}{?start}{?end}<https://log.example.net/ct/v2/red/%7Baction%7D%7B?first%7D%7B?second%7D%7B?hash%7D%7B?start%7D%7B?end%7D>

It is still easy to write a single template that uses path segments for parameters, or a mix of path segments and the query string. Example:
https://log.example.net/ct/v2/red/{action}{/hash}{?first}{?second}{?start}{?end}<https://log.example.net/ct/v2/red/%7baction%7d%7b/hash%7d%7b?first%7d%7b?second%7d%7b?start%7d%7b?end%7d>

Some funky URL designs will not be possible with a single template (eg start & end in a single path segment separated by a hyphen, only for those actions needing those parameters). Other URL designs will work but be a bit messy (eg ?h={hash} to use “h” not “hash” as the query parameter name results in an extra empty param on actions that don’t take a hash).

The RFC6962 and draft-ietf-trans-rfc6962-bis templates don’t comply with BCP190 as they require:
1. An app-specific path (eg /ct/v1), even though it is only after a log-chosen prefix;
2. App-specific ‘action’ values must go into a path segment (eg /get-sth-consistency); and
3. App-specific parameters must go into the query string (eg ?first=20&second=30).

A single template allows a log to put the app-specific variables all in the query string, all in the path, or a mix. That is sufficient for all practical purposes, with minimal overhead. That is enough flexibility to satisfy BCP190 in my opinion, particularly given that every log will use the “obvious” design anyway.

A directory with a separate template for every action is a more pure design: it gives a log total flexibility in URL design; and is easily extendable if the log API gets far more complex in future. At the cost of extra calls & code to read, parse, and process the directory.

--
James Manger

From: Trans <[email-masked]> On Behalf Of Eran Messeri
Sent: Tuesday, 25 June 2019 8:12 PM
To: Manger, James <[email-masked]>
Cc: Rob Stradling <[email-masked]>; [email-masked]
Subject: Re: [Trans] Directory instead of .well-known for URL structure

Can you clarify if you mean a single template to cover all possible actions or separate one for each action?

The former would, I think, be tricky to define as different actions have different parameters.
The latter is what we had, and is not in compliance with BCP190.

Either way, the issue here is compliance with BCP190, so if your proposal is in compliance with BCP190 it's worth highlighting why (as it's not clear to me personally).

Thanks,
Eran

On Tue, Jun 25, 2019 at 1:45 AM Manger, James <[email-masked]<mailto:[email-masked]>> wrote:
Would it be sufficient to provide a URL template for each log, instead of a directory URL?
Possible text:

  Example: https://log.example.net/ct/v2/red/{action}{?first}{?second}{?hash}{?start}{?end}<https://log.example.net/ct/v2/red/%7Baction%7D%7B?first%7D%7B?second%7D%7B?hash%7D%7B?start%7D%7B?end%7D>

--
James Manger

-----Original Message-----
From: Trans <[email-masked]<mailto:[email-masked]>> On Behalf Of Rob Stradling
Sent: Tuesday, 25 June 2019 12:16 AM
To: Melinda Shore <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Trans] Directory instead of .well-known for URL structure

--
Rob Stradling
Senior Research & Development Scientist
Email: [email-masked]<mailto:[email-masked]>
Bradford, UK
Office: +441274024707
Sectigo Limited

This message and any files associated with it may contain legally privileged, confidential, or proprietary information. If you are not the intended recipient, you are not permitted to use, copy, or forward it, in whole or in part without the express consent of the sender. Please notify the sender by reply email, disregard the foregoing messages, and delete it immediately.
_______________________________________________
Trans mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/trans

_______________________________________________
Trans mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/trans

---

From: Rob Stradling <[email-masked]>
Date: Wed, 26 Jun 2019 10:19:38 +0000

On 26/06/2019 04:00, Manger, James wrote:
<snip>
> A directory with a separate template for every action is a more pure 
> design: it gives a log total flexibility in URL design; and is easily 
> extendable if the log API gets far more complex in future.

I prefer the directory approach (vs URI templates), for these very reasons.

Some logs might want to support additional API endpoints (e.g., there 
was a "get-sths" proposal a while ago - 
https://github.com/google/certificate-transparency-rfcs/pull/200).

And some "logs" might want to support a subset of the standard log API 
(e.g., imagine a log submission proxy that only offers a "submit-entry" 
endpoint that, when called, forwards the submit-entry request to 
multiple real logs).

> At the cost of extra calls & code to read, parse, and process the directory.

Agreed, but I think it's worth this cost.

> --
> 
> James Manger
> 
> *From:*Trans <[email-masked]> *On Behalf Of *Eran Messeri
> *Sent:* Tuesday, 25 June 2019 8:12 PM
> *To:* Manger, James <[email-masked]>
> *Cc:* Rob Stradling <[email-masked]>; [email-masked]
> *Subject:* Re: [Trans] Directory instead of .well-known for URL structure
> 
> Either way, the issue here is compliance with BCP190, so if your 
> proposal is in compliance with BCP190 it's worth highlighting why (as 
> it's not clear to me personally).
> 
> Thanks,
> 
> Eran
> 
> On Tue, Jun 25, 2019 at 1:45 AM Manger, James 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     <https://log.example.net/ct/v2/red/%7Baction%7D%7B?first%7D%7B?second%7D%7B?hash%7D%7B?start%7D%7B?end%7D>
> 
>     --
>     James Manger
> 
>     From: Trans <[email-masked] <mailto:[email-masked]>>
>     On Behalf Of Rob Stradling
>     To: Melinda Shore <[email-masked]
>     <mailto:[email-masked]>>; [email-masked]
>     <mailto:[email-masked]>
>     Email: [email-masked] <mailto:[email-masked]>
>     Bradford, UK
>     Office: +441274024707
>     Sectigo Limited
> 
>     privileged, confidential, or proprietary information. If you are not
>     the intended recipient, you are not permitted to use, copy, or
>     forward it, in whole or in part without the express consent of the
>     sender. Please notify the sender by reply email, disregard the
>     foregoing messages, and delete it immediately.
>     [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>