Subject: [rfc-i] rfcXXXX.json

From: Russ Housley <[email-masked]>
Date: Mon, 23 Sep 2019 12:51:01 -0400

The rfcXXXX.json files were recently added to the rsync.  However, it appears that some automated process touches all of the files very frequently.  Can we please change that process so that the file in the rsync source directory only gets changed if there is actually a change to the file.

Thanks,
  Russ

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Sep 2019 19:00:21 +0200

On Sep 23, 2019, at 18:51, Russ Housley <[email-masked]> wrote:
> 
> The rfcXXXX.json files were recently added to the rsync.  

Nice.  (Is there documentation about the format?)

> However, it appears that some automated process touches all of the files very frequently.  Can we please change that process so that the file in the rsync source directory only gets changed if there is actually a change to the file.

+1.  

(This should be standard practice for other rsyncable directories, too, such as the various generated files for bibxml etc.)

As in

if cmp $file.new $file.json
then rm $file.new # Discard new version if file hasn’t changed
else mv $file.new $file.json
fi

(Replace $file.new and $file.json by build/$file and export/$file etc. as needed.)

Grüße, Carsten

---

From: Michael StJohns <[email-masked]>
Date: Mon, 23 Sep 2019 13:21:46 -0400

> (This should be standard practice for other rsyncable directories, too, such as the various generated files for bibxml etc.)

Short term there's always rsync --checksum.  Mike

---

From: Julian Reschke <[email-masked]>
Date: Tue, 24 Sep 2019 11:20:21 +0200

On 23.09.2019 19:00, Carsten Bormann wrote:
> ...

Or, alternatively, as the format isn't that hard to read: is this
considered stable and available for use?

Best regards, Julian

---

From: Alice Russo <[email-masked]>
Date: Tue, 24 Sep 2019 16:48:54 -0700

Hi all,

Russ,
Thanks for mentioning this. We agree with the sentiment. As an initial approach, we were regenerating the files every 24 hours, just to ensure fast deployment. My understanding is that this triggers datestamp updates in rsync, resulting in the increased file counts you're seeing.  We are already looking at several mechanisms to switch to on-demand updates, and hope to have those implemented - and this process improved - as quickly as possible.

Also, just FYI, that 2 rsync modules have been added to make it easier to choose to get (or not get) the JSON files; details are on https://www.rfc-editor.org/retrieve/rsync/.

Carsten,
We plan to post some basic documentation. For now, regarding this new resource - JSON files of RFC metadata:
	* One JSON file per RFC was made available 11 September 2019.
	* The goal is to provide essential metadata about each RFC.
        * Each file contains data similar to an entry in rfc-index.xml.
	* Sample file: https://www.rfc-editor.org/rfc/rfc5234.json

Julian,
They are available for use and fairly stable after an update that was completed today. And, we may continue to iron things out. Feedback welcome. 

Thanks,

Alice Russo
RFC Production Center

> On Sep 23, 2019, at 9:51 AM, Russ Housley <[email-masked]> wrote:
>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Sep 2019 06:18:19 +0200

> They are available for use and fairly stable after an update that was completed today. And, we may continue to iron things out. Feedback welcome.

Thanks so far. Will have a look.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Sep 2019 09:57:23 +0200

On 25.09.2019 06:18, Julian Reschke wrote:
> ...
>> Julian,
>> They are available for use and fairly stable after an update that was
>> completed today. And, we may continue to iron things out. Feedback
>> welcome.
>> ...
>

OK, had a look, and did a test implementation in the JS used by
rfc2629.xslt (which currently goes to tools.ietf.org for that kind of
information).

I'm currently looking at <https://www.rfc-editor.org/rfc/rfc2617.json>
as example.

Question: why is the data coming back as JSON array? It currently
contains one object, but what would it mean if it contains zero, or more
than one?

Question: I note that apparently a four-digit RFC number is expected in
the URL. That probably should be documented (with a plan for RFC >=
10000 of course).

Bug: when the RFC is not updated by anything, the "updated_by" member
comes back as array with *one* item, and that item is *null*. Easy to
work around, but should be fixed to return an empty array instead. The
same issue probably applies to other multi-valued fields.

Suggestion: later on, it would be great not to only get the errata URL,
but also some basic information (individual errata IDs with status and
section numbers, if available).

Best regards, Julian