Subject: qlog: new streaming serialization format

From: Robin MARX <[email-masked]>
Date: Wed, 20 Oct 2021 16:19:16 +0200

Hello everyone,

As discussed during IETF 111, we are looking to update qlog to move to a
new streaming serialization format.

The current version uses the "Newline Delimited JSON" (NDJSON) format for
event streaming, which, while it works well, is not an official standard
(in the IETF or elsewhere).

As such, it was proposed we move to the "JSON text-sequences" format
defined in RFC 7464 [1]. json-seq separates each record (qlog event in our
case) with the Record Separator character (\x1E) and a \n at the end. As
such, it should be easy to update an existing NDJSON implementation (as it
just requires outputting one additional character per event).

We have prototyped and tested this approach in qvis and various other tools
like jq and the early outcomes are looking good [2].

As such, we will be preparing a PR and associated draft-01 for IETF 112
that introduces this change.

In the meantime, it would be great if people interested in event streaming
or with existing NDJSON qlog stacks could look at the proposal/early
results [2] and provide feedback.

[1]: https://datatracker.ietf.org/doc/html/rfc7464
[2]: https://github.com/quicwg/qlog/issues/172

With best regards,
the qlog editors

P.S. note that we also still plan to keep the normal JSON non-streaming
format intact, leading to two "main" formats defined in the spec: 1) normal
JSON and 2) JSON text sequences.

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

*Cellphone *+32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

---

From: Robin MARX <[email-masked]>
Date: Tue, 26 Oct 2021 17:04:48 +0200

Hello all,

Thanks to everyone who provided feedback on the github issue.

We have now released a new draft of the qlog main schema detailing the new
streaming format [1].
qvis [2] has also been updated to support the new approach, so you can load
files using the new format/extension.
If you encounter any issues, please let me know.

I will be available during the hackathon next week for people who want to
implement this new setup and have questions/find bugs/etc.

I am also open to discuss qlog/qvis support for other emerging use cases,
such as multipath, masque, webtransport, etc. if people are interested.

[1]: https://www.ietf.org/id/draft-ietf-quic-qlog-main-schema-01.html
[2]: https://qvis.quictools.info/

With best regards,
Robin

On Wed, 20 Oct 2021 at 16:19, Robin MARX <[email-masked]> wrote:

> We have prototyped and tested this approach in qvis and various other
> tools like jq and the early outcomes are looking good [2].
>

-- 

dr. Robin Marx

www.uhasselt.be