Subject: Opsdir last call review of draft-ietf-quic-transport-32

From: Ron Bonica via Datatracker <[email-masked]>
Date: Tue, 17 Nov 2020 08:28:24 -0800

Reviewer: Ron Bonica
Review result: Ready

Amazingly complete. I am sure that a naive implementer can code from this.

---

From: Behcet Sarikaya <[email-masked]>
Date: Tue, 17 Nov 2020 10:34:57 -0600

> Amazingly complete. I am sure that a naive implementer can code from this.
You nailed it Ron.

I think this is a problem generally in Quic specs.
They are written for implementers.

A protocol specification should not be an implementation spec.
I think this is a deep issue maybe most Quic people do not appreciate
because it seems those people are mostly implementers.

Maybe IAB should discuss this.
Comments  Stewart Bryant gave could help when he reviewed one of the Quic
drafts.

Behcet

> --
> last-call mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/last-call
>

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 17 Nov 2020 17:20:54 +0000

Hi,

> They are written for implementers.
I read Ron's comment as a compliment, not a complaint.

> because it seems those people are mostly implementers.
The specs have been in development since the QUIC WG was chartered in 2016.
Can you point me to the GitHub issues that were raised to address the deep
issues you hint at? Now is a great time to remind people that the WG is in
parallel working on draft-ietf-quic-manageability [1], its intended
audience is network operators and I'm sure that Mirja and Brian would
appreciate more reviews.

Cheers
Lucas
QUIC WG Co-chair

[1] - https://tools.ietf.org/html/draft-ietf-quic-manageability-08

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 17 Nov 2020 17:25:28 +0000

Hi Ron,

Thanks for the review! Since the QUIC WG uses a Github Workflow I've
created a separate issue for each of the items in your review and tagged
you in it, see in-line responses for the precise issue link. All issues are
tracked in the milestone https://github.com/quicwg/base-drafts/milestone/20

Since there are no issues raised, the single ticket at
https://github.com/quicwg/base-drafts/issues/4386 is already closed.

Kind regards,
Lucas
QUIC WG Co-chair

---

From: Behcet Sarikaya <[email-masked]>
Date: Tue, 17 Nov 2020 11:36:05 -0600

Hi My Friend Lucas,

On Tue, Nov 17, 2020 at 11:21 AM Lucas Pardue <[email-masked]>
wrote:

> The specs have been in development since the QUIC WG was chartered in
> 2016. Can you point me to the GitHub issues that were raised to address the
> deep issues you hint at? Now is a great time to remind people that the WG
> is in parallel working on draft-ietf-quic-manageability [1], its intended
I expected that you would copy Stewart's review if you really wish to
discuss this issue in detail seriously..

I don't have it on hand but I kept it somewhere, maybe I had resent it to
quic list?

Stewart gives good comments on the style and directs how they could be
corrected.

I just want to say that in Software Engineering, specification teams and
implementation teams are usually different, that is my background.
As I said this is a deep issue. Certainly it is good the chairs pay
attention to it.

Behcet
Behcet

---

From: Christian Huitema <[email-masked]>
Date: Tue, 17 Nov 2020 09:45:24 -0800

On 11/17/2020 9:36 AM, Behcet Sarikaya wrote:
> ...
>
> I just want to say that in Software Engineering, specification teams
> and implementation teams are usually different, that is my background.
Remember the motto of the IETF, "rough consensus and running code?"
That's pretty much how the QUIC WG has been operating, and I find that
refreshing.

-- Christian Huitema

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 17 Nov 2020 17:52:34 +0000

Hi,

> discuss this issue in detail seriously..

As per-Magnus Last Call instructions, discussion of all Last Call issues
should take place via issues on the GitHub repository.

> Stewart gives good comments on the style and directs how they could be

These have been addressed in the issues tracked under the milestone
https://github.com/quicwg/base-drafts/milestone/4

Cheers,
Lucas

---

From: Behcet Sarikaya <[email-masked]>
Date: Tue, 17 Nov 2020 11:53:28 -0600

On Tue, Nov 17, 2020 at 11:45 AM Christian Huitema <[email-masked]>
wrote:

> Remember the motto of the IETF, "rough consensus and running code?" That's
> pretty much how the QUIC WG has been operating, and I find that refreshing.
>

I respect that.

But I think it is taking it to the extreme.
I personally find quic drafts full of detail that I do not appreciate that
content because I am not an implementer.

So abstracting out these implementation details would make the reading much
more pleasant.
As I said such a style change requires changing the team :)

Behcet

Behcet

---

From: Behcet Sarikaya <[email-masked]>
Date: Tue, 17 Nov 2020 11:54:53 -0600

> https://github.com/quicwg/base-drafts/milestone/4
Good to hear. Now this has to be repeated for all drafts :)

Behcet

---

From: Ian Swett <[email-masked]>
Date: Tue, 17 Nov 2020 12:56:38 -0500

> As I said this is a deep issue. Certainly it is good the chairs pay

I agree that's all too often the case, but I don't see how one can be sure
a spec works if the specification team is separate from the implementation
team?

---

From: Christian Huitema <[email-masked]>
Date: Tue, 17 Nov 2020 11:27:50 -0800

On 11/17/2020 9:56 AM, Ian Swett wrote:

>     I just want to say that in Software Engineering, specification
>     teams and implementation teams are usually different, that is my
>     background.
>     As I said this is a deep issue. Certainly it is good the chairs
>     pay attention to it.
>
>
> I agree that's all too often the case, but I don't see how one can be
> sure a spec works if the specification team is separate from the
> implementation team?

That has been an ongoing issue in the software industry for a long time.
The old 1980-IBM-style "cascade" model supposed a team of analysts
studying the market and the customers requirements, and then a somewhat
subservient team of programmers studying the spec and producing the
code. To complete the cascade model, the less regarded team of testers
would verify that the code actually implements the spec. Let's say that
the successful part of the software industry has moved away from that
model, and uses now constant feedback models in which specs are updated
with feedback from development, tests and user trials, and in which
there is definitely not a hierarchical relation between program
managers, developers, and quality insurance. This is precisely what the
QUIC WG did: there was lots of feedback throughout the process, based on
experience with networks, deployment on small systems and in server
farms, interop testing, etc. I wish more WG would use such processes.

-- Christian Huitema

---

From: Ron Bonica <[email-masked]>
Date: Wed, 18 Nov 2020 05:00:46 +0000

Folks,

It was a compliment!

                           Ron

Juniper Business Use Only
From: Lucas Pardue <[email-masked]>
Sent: Tuesday, November 17, 2020 12:21 PM
To: [email-masked]
Cc: Ron Bonica <[email-masked]>; QUIC WG <[email-masked]>; [email-masked]
Subject: Re: [Last-Call] Opsdir last call review of draft-ietf-quic-transport-32

[External Email. Be cautious of content]

Hi,

On Tue, Nov 17, 2020 at 4:35 PM Behcet Sarikaya <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Nov 17, 2020 at 10:28 AM Ron Bonica via Datatracker <[email-masked]<mailto:[email-masked]>> wrote:
Reviewer: Ron Bonica
Review result: Ready

You nailed it Ron.

A protocol specification should not be an implementation spec.
I think this is a deep issue maybe most Quic people do not appreciate because it seems those people are mostly implementers.

The specs have been in development since the QUIC WG was chartered in 2016. Can you point me to the GitHub issues that were raised to address the deep issues you hint at? Now is a great time to remind people that the WG is in parallel working on draft-ietf-quic-manageability [1], its intended audience is network operators and I'm sure that Mirja and Brian would appreciate more reviews.

Cheers
Lucas
QUIC WG Co-chair

[1] - https://tools.ietf.org/html/draft-ietf-quic-manageability-08<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-ietf-quic-manageability-08__;!!NEt6yMaO-gk!RMV5WFfKMS698-wK5ktX0HpdUEjZyCopB69erusSYE5miVm4WEgRkaEiEXiO8c07$>

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 18 Nov 2020 08:22:23 +0000

On Tue, 2020-11-17 at 10:34 -0600, Behcet Sarikaya wrote:
> I think this is a deep issue maybe most Quic people do not appreciate because
> it seems those people are mostly implementers.

As responsible AD I do want to respond to this. Protocol specification exists to
enable implementation. And that it is written for implementors are actually
great as it will avoid many interoperability issues. Other usage of the
specification I think will not be greately challenged by the detail level. This
is not a novel, it is a protocol specification. So I don't consider this an
issue, rather the opposite. 

From my perspective the QUIC documents are in the top percentile of documents
when it comes to specification quality that I have seen during my soon 6 years
as AD from across the whole IETF.

Cheers

Magnus Westerlund
TSV AD

---

From: Mike Bishop <[email-masked]>
Date: Wed, 18 Nov 2020 08:43:01 +0000

You, like anyone else who has issues with the draft, have been invited to file those issues on GitHub or by sending a review e-mail.  Bearing in mind that we’re now past the end of Last Call, I can assure you that we as editors will attempt to address all usable feedback on the style and content of the documents.

From: QUIC <[email-masked]> On Behalf Of Behcet Sarikaya
Sent: Tuesday, November 17, 2020 12:55 PM
To: Lucas Pardue <[email-masked]>
Cc: Ron Bonica <[email-masked]>; Behcet Sarikaya <[email-masked]>; QUIC WG <[email-masked]>; [email-masked]
Subject: Re: [Last-Call] Opsdir last call review of draft-ietf-quic-transport-32

On Tue, Nov 17, 2020 at 11:52 AM Lucas Pardue <[email-masked] <mailto:[email-masked]> > wrote:

Hi,

On Tue, Nov 17, 2020 at 5:36 PM Behcet Sarikaya <[email-masked] <mailto:[email-masked]> > wrote:

Hi My Friend Lucas,

I expected that you would copy Stewart's review if you really wish to discuss this issue in detail seriously.. 

As per-Magnus Last Call instructions, discussion of all Last Call issues should take place via issues on the GitHub repository. 

I don't have it on hand but I kept it somewhere, maybe I had resent it to quic list?

Stewart gives good comments on the style and directs how they could be corrected.

These have been addressed in the issues tracked under the milestone https://github.com/quicwg/base-drafts/milestone/4

Behcet 

Cheers,

Lucas

---

From: Behcet Sarikaya <[email-masked]>
Date: Wed, 18 Nov 2020 10:20:03 -0600

On Wed, Nov 18, 2020 at 2:22 AM Magnus Westerlund <
[email-masked]> wrote:

> because

I am not sure. I think IESG could know. How many people that read protocol
RFCs
go ahead and implement them?
I for one read a lot of RFCs but I have never implemented protocols, other
teams do that, it is not my job.

Maybe many these days because QUIC is being deployed. But later on
the statistics could drastically change.
Also as we know from Software Engineering, the process does not go direct,
i.e read the RFC and  give it to the implementation team.

In short, I think ADs, IESG should consider this issue seriously and I
believe in the end, spec view will win.
We need the implementation detail removed with a great thank you to the
editors.
 That said, I am not going to fight in this as I have no dog in this fight
:)

Behcet

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 18 Nov 2020 16:50:13 +0000

Hi Behcet,

On Wed, Nov 18, 2020 at 4:20 PM Behcet Sarikaya <[email-masked]>
wrote:

> teams do that, it is not my job
>

>  That said, I am not going to fight in this as I have no dog in this fight

I have to agree with Magnus. The specs are not non-fiction accounts of
technology or layperson PR-friendly nutshell soundbites. If you remove
implementation detail there is no information for anyone to make
interoperable implementations.

There's a whole industry of folks that do a fantastic job of turning these
very detailed specs into deployments, products, books, video and podcasts
that suit end-users, explaining things more user-firendly terms. Dumbing
down specification just duplicates that work and is a disservice to
engineers. Rather than reading RFCs, I suggest people go and read something
like Daniel Stenberg's "HTTP/3 explained" [1], Ilya Grigorik's
"High-Performance Browser Networking", or get a large pot of coffee and
watch the 12+ hours of Video-on-Demand content that I produced on the topic
of QUIC & HTTP/3 [3].

Cheers,
Lucas

[1] - https://daniel.haxx.se/http3-explained/
[2] - https://hpbn.co/
[3] - https://blog.cloudflare.com/last-call-for-quic/

---

From: Behcet Sarikaya <[email-masked]>
Date: Wed, 18 Nov 2020 11:05:36 -0600

On Wed, Nov 18, 2020 at 10:50 AM Lucas Pardue <[email-masked]>
wrote:

>> I am not sure. I think IESG could know. How many people that read
>> protocol RFCs
>> I for one read a lot of RFCs but I have never implemented protocols,
>> other teams do that, it is not my job
>>
>
>> Also as we know from Software Engineering, the process does not go
>> direct, i.e read the RFC and  give it to the implementation team.
>>
>>  That said, I am not going to fight in this as I have no dog in this
>> fight :)
>>
>

Lucas,
Please stop this. I have not made any personal "attacks" like this to
anyone on this list or elsewhere in IETF.

I am an ex-academic who has written a book on Principles of Protocol
Engineering and Conformance Testing
in 1993 published by Ellis Horwood
which  basically discussed how to formally specify protocols (that time it
was OSI protocols, pre-IETF years) in a text book setting.

Behcet

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 18 Nov 2020 17:15:54 +0000

Hi Behcet,

> was OSI protocols, pre-IETF years) in a text book setting.

Apologies if you felt attacked, that was not the intention. But I am
struggling to understand what you're asking for in the context of the
documents that the QUIC working group was chartered to deliver.

I'd also like to highlight that the editors (and many contributors) have
worked very tirelessly on this protocol specification over many years. The
notion that we would give them a pat on the back and delete swathes of text
would frankly be a disservice.

Cheers
Lucas

---

From: Matt Joras <[email-masked]>
Date: Wed, 18 Nov 2020 09:46:50 -0800

One response inline.

On Wed, Nov 18, 2020, 9:06 AM Behcet Sarikaya <[email-masked]>
wrote:

>> There's a whole industry of folks that do a fantastic job of turning
>> these very detailed specs into deployments, products, books, video and
>> podcasts that suit end-users, explaining things more user-firendly terms.
>> Dumbing down specification just duplicates that work and is a disservice to

Calling OSI "pre-IETF" is disingenuous. There were RFCs before the IETF
itself was formalized, and there is a direct line of continuation for the
culture of Internet standards going all the way back to RFC 1 from 1969.
Core to this culture is specification involvement by implementers and fast
feedback from running code. This is distinctly different from the classical
engineering culture of standardization employed by bodies like the ISO,
which lead to the OSI protocols. The undeniable success of the early
ARPANET protocols and the eventual TCP/IP protocol and subsequent evolution
of the Internet and networking in general is due to the difference in
culture and practice of standardization.

As Christian and others pointed out, QUIC as a working group and set of
specifications embodies this ideal. The number and diversity of
interoperating implementations is a testament to the quality and utility of
the QUIC specifications. They may not match your expectations, but I would
argue then that your expectations are not in line with what makes a
successful RFC specification.