Subject: Re: [PANRG] [irsg] IRSG review request draft-irtf-panrg-what-not-to-do-13

From: Mallory Knodel <[email-masked]>
Date: Tue, 20 Oct 2020 15:47:40 -0400

Dear Spencer, panrg and irsg:

It was my pleasure to read this document as it was a thorough compendium 
of Path Aware Networking in the IETF and serves as a great primer!

My main comments on the draft are really to increase accessibility and 
readability of what surely is thorough, complete and clear documentation 
of a high quality appropriate for IRTF documents. I also realise that 
because my suggested edits are quite structural, that it's very likely 
you've had these conversations in the RG already and so if this walks 
back a previous agreement, feel free to disregard.

While I read, I did a lot of flipping back and forth, particularly 
between Sections 2 and 5. I usually do a lot of flipping anyway; this 
review was uniquely flip-heavy. You also have a lot of internal 
references. I'd suggest a restructure thusly:

  * Section 1: Pairing down mostly meta information about the RG 
(documented elsewhere), doc (not all the background is pertienent 
context), etc so that the reader gets to the lessons quicker.

  * Section 4: Make this a methodology section right after the 
introduction, also to potentially guide future contributions. You can 
then explain how you first collected contributions and then extracted 
the important details, drew lessons learned and conclusions. You can 
explain here categories for Table 1.

  * Compress Sections 2, 3 and 5. I can see value in just getting the 
lessons upfront, but if you perhaps nest the various contributions 
wisely you can create sub-sections and even sub-sub-sections here, 
allowing you to give the reader the highest points (Section 3), followed 
by generalisations (Section 2), followed by Sub-sections (Section 5). 
Your headings are well phrased, such that if one doesn't want to read 
the details of Section 5, they can stop reading and/or skip. This also 
allows your Section 5 to be sparse-- perhaps there's sufficient 
documentation elsewhere on the internet that you could simply reference 
and leave in just a summary/abstract of lessons learned without the 
burden of having to include the data of a full contribution.

If you take my last suggestion, you might want to add to your template 
for contributions that you'd like a summary/abstract (a la Section 2) 
and a suggested category (a la Table 1).

Lastly I'll point out that "What Not To Do" is a terrific framing, but 
then you need to ensure that your Section 2 headings/section titles have 
the same value frame (positive vs negative). Eg "DO Justify Deployment" 
or "DON'T React to Distant Signals". Whatever it is, just pick one and 
then adjust the rest. Or spell it out in each sub-section title.

Nits:

  * Someone like me needs just one sentence in the introduction about 
the problem that PAN solves. It's probably just so obvious to you all!

  * The first paragraph of 1.1. seems to be missing a specific reference 
to "path aware networking" otherwise it's quite generic a statement.

  * RFC 5218 seems like an appropriate reference for 2.1, 2.2 and 2.3? 
Again, pretty generic and not PAN specific, seemingly.

  * 2.1 would suggest "_U.S._ American English" since most of America 
(the hemisphere) doesn't even speak English, and we also appreciate 
Canadian Anglophones who may or may not use this phrase.

  * 2.6 last sentence: You might have meant "These _changes_ may make 
deployment..."

  * 2.7 Needs perhaps another sentence that answers "Why?".

  * 1.4, 2.12 there are sometimes notes about what might be done to 
mitigate the problem faced. These might be out of scope? But probably 
they are extra valuable and should be emphasized in a separate 
sub-section or summary, somewhere.

-Mallory

On 9/6/20 2:45 PM, Colin Perkins wrote:
> IRSG members,
>
> The Path Aware Networking Research Group has requested that draft-irtf-panrg-what-not-to-do-13 be considered for publication as an IRTF RFC. To progress this draft, we now need *at least one* IRSG member to volunteer to provide a detailed review of the draft, as follows:
>
>> The purpose of the IRSG review is to ensure consistent editorial and technical quality for IRTF publications. IRSG review is not a deep technical review. (This should take place within the RG.) At least one IRSG member other than the chair of the RG bringing the work forth must review the document and the RG’s editorial process.
>>
>> IRSG reviewers should look for clear, cogent, and consistent writing. An important aspect of the review is to gain a critical reading from reviewers who are not subject matter experts and, in the process, assure the document will be accessible to those beyond the authoring research group. Also, reviewers should assess whether sufficient editorial and technical review has been conducted and the requirements for publication described in RFC 5743  have been met. Finally, reviewers should check that appropriate citations to related research literature have been made.
>>
>> Reviews should be written to be public. Review comments should be sent to the IRSG and RG mailing lists and entered into the tracker. All IRSG review comments must be addressed. However, the RG need not accept every comment. It is the responsibility of the shepherd to understand the comments and ensure that the RG considers them including adequate dialog between the reviewer and the author and/or RG. Reviews and their resolution should be entered into the tracker by the document shepherd.
>>
>> The IRSG review often results in the document being revised. Once the reviewer(s), authors, and shepherd have converged on review comments, the shepherd starts the IRSG Poll on whether the document should be published.
> Please respond to this message if you’re able to perform such a review, and indicate the approximate time-frame by which you’ll be able to complete it. The document shepherd write-up is available at  https://datatracker.ietf.org/doc/draft-irtf-panrg-what-not-to-do/shepherdwriteup/
>
> Thanks!
>
> Colin (as IRTF chair)

-- 
Mallory Knodel
CTO, Center for Democracy and Technology
gpg fingerprint :: E3EB 63E0 65A3 B240 BCD9 B071 0C32 A271 BD3C C780

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 20 Oct 2020 16:07:27 -0500

Hi, Mallory,

Thanks for looking at this draft with new eyes!

I'll see what the shepherd/chairs/research group think, and I hope we can
resolve this before IETF 109 (but that's just me talking).

Best,

Spencer

On Tue, Oct 20, 2020 at 2:47 PM Mallory Knodel <[email-masked]> wrote:

> > The Path Aware Networking Research Group has requested that
> draft-irtf-panrg-what-not-to-do-13 be considered for publication as an IRTF
> RFC. To progress this draft, we now need *at least one* IRSG member to
> volunteer to provide a detailed review of the draft, as follows:
> >
> >> The purpose of the IRSG review is to ensure consistent editorial and
> technical quality for IRTF publications. IRSG review is not a deep
> technical review. (This should take place within the RG.) At least one IRSG
> member other than the chair of the RG bringing the work forth must review
> the document and the RG’s editorial process.
> >>
> >> IRSG reviewers should look for clear, cogent, and consistent writing.
> An important aspect of the review is to gain a critical reading from
> reviewers who are not subject matter experts and, in the process, assure
> the document will be accessible to those beyond the authoring research
> group. Also, reviewers should assess whether sufficient editorial and
> technical review has been conducted and the requirements for publication
> described in RFC 5743  have been met. Finally, reviewers should check that
> appropriate citations to related research literature have been made.
> >>
> >> Reviews should be written to be public. Review comments should be sent
> to the IRSG and RG mailing lists and entered into the tracker. All IRSG
> review comments must be addressed. However, the RG need not accept every
> comment. It is the responsibility of the shepherd to understand the
> comments and ensure that the RG considers them including adequate dialog
> between the reviewer and the author and/or RG. Reviews and their resolution
> should be entered into the tracker by the document shepherd.
> >>
> >> The IRSG review often results in the document being revised. Once the
> reviewer(s), authors, and shepherd have converged on review comments, the
> shepherd starts the IRSG Poll on whether the document should be published.
> > Please respond to this message if you’re able to perform such a review,
> and indicate the approximate time-frame by which you’ll be able to complete
> it. The document shepherd write-up is available at
> https://datatracker.ietf.org/doc/draft-irtf-panrg-what-not-to-do/shepherdwriteup/
> >
> > Thanks!
> >
> > Colin (as IRTF chair)
>
> --
> Mallory Knodel

---

From: Colin Perkins <[email-masked]>
Date: Tue, 20 Oct 2020 23:44:19 +0100

Thanks for the review, Mallory!

Colin

> On 20 Oct 2020, at 20:47, Mallory Knodel <[email-masked]> wrote:
> 
> It was my pleasure to read this document as it was a thorough compendium of Path Aware Networking in the IETF and serves as a great primer!
> 
> My main comments on the draft are really to increase accessibility and readability of what surely is thorough, complete and clear documentation of a high quality appropriate for IRTF documents. I also realise that because my suggested edits are quite structural, that it's very likely you've had these conversations in the RG already and so if this walks back a previous agreement, feel free to disregard.
> 
> While I read, I did a lot of flipping back and forth, particularly between Sections 2 and 5. I usually do a lot of flipping anyway; this review was uniquely flip-heavy. You also have a lot of internal references. I'd suggest a restructure thusly:
> 
>  * Section 1: Pairing down mostly meta information about the RG (documented elsewhere), doc (not all the background is pertienent context), etc so that the reader gets to the lessons quicker.
> 
>  * Section 4: Make this a methodology section right after the introduction, also to potentially guide future contributions. You can then explain how you first collected contributions and then extracted the important details, drew lessons learned and conclusions. You can explain here categories for Table 1.
> 
>  * Compress Sections 2, 3 and 5. I can see value in just getting the lessons upfront, but if you perhaps nest the various contributions wisely you can create sub-sections and even sub-sub-sections here, allowing you to give the reader the highest points (Section 3), followed by generalisations (Section 2), followed by Sub-sections (Section 5). Your headings are well phrased, such that if one doesn't want to read the details of Section 5, they can stop reading and/or skip. This also allows your Section 5 to be sparse-- perhaps there's sufficient documentation elsewhere on the internet that you could simply reference and leave in just a summary/abstract of lessons learned without the burden of having to include the data of a full contribution.
> 
> If you take my last suggestion, you might want to add to your template for contributions that you'd like a summary/abstract (a la Section 2) and a suggested category (a la Table 1).
> 
> Lastly I'll point out that "What Not To Do" is a terrific framing, but then you need to ensure that your Section 2 headings/section titles have the same value frame (positive vs negative). Eg "DO Justify Deployment" or "DON'T React to Distant Signals". Whatever it is, just pick one and then adjust the rest. Or spell it out in each sub-section title.
> 
> Nits:
> 
>  * Someone like me needs just one sentence in the introduction about the problem that PAN solves. It's probably just so obvious to you all!
> 
>  * The first paragraph of 1.1. seems to be missing a specific reference to "path aware networking" otherwise it's quite generic a statement.
> 
>  * RFC 5218 seems like an appropriate reference for 2.1, 2.2 and 2.3? Again, pretty generic and not PAN specific, seemingly.
> 
>  * 2.1 would suggest "_U.S._ American English" since most of America (the hemisphere) doesn't even speak English, and we also appreciate Canadian Anglophones who may or may not use this phrase.
> 
>  * 2.6 last sentence: You might have meant "These _changes_ may make deployment..."
> 
>  * 1.4, 2.12 there are sometimes notes about what might be done to mitigate the problem faced. These might be out of scope? But probably they are extra valuable and should be emphasized in a separate sub-section or summary, somewhere.
> 
> -Mallory
> 

-- 
Colin Perkins
https://csperkins.org/