Subject: [CFRG] Proposed CFRG process for handling errata

From: Alexey Melnikov <[email-masked]>
Date: Thu, 24 Dec 2020 10:25:30 +0000

Dear CFRG participants,

Below is the proposed process that CFRG chairs would follow when 
handling errata submitted on CFRG documents.

Please let chairs know by January 16th if you have comments or concerns. 
Statements of support for this proposal are also welcome.
------------

An erratum is submitted through www.rfc-editor.org website An erratum on 
a CFRG document results in email to [email-masked] (+authors/editors of 
the RFC) with subject like "[Technical Errata Reported] RFCXXXX (YYYY)", 
where XXXX is the relevant RFC number and YYYY is the corresponding 
erratum number (assigned automatically by RFC Editor's website).

Note that the current errata system is not designed for reporting of 
extensions and things that were not known or intended at the time the 
document was written. It is only designed for reporting problems/bugs in 
documents.

One of CFRG chairs becomes the response CFRG chair for the erratum. 
He/she verifies that the erratum designation (Technical versa Editorial) 
is correct. (Note that the designation can be changed later and it is Ok 
if initially it is unclear for some errata which one it is.) The CFRG 
chair can also request deletion of obviously bogus erratum, such as 
submitted by spammers.

The CFRG chair then can optionally request review from the Crypto Review 
Panel <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel>, by 
emailing [email-masked] and asking for comments on how to resolve 
the erratum. A proposed resolution can be suggested, if available. The 
request should specify a deadline, typically 2 weeks. This deadline can 
be extended by request from Crypto Review Panel members. Note that 
feedback from Crypto Review Panel is advisory in nature.

The CFRG chair then emails the CFRG mailing list <[email-masked]> asking 
for comments on how to resolve the erratum. (Possible subject to use: 
"Proposed resolution for erratum YYYY on RFC XXXX") A proposed 
resolution can be suggested, if available. The proposed resolution takes 
into consideration feedback received from the Crypto Review Panel (if 
requested). The request should specify a recommended deadline for 
discussions.

The CFRG chair follows the CFRG mailing list discussion of the erratum 
resolution and posts a summary email after the deadline expires. If 
there is a clear resolution ("accept", "reject" or "hold for document 
update") the suggested resolution (and possible changes to the erratum 
text) are included in the summary email. The IRTF Chair should be 
notified about the proposed resolution for the erratum.

The IRTF Chair verifies outcome of the process and either acts on the 
erratum as proposed by the CFRG chair or delegates this decision to 
another person (who might be the CFRG chair). (The IRTF chair can 
provide technical feedback on the erratum in his/her personal capacity. 
The IRTF Chair can also restart discussion of the erratum on the CFRG 
mailing list.)

------------

Best Regards,

Alexey, for the CFRG chairs

---

From: Russ Housley <[email-masked]>
Date: Thu, 24 Dec 2020 12:05:35 -0500

Seems fine to me.  Hopefully most errata will not require Crypto Review Panel consultation.

Russ

> On Dec 24, 2020, at 5:25 AM, Alexey Melnikov <[email-masked]> wrote:
> 
> Dear CFRG participants,
> 
> Below is the proposed process that CFRG chairs would follow when handling errata submitted on CFRG documents.
> 
> Please let chairs know by January 16th if you have comments or concerns. Statements of support for this proposal are also welcome.
> ------------
> 
> An erratum is submitted through www.rfc-editor.org website An erratum on a CFRG document results in email to [email-masked] (+authors/editors of the RFC) with subject like "[Technical Errata Reported] RFCXXXX (YYYY)", where XXXX is the relevant RFC number and YYYY is the corresponding erratum number (assigned automatically by RFC Editor's website).
> 
> Note that the current errata system is not designed for reporting of extensions and things that were not known or intended at the time the document was written. It is only designed for reporting problems/bugs in documents.
> 
> 
> One of CFRG chairs becomes the response CFRG chair for the erratum. He/she verifies that the erratum designation (Technical versa Editorial) is correct. (Note that the designation can be changed later and it is Ok if initially it is unclear for some errata which one it is.) The CFRG chair can also request deletion of obviously bogus erratum, such as submitted by spammers.
> 
> The CFRG chair then can optionally request review from the Crypto Review Panel <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel>, by emailing [email-masked] and asking for comments on how to resolve the erratum. A proposed resolution can be suggested, if available. The request should specify a deadline, typically 2 weeks. This deadline can be extended by request from Crypto Review Panel members. Note that feedback from Crypto Review Panel is advisory in nature.
> 
> The CFRG chair then emails the CFRG mailing list <[email-masked]> asking for comments on how to resolve the erratum. (Possible subject to use: "Proposed resolution for erratum YYYY on RFC XXXX") A proposed resolution can be suggested, if available. The proposed resolution takes into consideration feedback received from the Crypto Review Panel (if requested). The request should specify a recommended deadline for discussions.
> 
> The CFRG chair follows the CFRG mailing list discussion of the erratum resolution and posts a summary email after the deadline expires. If there is a clear resolution ("accept", "reject" or "hold for document update") the suggested resolution (and possible changes to the erratum text) are included in the summary email. The IRTF Chair should be notified about the proposed resolution for the erratum.
> 
> The IRTF Chair verifies outcome of the process and either acts on the erratum as proposed by the CFRG chair or delegates this decision to another person (who might be the CFRG chair). (The IRTF chair can provide technical feedback on the erratum in his/her personal capacity. The IRTF Chair can also restart discussion of the erratum on the CFRG mailing list.)
> 
> ------------
> 
> Best Regards,
> 
> _______________________________________________
> CFRG mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/cfrg

---

From: James Muir <[email-masked]>
Date: Fri, 25 Dec 2020 14:08:35 -0500

> Statements of support for this proposal are also welcome.

This sounds reasonable.  I especially like the inclusion of deadlines.

You can clarify what role the authors of the RFC have in this process?

-James M

---

From: Michael StJohns <[email-masked]>
Date: Fri, 25 Dec 2020 19:59:28 -0500

Hi Alexey -

 From a general philosophy, this seems to conflict with the general 
guidance for RGs - specifically (RFC 2014 "IRTF Research Group 
Guidelines and Procedures" - section 1.1 ):

>   Even more than the IETF, the work of the IRSG is expected to be
>     marked by informality.  The goal is to encourage and foster valuable
>     research,*_not to add burdensome bureaucracy to the endeavo_*r.

I brought up a while back that the CFRG has taken on more of the 
characteristics of a WG than an RG and it may be appropriate to 
recharter the group under the IETF as a WG or even co-charter it as both 
an RG and WG.  Let me make that point again.

Also in section 1.1:
>   The IRTF does not set standards, and thus has somewhat different and
>     complementary philosophy and procedures.  In particular, an IRTF
>     Research Group is expected to be long-lived, producing a sequence of
>    "products" over time._*The products of a Research Group are research results*_  that may be disseminated by publication in scholarly journals
>     and conferences, as white papers for the community, as Informational
>     RFCs, and so on.  In addition, i_*t is expected that technologies developed in a Research Group will 
> be brought to the IETF as input to IETF Working Group(s) for possible 
> standardization.*_    However,
>     Research Group input carries no more weight than other community
>     input, and goes through the same standards setting process as any
>     other proposal.

Errata is appropriate for standards - less so for research papers.

Or to put it more succinctly - I object to the addition of more process 
to the RG.  I further suggest that if this isn't apropriate as  IRTF 
wide process, then it's probably not within the remit of the CFRG to 
create its own processes - no more than it would for a WG.

Later, Mike

ps - feel free to propose amendments to RFC2014, but at AFAICT that 
document  is the one that describes the current general contract for RGs.

On 12/24/2020 5:25 AM, Alexey Melnikov wrote:
> Please let chairs know by January 16th if you have comments or 
> concerns. Statements of support for this proposal are also welcome.
> ------------
>
> An erratum is submitted through www.rfc-editor.org website An erratum 
> on a CFRG document results in email to [email-masked] (+authors/editors 
> of the RFC) with subject like "[Technical Errata Reported] RFCXXXX 
> (YYYY)", where XXXX is the relevant RFC number and YYYY is the 
> corresponding erratum number (assigned automatically by RFC Editor's 
> website).
>
> document was written. It is only designed for reporting problems/bugs 
> in documents.
>
>
> He/she verifies that the erratum designation (Technical versa 
> Editorial) is correct. (Note that the designation can be changed later 
> and it is Ok if initially it is unclear for some errata which one it 
> is.) The CFRG chair can also request deletion of obviously bogus 
> erratum, such as submitted by spammers.
>
> The CFRG chair then can optionally request review from the Crypto 
> Review Panel 
> <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel>, by 
> emailing [email-masked] and asking for comments on how to 
> resolve the erratum. A proposed resolution can be suggested, if 
> available. The request should specify a deadline, typically 2 weeks. 
> This deadline can be extended by request from Crypto Review Panel 
> members. Note that feedback from Crypto Review Panel is advisory in 
> nature.
>
> The CFRG chair then emails the CFRG mailing list <[email-masked]> 
> asking for comments on how to resolve the erratum. (Possible subject 
> to use: "Proposed resolution for erratum YYYY on RFC XXXX") A proposed 
> resolution can be suggested, if available. The proposed resolution 
> takes into consideration feedback received from the Crypto Review 
> Panel (if requested). The request should specify a recommended 
> deadline for discussions.
>
> provide technical feedback on the erratum in his/her personal 
> capacity. The IRTF Chair can also restart discussion of the erratum on 
> the CFRG mailing list.)
>
> ------------
>
> Best Regards,
>

---

From: Colin Perkins <[email-masked]>
Date: Sat, 26 Dec 2020 13:24:34 +0000

Mike,

Corrections and errata are common for published for research papers, whether published as RFCs or by more traditional academic publishers.

As IRTF Chair, and approver of errata for IRTF stream RFCs, I asked the CFRG Chairs to develop a process to help me ensure errata for RFCs produced by CFRG are correctly resolved. Nothing in this proposal conflicts with RFC 2014.

Colin

> On 26 Dec 2020, at 00:59, Michael StJohns <[email-masked]> wrote:
> 
> Hi Alexey -
> 
> From a general philosophy, this seems to conflict with the general guidance for RGs - specifically (RFC 2014 "IRTF Research Group Guidelines and Procedures" - section 1.1 ): 
> 
>>    research, not to add burdensome bureaucracy to the endeavor.
> 
> 
> I brought up a while back that the CFRG has taken on more of the characteristics of a WG than an RG and it may be appropriate to recharter the group under the IETF as a WG or even co-charter it as both an RG and WG.  Let me make that point again.
> 
> Also in section 1.1: 
>>   "products" over time.  The products of a Research Group are research
>>    results that may be disseminated by publication in scholarly journals
>>    RFCs, and so on.  In addition, it is expected that technologies
>>    developed in a Research Group will be brought to the IETF as input to
>>    IETF Working Group(s) for possible standardization.   However,
> Or to put it more succinctly - I object to the addition of more process to the RG.  I further suggest that if this isn't apropriate as  IRTF wide process, then it's probably not within the remit of the CFRG to create its own processes - no more than it would for a WG.
> 
> Later, Mike
> 
> ps - feel free to propose amendments to RFC2014, but at AFAICT that document  is the one that describes the current general contract for RGs.
> 
> 
>> An erratum is submitted through www.rfc-editor.org <http://www.rfc-editor.org/> website An erratum on a CFRG document results in email to [email-masked] <mailto:[email-masked]> (+authors/editors of the RFC) with subject like "[Technical Errata       Reported] RFCXXXX (YYYY)", where XXXX is the relevant RFC number and YYYY is the corresponding erratum number (assigned automatically by RFC Editor's website). 
>> 
>> The CFRG chair then can optionally request review from the Crypto Review Panel<https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel> <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel>, by emailing [email-masked] <mailto:[email-masked]> and asking for comments on how to resolve the erratum. A proposed resolution can be suggested, if available. The request should specify a deadline, typically 2 weeks. This deadline can be extended by request from Crypto Review Panel members. Note that feedback from Crypto Review Panel is advisory in nature. 
>> 
>> The CFRG chair then emails the CFRG mailing list <[email-masked]> <mailto:[email-masked]> asking for comments on how to resolve the erratum. (Possible subject to use: "Proposed resolution for erratum YYYY on RFC XXXX") A proposed resolution can be suggested, if available. The proposed resolution takes into consideration feedback received from the Crypto Review Panel (if requested). The request should specify a recommended deadline for discussions. 
>> 
>> [email-masked] <mailto:[email-masked]> 
>> https://www.irtf.org/mailman/listinfo/cfrg <https://www.irtf.org/mailman/listinfo/cfrg> 
> 

-- 
Colin Perkins
https://csperkins.org/

---

From: Russ Housley <[email-masked]>
Date: Sun, 27 Dec 2020 14:50:17 -0500

As you say, the CFRG is not like many other RGs.  Crypto algorithm specifications, while informational, ofter become normative references in standards.  For this reason, I think robust handling of errata is desirable.

Russ

> On Dec 25, 2020, at 7:59 PM, Michael StJohns <[email-masked]> wrote:
> 
> Hi Alexey -
> 
>> The CFRG chair then can optionally request review from the Crypto Review Panel <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel> <https://trac.ietf.org/trac/irtf/wiki/Crypto%20Review%20Panel>, by emailing [email-masked] <mailto:[email-masked]> and asking for comments on how to resolve the erratum. A proposed resolution can be suggested, if available. The request should specify a deadline, typically 2 weeks. This deadline can be extended by request from Crypto Review Panel members. Note that feedback from Crypto Review Panel is advisory in nature. 
>>

---

From: "Salz, Rich" <[email-masked]>
Date: Sun, 27 Dec 2020 21:36:27 +0000

*   As you say, the CFRG is not like many other RGs.  Crypto algorithm specifications, while informational, ofter become normative references in standards.  For this reason, I think robust handling of errata is desirable.

I agree.  Four other things come to mind:

  *   Much “crypto research” finds its home here (e.g., the pairing stuff most recently), and I think would be less likely if this were an IETF WG.
  *   I also wonder what a WG group with no milestones would be like

  *   There’s no place good IESG home for such a WG
  *   IESG “supervision” (or at least approval) seems like a really bad fit

---

From: Michael StJohns <[email-masked]>
Date: Sun, 27 Dec 2020 16:37:24 -0500

Hi Russ -

We ought to stop promoting the fiction that the CFRG isn't doing 
standards and move the CWG part over to the IETF where it belongs.  I 
don't actually have a problem with the errata by itself, but I do when 
you add it to calls for adoption, group consensus, last calls and the 
whole set of things more appropriate for the standards process than what 
the IRTF was intended to be.

Mike

On 12/27/2020 2:50 PM, Russ Housley wrote:
> As you say, the CFRG is not like many other RGs.  Crypto algorithm 
> specifications, while informational, ofter become normative references 
> in standards.  For this reason, I think robust handling of errata is 
> desirable.
>
> Russ
>
>> On Dec 25, 2020, at 7:59 PM, Michael StJohns <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi Alexey -
>>
>> recharter the group under the IETF as a WG or even co-charter it as 
>> both an RG and WG.  Let me make that point again.
>>
>> Also in section 1.1:
>>> be brought to the IETF as input to IETF Working Group(s) for 
>>> possible standardization.*_    However,
>> Or to put it more succinctly - I object to the addition of more 
>> process to the RG.  I further suggest that if this isn't apropriate 
>> as  IRTF wide process, then it's probably not within the remit of the 
>> CFRG to create its own processes - no more than it would for a WG.
>>
>> Later, Mike
>>
>>> An erratum is submitted through www.rfc-editor.org website An 
>>> erratum on a CFRG document results in email to [email-masked] 
>>> (+authors/editors of the RFC) with subject like "[Technical Errata 
>>> Reported] RFCXXXX (YYYY)", where XXXX is the relevant RFC number and 
>>> YYYY is the corresponding erratum number (assigned automatically by 
>>> RFC Editor's website).
>>>
>>> extensions and things that were not known or intended at the time 
>>> the document was written. It is only designed for reporting 
>>> problems/bugs in documents.
>>>
>>>
>>> Editorial) is correct. (Note that the designation can be changed 
>>> later and it is Ok if initially it is unclear for some errata which 
>>> one it is.) The CFRG chair can also request deletion of obviously 
>>> bogus erratum, such as submitted by spammers.
>>>
>>> to use: "Proposed resolution for erratum YYYY on RFC XXXX") A 
>>> proposed resolution can be suggested, if available. The proposed 
>>> resolution takes into consideration feedback received from the 
>>> Crypto Review Panel (if requested). The request should specify a 
>>> recommended deadline for discussions.
>>>
>>> The CFRG chair follows the CFRG mailing list discussion of the 
>>> erratum resolution and posts a summary email after the deadline 
>>> expires. If there is a clear resolution ("accept", "reject" or "hold 
>>> for document update") the suggested resolution (and possible changes 
>>> to the erratum text) are included in the summary email. The IRTF 
>>> Chair should be notified about the proposed resolution for the erratum.
>>>
>>> The IRTF Chair verifies outcome of the process and either acts on 
>>> the erratum as proposed by the CFRG chair or delegates this decision 
>>> to another person (who might be the CFRG chair). (The IRTF chair can 
>>> capacity. The IRTF Chair can also restart discussion of the erratum 
>>> on the CFRG mailing list.)
>>>
>>> ------------
>>>
>>> Best Regards,
>>>

---

From: Michael StJohns <[email-masked]>
Date: Sun, 27 Dec 2020 16:37:33 -0500

Hi Russ -

Mike

---

From: Michael StJohns <[email-masked]>
Date: Sun, 27 Dec 2020 16:45:14 -0500

On 12/27/2020 4:36 PM, Salz, Rich wrote:
>
>   * As you say, the CFRG is not like many other RGs.  Crypto algorithm
>     specifications, while informational, ofter become normative
>     references in standards.  For this reason, I think robust handling
>     of errata is desirable.
>
>   * Much “crypto research” finds its home here (e.g., the pairing
>     stuff most recently), and I think would be less likely if this
>     were an IETF WG.
All good points - but the fact remains that the CFRG is at times 
(mostly?) doing standards work and frankly, the process they're using 
for documents is much more suited for standards than research papers.

If we kept the CFRG completely to the math side, and let the IETF side 
deal with on the wire/at rest/implementation issues and we were brutal 
about ensuring that, I'd be less bothered about this creeping change to 
the model.

E.g. -> CFRG - crypto math and proofs; CWG -> on the wire and at rest 
formats, equivalent results, test vectors, protocol interfaces.

Later, Mike

---

From: [email-masked]
Date: Sun, 27 Dec 2020 21:52:13 +0000

Hi Mike,

> the IRTF was intended to be.

I disagree. We have what seems to me to be a setup that's worked these past years with technologists and academic cryptographers collaborating  and I see no reason to risk that success for possible process purity.

Cheers, 
S.

---

From: "Salz, Rich" <[email-masked]>
Date: Sun, 27 Dec 2020 21:55:12 +0000

I am glad you said “all good points”

I am worried about making any changes because what we have now is working very well.

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Sun, 27 Dec 2020 21:58:54 +0000

I agree with Stephen and Rich, and strongly disagree with Mike.

I wish no changes to the current CFRG, except maybe for a better errata processing.

Regards,
Uri

> On Dec 27, 2020, at 16:56, Salz, Rich <rsalz=[email-masked]> wrote:
> 
>

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 27 Dec 2020 14:08:37 -0800

To be honest, I'm torn.

IMO Mike is right that what CFRG is doing is making standards and that
that's traditionally the domain of the IETF. OTOH, as Rich, Stephen, and
Uri have pointed out, the current process seems to be working well, so I'd
prefer not to disrupt that. I do think we should acknowledge that these are
standards (we implement based on them, they're full of normative language)
and perhaps it's worth looking into the question of how to publish them on
the standards track. Just thinking out loud, what if the IESG could
cherry-pick to-be-published CFRG documents for ST publication [potentially
at the request of CFRG/IRTFC Chair], do a last call, and if there was
consensus, publish them as Proposed Standard RFCs [perhaps on the IETF
Stream?].

-Ekr

---

From: [email-masked]
Date: Sun, 27 Dec 2020 22:21:06 +0000

Hiya,

> consensus, publish them as Proposed Standard RFCs [perhaps on the IETF

Not sure how much better that'd be vs. the downref registry?

S.

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 27 Dec 2020 14:28:20 -0800

> Not sure how much better that'd be vs. the downref registry?

Well, to the extent to which we think that things being labelled as
Standards Track matters, then it seems suboptimal to have a bunch of
documents which are effectively standards (and in many cases are far more
widely deployed than many of our Standards) being Informational.  And if it
doesn't matter, than one might ask why we bother to publish our protocol
specifications as PS.

I recognize that in the past a number of cryptographic algorithms were
brought in as Informational, but those were largely cases like MD5 where we
were just creating a stable reference for something that was externally
developed. ISTM that that's not the case for much of the CFRG's current
work.

-Ekr

> > > I am worried about making any changes because what we have now is
> working
> > > very well.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 30 Dec 2020 10:46:37 +0000

Hi Michael,

>>     research,*_not to add burdensome bureaucracy to the endeavo_*r.
Colin already replied with most I had to say, but I will add one thing: 
we (CFRG chairs) are not trying to create extra process just for the 
process sake. We handled errata on ad-hoc basis in the past and we 
dropped the ball several times. We got a complain about lack of 
transparency, so the proposed process is to help address that.
> CFRG to create its own processes - no more than it would for a WG.

I was always told that IRTF RG can create own process that suites them.

Best Regards,

Alexey

>> on a CFRG document results in email to [email-masked] 
>> resolution takes into consideration feedback received from the Crypto 
>> Review Panel (if requested). The request should specify a recommended

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 31 Dec 2020 13:19:19 +0000

Hi James,

> You can clarify what role the authors of the RFC have in this process?

I think it is generally very sensible to get authors involved in the 
process and they are typically the first to be asked for opinion. I will 
clarify this in the proposal.

Best Regards,

Alexey