Subject: [arch-d] Proposed IAB program: Evolvability, Deployability, & Maintainability.

From: Tommy Pauly <[email-masked]>
Date: Mon, 06 Jul 2020 12:01:58 -0700

The IAB is considering starting a new program that would look at the Evolvability, Deployability, and Maintainability of Internet protocols. The description of the program is included at the bottom of this email.

We’d like to get feedback on the proposed program topic and description. We’ll be considering this over the next few weeks, and your thoughts and input are welcome!

Best,
Tommy

======

Evolvability, Deployability, & Maintainability (EDM) Program

This program will focus on how to practically encourage best practices within the IETF to ensure that the IETF’s standards process considers protocol evolution, deployability, and long-term maintainability. This group will work on documents that catalog and analyze successful strategies for protocol evolution, deployment, and maintenance, such as updating and extending RFC 6709. Moreover, it will focus on ways to help promote and increase awareness of these strategies within the IETF via program meetings, workshops, and helping organize efforts within working groups and various IETF teams. The program will include members of the IESG and the tools team to help implement these strategies.

The topics this group will consider include:

Evolvability: Encourage protocols to design for extensibility and greasing, and promote the use of extension points to prevent ossification. Make it easy for people, especially those who aren’t steeped in IETF process, to know which extension points are the right ones to use for a given protocol (and which ones should be considered more stable/ossified), and make sure there aren't high allocation barriers to use those extension points.

Deployability: Focus on how we make “running code” something that is better integrated into the working group process. Look at methods to catalog implementations, tools for tracking interoperability testing on different protocol versions, or repositories of which versions of protocols are implemented and used on the Internet. Discuss how deployments can advertise experiments they are running.

Maintainability: Many working groups have expressed a desire to have ways for the community of protocol designers and implementers to keep track of the current state of affairs in deployment patterns, known errata/bugs, or best practices. Help promote consistent ways to host non-RFC working group output, like FAQs, wikis, and discussion venues. Consider how this community involvement can continue and involve the right experts, even after a working group closes.

The program will:
	• Engage with the IETF community and implementers outside of the IETF to identity successes and problem areas, and collect ideas for improvements.
	• Work with the IESG and directorates to determine how best practices for evolvability can best be applied during IESG review and IETF last call.
	• Work with the IESG and tools team on ways to connect implementers and communities that develop and maintain protocols, both while a working group is active, and after a working group has concluded its charter.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 6 Jul 2020 22:01:33 +0200

Thanks, Tommy,

inline

> The IAB is considering starting a new program that would look at the Evolvability, Deployability, and Maintainability of Internet protocols. The description of the program is included at the bottom…

Std toerless rant#1: The Internet is the 10% above waterline of the iceberg which is IETF protocols. Please do not ignore the remaining other 90% of "private network" use. Maybe use term "ITF protocols" or something similar to make it clear whether you want to target 10% or 100% (or anything in between).

> We???d like to get feedback on the proposed program topic and description. We???ll be considering this over the next few weeks, and your thoughts and input are welcome!
> 
> Best,
> Tommy
> 
> ======
> 
> This program will focus on how to practically encourage best practices within the IETF to ensure that the IETF???s standards process considers protocol evolution, deployability, and long-term maintainability. This group will work on documents that catalog and analyze successful strategies for protocol evolution, deployment, and maintenance, such as updating and extending RFC 6709. Moreover, it will focus on ways to help promote and increase awareness of these strategies within the IETF via program meetings, workshops, and helping organize efforts within working groups and various IETF teams. The program will include members of the IESG and the tools team to help implement these strategies.
> 
> Evolvability: Encourage protocols to design for extensibility and greasing, and promote the use of extension points to prevent ossification. Make it easy for people, especially those who aren???t steeped in IETF process, to know which extension points are the right ones to use for a given protocol (and which ones should be considered more stable/ossified), and make sure there aren't high allocation barriers to use those extension points.

I would like to see the work item point:
  "IESG safe documentation of extension point / behavior / possible benefits"

Explanation: i had a painfull IESG review of a draft of mine where we used
a simple terminology at the end of sections saying things like 

"future work may <example of use of an extension point / benefit / ...>".

The IESG review feedback (primarily from GEN) was that this draft looked
unfinished and should not be standardized because of the repeated use of the
term "future work". I had to remove all of this text, partially putting
things into non-normative appendices, etc.. pp.

In later IESG review i had to add some notes about future work with this
terminology, 

Now it is impossible (like in IMHO all other RFC) to easily find extension
point documentation in my draft consistently.

At the risk of getting into a similar endless debate as Mirja's good intentioned
update to "update", maybe there could be terminology like "FUTURE" that
allows to formally show / explain extension points. Or anything else to
easier document them through the IESG review process.

> Deployability: Focus on how we make ???running code??? something that is better integrated into the working group process. Look at methods to catalog implementations, tools for tracking interoperability testing on different protocol versions, or repositories of which versions of protocols are implemented and used on the Internet. Discuss how deployments can advertise experiments they are running.

I think you are listing what likely are the key issues for application/software
centric issues with deployability (down to at most host-stack).

For deployability on network "equipment", router/switches/gateways with
hardware accelerated forwarding and other processing planes, there
are additional considerations IMHO most important for this space:

Models for internal behavior of forwarding/control-plane against which to
assess behavior/scalability/cost of proposed protocols (amount of state,
complexity of QoS / packet processing),.. Architecture direction to evolve
these type of protocol devices into a more "software centric" model.

For example, it took about a decade in my memory before IETF would even
include terms like RIB or FIB into routing area protocol RC because
"dogmatic" IETF persons did reject them in before as something not
externally visible and hence irrelevant to interop behavior.

We had last year a session to talk about router forwarding planes where
a bit of this was touched, but we never followed up with any documentation
in form of BCPs and the like. Obviously this is a lot of work, but
IMHO well worth it if not required for future evolution.

For example 2: I still have ongoing discussions with IESG about the validity
of internal interfaces between SW components in routers in standards track
documents, because it supposedly does not impact interoperability (between
devices), and the mere thought that these software components could be
from different vendors has not entered the mindset of most IESG people working
 with routers.

> Maintainability: Many working groups have expressed a desire to have ways for the community of protocol designers and implementers to keep track of the current state of affairs in deployment patter…

Definitely, but... (see below)

> The program will:
> 	??? Engage with the IETF community and implementers outside of the IETF to identity successes and problem areas, and collect ideas for improvements.
> 	??? Work with the IESG and directorates to determine how best practices for evolvability can best be applied during IESG review and IETF last call.
> 	??? Work with the IESG and tools team on ways to connect implementers and communities that develop and maintain protocols, both while a working group is active, and after a working group has concluded its charter.

This would all be great work to do if it was not executed ONLY via a current
IAB program. This is too important work to be subject to the whim of
IAB member interest. Its not maintaineable if its only done via an
IAB program.

IMHO, this should be an IAB AG (architecture group) that has IAB
members (like IESG members) as sponsors and creators, but which has
AG chairs running a standard RG/WG process, that can persist beyond
the terms of individual IAB members interests, and that isn't subject
to all the other issues i have with IAB processes.

Aka: No need to work on maintaineability if the process to work on
maintainability is not maintainable.

Cheers
    Toerless

> _______________________________________________
> Architecture-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/architecture-discuss

-- 
---
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 7 Jul 2020 10:05:52 +1200

Tommy,

A good topic. But there's an elephant in the corner of the room and I'm not sure you can ignore it. The same elephant is in the corner of the room where we're discussing the future role of the RFC Series Editor, although I've argued that it shouldn't be there because it's an IETF problem.

To what extent is the standards process itself the enemy of Evolvability, Deployability, and Maintainability? In particular, does the way we handle document revisions and interdependencies make it harder for implementors and operators do decide which features to implement and deploy? Why don't we issue many more of what RFC2026 calls "applicability statements"? Why don't we issue living documents that define which documents are part of a particular standard at a particular time?

It's not that we've never done this to some extent. RFC4294->RFC6434->RFC8504 is an example, but only 3 versions over 13 years is grossly inadequate. An update every 6 months might have been more appropriate.

SMTP has been the classical example for many years, and I suspect that RTCWEB might furnish another good example of this problem before long.

Regards
   Brian Carpenter

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 7 Jul 2020 00:20:20 +0200

Brian:

I thought to remember tht Warren mentioned last year some interest
in new processes for "living documents" primarily in OPS. Have
not tracked enough mailing lists to know if/what is happening (if anything),
but i agree that those different type of document lifecycle models
would be crucial to tackle the topics proposed.

But i think it would be perfect to simply learn by doing, and not
bother too much about FIRST working out the document structure and THEN
commit the payload.

Aka: this could be a good set of topics for a living-document-hackathon.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 7 Jul 2020 02:14:03 +0100

Hi Brian,

(Maybe not directly a question for you but...)

On 06/07/2020 23:05, Brian E Carpenter wrote:
> SMTP has been the classical example for many years, and I suspect
> that RTCWEB might furnish another good example of this problem before
> long.
Do we have evidence as to the (good, bad or neutral) impact
of the stately pace of the evolution of email standards-
track  RFCs? I could imagine the net effect being overall
positive or negative but am unsure which may be the case. I
suspect it may be more likely that the actual impact is not
correlated with RFC numbers at all, but that may also be
wrong.

Cheers,
S.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 7 Jul 2020 15:36:58 +1200

> (Maybe not directly a question for you but...)

No, there are many people more qualified to answer than me for the SMTP
case. I'm pretty sure IPv6 would have gone more smoothly with 6-monthly
issues of "Node Requirements", though. There are too many interlocking
pieces and developers were left to make their own choices.

   Brian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 6 Jul 2020 23:53:54 -0400

Brian, I doubt that we could have reached anything resembling rough 
consensus on changes (to IPv6) every six months.  I suspect (although 
this is very much a subjective guess) that focusing on such six-monthly 
deliverables would have made it even harder to get agreement on the hard 
issues that did finally get addressed (however badly) in the revision.

Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 7 Jul 2020 16:26:20 +1200

> issues that did finally get addressed (however badly) in the revision.

Well yes, it would have been extra work and six-monthly might be too
ambitious, but once every 6.5 years was definitely too little. The
underlying issue is that we don't have a mechanism for this at all,
so either people make one up (like Router Requirements, which also
seems to be a little out of date), or we leave implementors and
operators on their own with a heap of RFCs.

Regards
    Brian

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 07 Jul 2020 14:27:40 +1000

Hi Tommy,

This looks like a good initiative.  If there is something the IAB can do to pave the way for multiple protocol communities, then that is good.  There are commonalities in the problems we face, so collating different means of managing those problems is valuable.

On Tue, Jul 7, 2020, at 08:05, Brian E Carpenter wrote:
> To what extent is the standards process itself the enemy of 
> Evolvability, Deployability, and Maintainability? 

>From my perspective, this is a relevant question for such a program.  But I don't see this program as having anything more than an input into other (parallel) work that examines this problem more closely.

More concretely, if this group decides that publishing an RFC is anathema to a particular goal, that is information for others to consider, and a problem to engineer around for this group.

This would seem to support the view that the IAB are well positioned to manage both processes :)

---

From: S Moonesamy <[email-masked]>
Date: Mon, 06 Jul 2020 16:18:36 -0700

Hi Tommy,
At 12:01 PM 06-07-2020, Tommy Pauly wrote:
>The IAB is considering starting a new program that would look at the 
>Evolvability, Deployability, and Maintainability of Internet 
>protocols. The description of the program is included at the bottom 
>of this email.
>
>We'd like to get feedback on the proposed program topic and 
>description. We'll be considering this over the next few weeks, and 
>your thoughts and input are welcome!

One of the roles of the Internet Architecture Board (IAB) is to 
"provide oversight of the architecture for the protocols and 
procedures used by the Internet".  It is not a good idea, in my 
opinion, to stray too far from that role by, for example, replicating 
the role of the Internet Engineering Steering Group (IESG) or being 
involved in matters which are not under its (RFC) policy authority.

I doubt that it is possible to make "running code" part of the 
working group process unless "running code" is interpreted as 
implementing a specification as software-in-the-lab.

In 2017, the IAB issued a response to FCC-17-89.  It stated that "The 
IETF TRANS working group is nearing completion of work on a revision 
to Certificate Transparency".  The work is still incomplete.

Is setting up a program to tackle a people problem is not a good 
recipe for success?  I suggest giving some thought to that.

Regards,
S. Moonesamy

---

From: Tommy Pauly <[email-masked]>
Date: Tue, 07 Jul 2020 08:47:39 -0700

Hi Toerless,

> On Jul 6, 2020, at 1:01 PM, Toerless Eckert <[email-masked]> wrote:
> 
> Thanks, Tommy,
> 
> inline
> 

Sure! “IETF protocols” can be used here—or perhaps just “networking protocols”. This should be inclusive.
> easier document them through the IESG review process.

This is a useful point to bring up. I think discussing ways of highlighting extension points, either via standard text in documents, or by highlighting in a registry like IANA, would be a valuable tool to have across protocols.

> are additional considerations IMHO most important for this space:

Yes, this language is a bit focused on end-hosts, but analogous considerations for routers, etc, would be good to include.

> maintainability is not maintainable.

The work of maintaining mechanisms here long-term probably doesn’t belong in an IAB program or group, but rather as something that is run by the IESG or a WG. This program could certainly recommend such a function.

Best,
Tommy

---

From: Tommy Pauly <[email-masked]>
Date: Tue, 07 Jul 2020 09:13:53 -0700

Hi Brian,

Thanks for your thoughts. I definitely do see the elephant there—but I think there is some good work to do that may in time let us better address the elephant.

I think our processes around standards can, at times, be stumbling blocks to these goals Evolvability, Deployability, and Maintainability. That being said, I don’t think they are fundamentally at odds. The RFC process we have works well for the pieces of work that need to be carefully reviewed, and “set in stone”. Having these stable references is good. But, as you point out, that same process and same format can be less conducive to the more “fluid” work we do, such as applicability statements, surveys of current deployments, etc.

I hope that discussions in this area can explore methods to complement our RFCs, without requiring an overhaul such as adding RFC-series living documents.

For example, we already have choices when we set up IANA registries to set a difficult bar (RFC required) or a very easy bar (first-come-first-served). That kind of choice has a large impact on how a protocol can develop and be used.

I’ve also seen fantastic work being done in some working groups to coordinate implementation testing and deployments, but this is ad-hoc and not easily found from within IETF tools like datatracker.

It may be that the discussions end up determining we need to update our relationship with the RFC process, but I think we can also point to successful ways to complement our standards documents with practices and resources that make our protocols safer to evolve, easier to evolve, and clearer to the community.

Best,
Tommy

> On Jul 6, 2020, at 3:05 PM, Brian E Carpenter <[email-masked]> wrote:
> 
> Tommy,
>

---

From: Tommy Pauly <[email-masked]>
Date: Tue, 07 Jul 2020 09:41:02 -0700

Hello S. Moonesamy,

Thanks for expressing your concern. I want to assure you that this program is not trying to replicate the role of the IESG or any working group, but rather to help coordinate conversations with the IESG, working groups, IETF community, and external communities. The specific focus of these discussions is to look at how we can achieve the architectural goals of an evolvable and maintainable protocol stack.

The problems we are trying to address are indeed architectural: can we avoid or mitigate the ossification of protocols? Can we promote the reuse and extension of IETF protocols and avoid duplication and non-interoperable proprietary solutions?

However, the solution-space involves improving how protocols are defined, how their extension points and allocation spaces are defined, and how working groups communicate and engage with implementer communities.

The proposed program would include IESG members, tools team members, WG chairs, implementers, etc. Those would be the bodies that actually enact changes. The role of the IAB here is to organize the discussions and highlight the impact on architecture.

Here are some references to highlight how this organizational role is indeed within the charter and role of the IAB:

“[The IAB] is also expected to play a role in assuring that
   the people responsible for evolving the Internet and its technology
   are aware of the essential elements of the Internet architecture.”
(https://www.iab.org/about/charter/ <https://www.iab.org/about/charter/>)

"The IAB also helps connect different fields of expertise when this is needed to understand the full situation affecting the evolution of the Internet.” (https://www.iab.org/about/iab-overview/ <https://www.iab.org/about/iab-overview/>)

Best,
Tommy

> On Jul 6, 2020, at 4:18 PM, S Moonesamy <[email-masked]> wrote:
> 
> Hi Tommy,
>> We'd like to get feedback on the proposed program topic and description. We'll be considering this over the next few weeks, and your thoughts and input are welcome!
> 
> One of the roles of the Internet Architecture Board (IAB) is to "provide oversight of the architecture for the protocols and procedures used by the Internet".  It is not a good idea, in my opinion, to stray too far from that role by, for example, replicating the role of the Internet Engineering Steering Group (IESG) or being involved in matters which are not under its (RFC) policy authority.
> 
> I doubt that it is possible to make "running code" part of the working group process unless "running code" is interpreted as implementing a specification as software-in-the-lab.
> 
> In 2017, the IAB issued a response to FCC-17-89.  It stated that "The IETF TRANS working group is nearing completion of work on a revision to Certificate Transparency".  The work is still incomplete.
> 
> Is setting up a program to tackle a people problem is not a good recipe for success?  I suggest giving some thought to that.
> 
> Regards,
> S. Moonesamy

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 7 Jul 2020 19:37:52 +0200

> Yes, this language is a bit focused on end-hosts, but analogous considerations for routers, etc, would be good to include.

Happy to help, let me know if/when you want to go refining text.

> The work of maintaining mechanisms here long-term probably doesn???t belong in an IAB program or group, but rather as something that is run by the IESG or a WG. This program could certainly recommend such a function.

Well, this is a more generic criticism on my behalf on the structure and IAB processes:
IETF WGs are not mean to work on long term architecture topics. thats for IAB.
IRTF GGs are not mean to work on long term architecture topics. thats for IAB.

Aka: Why are there no architecture groups equalling WG/RGs ? IMHO we clearly see
limits to the current process of IAB programs, and IMHO RG/WGs working better.

All the things you bring up ar goin to be long-term architecture work items,
i just can't see this to work well within IAB programs and it does not belong into
RG/WGs, we need AGs too IMHO

Cheers
    Toerless

---

From: S Moonesamy <[email-masked]>
Date: Tue, 07 Jul 2020 11:59:22 -0700

Hi Tommy,

Thanks for the response.  I'll comment below.

At 09:41 AM 07-07-2020, Tommy Pauly wrote:
>Thanks for expressing your concern. I want to assure you that this 
>program is not trying to replicate the role of the IESG or any 
>working group, but rather to help coordinate conversations with the 
>IESG, working groups, IETF community, and external communities. The 
>specific focus of these discussions is to look at how we can achieve 
>the architectural goals of an evolvable and maintainable protocol stack.

That may not be the intent of the IAB.  However, that is what will 
happen as you (used in general terms) get in working group details.

>The problems we are trying to address are indeed architectural: can 
>we avoid or mitigate the ossification of protocols? Can we promote 
>the reuse and extension of IETF protocols and avoid duplication and 
>non-interoperable proprietary solutions?

Ok.

>However, the solution-space involves improving how protocols are 
>defined, how their extension points and allocation spaces are 
>defined, and how working groups communicate and engage with 
>implementer communities.

If you ask about that during a review (unrelated to the proposed 
program) you'll most likely get the "it's the consensus" or some 
other answer which does not explain the how.

The implementers, with some exceptions, do not have much of a say in 
deployment.

>The proposed program would include IESG members, tools team members, 
>WG chairs, implementers, etc. Those would be the bodies that 
>actually enact changes. The role of the IAB here is to organize the 
>discussions and highlight the impact on architecture.

Ok.

>Here are some references to highlight how this organizational role 
>is indeed within the charter and role of the IAB:
>
>   "[The IAB] is also expected to play a role in assuring that
>    are aware of the essential elements of the Internet architecture."
>(<https://www.iab.org/about/charter/>https://www.iab.org/about/charter/)
>
>"The IAB also helps connect different fields of expertise when this 
>is needed to understand the full situation affecting the evolution 
>of the Internet."
>(<https://www.iab.org/about/iab-overview/>https://www.iab.org/about/iab-overview/)

Who are the people responsible for evolving the Internet?

Regards,
S. Moonesamy

---

From: "Christopher Wood" <[email-masked]>
Date: Fri, 10 Jul 2020 08:50:48 -0700

Hi Tommy,

I think this is a terrific idea. All three areas are certainly worth exploring. Deployability and Maintainability in particular might even make sense as topics of their own program, separate from Evolvability. 

However we cut it, I'd like to see progress in the proposed areas, especially for the TLS WG. Methods and tooling support to help track ongoing implementations of protocol extensions would be valuable for the documents we publish and many drafts we have in flight. We have [1] and [2], though they're not well maintained. A common way to track this information for the community would be great.

Best,
Chris

[1] https://github.com/tlswg/tlswg-wiki/blob/master/IMPLEMENTATIONS.md
[2] https://github.com/tlswg/tlswg-wiki/blob/master/TESTING.md

On Mon, Jul 6, 2020, at 12:01 PM, Tommy Pauly wrote:
> Evolvability, Deployability, and Maintainability of Internet protocols. 
> The description of the program is included at the bottom of this email.
> 
> We’d like to get feedback on the proposed program topic and 
> description. We’ll be considering this over the next few weeks, and 
> This program will focus on how to practically encourage best practices 
> within the IETF to ensure that the IETF’s standards process considers 
> protocol evolution, deployability, and long-term maintainability. This 
> group will work on documents that catalog and analyze successful 
> strategies for protocol evolution, deployment, and maintenance, such as 
> updating and extending RFC 6709. Moreover, it will focus on ways to 
> help promote and increase awareness of these strategies within the IETF 
> via program meetings, workshops, and helping organize efforts within 
> working groups and various IETF teams. The program will include members 
> of the IESG and the tools team to help implement these strategies.
> 
> > Maintainability: Many working groups have expressed a desire to have 
> ways for the community of protocol designers and implementers to keep 
> track of the current state of affairs in deployment patterns, known 
> errata/bugs, or best practices. Help promote consistent ways to host 
> non-RFC working group output, like FAQs, wikis, and discussion venues. 
> Consider how this community involvement can continue and involve the 
> right experts, even after a working group closes.
> The program will:
> • Engage with the IETF community and implementers outside of the IETF 
> to identity successes and problem areas, and collect ideas for 
> improvements.
> • Work with the IESG and directorates to determine how best practices 
> for evolvability can best be applied during IESG review and IETF last 
> call.
> • Work with the IESG and tools team on ways to connect implementers and 
> communities that develop and maintain protocols, both while a working 
> group is active, and after a working group has concluded its charter.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 21 Jul 2020 09:06:00 -0500

Because (for reasons I don't understand) I'm involved in AUTH48 processing
for RTCWeb, I find myself wondering whether the proposed IAB program might
include considering ways to avoid C-238-style protocol specification
clusters, whether from the point of view of architectural oversight,
standards process oversight, or RFC Editor oversight.

If you're not aware of C238, check out
https://www.rfc-editor.org/auth48/C238 for a truly amazing picture of what
we are doing today.

I'm sort of kidding, and sort of not.

Best,

Spencer

On Fri, Jul 10, 2020 at 10:51 AM Christopher Wood <[email-masked]>
wrote:

> Hi Tommy,
>
> I think this is a terrific idea. All three areas are certainly worth
> exploring. Deployability and Maintainability in particular might even make
> sense as topics of their own program, separate from Evolvability.
>
> However we cut it, I'd like to see progress in the proposed areas,
> especially for the TLS WG. Methods and tooling support to help track
> ongoing implementations of protocol extensions would be valuable for the
> documents we publish and many drafts we have in flight. We have [1] and
> [2], though they're not well maintained. A common way to track this
> information for the community would be great.
>
> Best,
> Chris
>
> > > Evolvability: Encourage protocols to design for extensibility and
> greasing, and promote the use of extension points to prevent ossification.
> Make it easy for people, especially those who aren’t steeped in IETF
> process, to know which extension points are the right ones to use for a
> given protocol (and which ones should be considered more stable/ossified),
> and make sure there aren't high allocation barriers to use those extension
> points.
> > > Deployability: Focus on how we make “running code” something that is
> better integrated into the working group process. Look at methods to
> catalog implementations, tools for tracking interoperability testing on
> different protocol versions, or repositories of which versions of protocols
> are implemented and used on the Internet. Discuss how deployments can
> advertise experiments they are running.

---

From: Mary Barnes <[email-masked]>
Date: Tue, 21 Jul 2020 09:12:10 -0500

You got put on that list because you had one or more documents as shepherd,
WG chair of any of the WGs in that cluster or you were an author for any of
the documents in the cluster.    So, you get to see all the documents in
the cluster go through the process (if you so desire).  I'm not sure there
was a way around it, since changes in some of the docs could have trickled
down/up to others. But, perhaps, processing the documents (at the
discretion of WG chairs and ADs)and then doing errata and -bises, if there
impacts, might have been a more efficient way of handling all of this
rather than the usual of  waiting for normative dependencies to get
published.

There were 2 docs I shepherded in this cluster, so I've gotten all these
emails.  You can get removed from that list if it's bothersome.

Regards,
Mary.

On Tue, Jul 21, 2020 at 9:06 AM Spencer Dawkins at IETF <
[email-masked]> wrote:

> https://www.rfc-editor.org/auth48/C238 for a truly amazing picture of
> what we are doing today.
>
>> protocols..
>> > The description of the program is included at the bottom of this email..
>> >

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 21 Jul 2020 10:24:04 -0500

Hi, Mary,

On Tue, Jul 21, 2020 at 9:12 AM Mary Barnes <[email-masked]>
wrote:

> You got put on that list because you had one or more documents as
> shepherd, WG chair of any of the WGs in that cluster or you were an author
> for any of the documents in the cluster.    So, you get to see all the
> documents in the cluster go through the process (if you so desire).  I'm
> not sure there was a way around it, since changes in some of the docs could
> have trickled down/up to others. But, perhaps, processing the documents (at
> the discretion of WG chairs and ADs)and then doing errata and -bises, if
> there impacts, might have been a more efficient way of handling all of this

At this point, it's just a marvel to see ;-)

Best,

Spencer, who is kind of hoping he's not STILL listed as responsible AD for
any of these documents, but since everyone is seeing everything, I don't
think that really matters! I don't think I'd qualify under any other
criteria ... not a WG chair, not a doc shepherd, not an author ...

>> Because (for reasons I don't understand) I'm involved in AUTH48
>> processing for RTCWeb, I find myself wondering whether the proposed IAB
>> program might include considering ways to avoid C-238-style protocol
>> specification clusters, whether from the point of view of architectural
>> oversight, standards process oversight, or RFC Editor oversight.
>>

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 22 Jul 2020 08:48:54 +1200

Yes. I don't see C238 as disjoint from the problem I already raised:
https://mailarchive.ietf.org/arch/msg/architecture-discuss/A0ZP4w-UhX7Cu5E8gSPK1haSDWI/
In fact I even mentioned RTCWEB there.

Unsurprisingly, I think that if we don't tackle this problem, technical efforts on Evolvability, Deployability , & Maintainability will be futile.

Regards
   Brian Carpenter

On 22-Jul-20 02:06, Spencer Dawkins at IETF wrote:
> Because (for reasons I don't understand) I'm involved in AUTH48 processing for RTCWeb, I find myself wondering whether the proposed IAB program might include considering ways to avoid C-238-style protocol specification clusters, whether from the point of view of architectural oversight, standards process oversight, or RFC Editor oversight. 
> 
> If you're not aware of C238, check out https://www.rfc-editor.org/auth48/C238 for a truly amazing picture of what we are doing today. 
> 
> On Fri, Jul 10, 2020 at 10:51 AM Christopher Wood <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Hi Tommy,
> 
>     > Evolvability, Deployability, and Maintainability of Internet protocols..
>     > [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: "John Levine" <[email-masked]>
Date: 22 Jul 2020 00:00:28 -0400

>standards process oversight, or RFC Editor oversight.

I think we all agree that we don't ever want to see another document
cluster as large or as long-lived as C238.

But I would rather not attempt to invent processes to try and guess
how best to prevent hypothetical superclusters. I'd rather that the
RPC and I (or a future RSE) work informally with ADs and WG chairs
when we see a potentially large cluster forming, or worse when a
document is about to link two normal sized clusters into a big one.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 22 Jul 2020 16:47:41 +1200

Hi John,

> document is about to link two normal sized clusters into a big one.

That would be wise, but by the time anybody notices, it may be too
late. I do agree that the answer is not to add more process. The 
question on the table, I think, is what factors lead to the formation
of such a cluster, and what design issues should be considered to avoid
it? (Plus the old newtrk question: how can we document clusters of
documents usefully for implementers and operators?)

    Brian

---

From: "John R Levine" <[email-masked]>
Date: 22 Jul 2020 10:18:38 -0400

On Wed, 22 Jul 2020, Brian E Carpenter wrote:
>> cluster as large or as long-lived as C238. ...
>>
>> how best to prevent hypothetical superclusters. ...

> That would be wise, but by the time anybody notices, it may be too late.

The RPC certainly knows how big the clusters are.  I've asked them to tell 
me if they notice any getting large.

> The question on the table, I think, is what factors lead to the formation
> of such a cluster, and what design issues should be considered to avoid it?

Nothing exotic, it's a bunch of drafts with interlinked cross-references. 
If I saw that happening, I'd try and do a topological sort and see if some 
of them can get un-linked, or at least if the linked ones can progress 
together.  The reason C238 gpt so big is that there were drafts in the 
queue that had references to other drafts that weren't done until two 
years later.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 22 Jul 2020 17:11:10 +0200

Obviously we could invent a new RFC publication status like "conditional",
so anything with unresolved normative references gets first published
as a conditional RFC having a new normative section of conditional
normative dependencies, pointing to the draft versions.

As soon as ALL the required normative references exist as RFCs too,
they are all updated in status from conditional to whatever their 
ultimate target status was.

Cheers
    Toerless

On Wed, Jul 22, 2020 at 10:18:38AM -0400, John R Levine wrote:
> Nothing exotic, it's a bunch of drafts with interlinked cross-references. If
> I saw that happening, I'd try and do a topological sort and see if some of
> them can get un-linked, or at least if the linked ones can progress
> together.  The reason C238 gpt so big is that there were drafts in the queue
> that had references to other drafts that weren't done until two years later.
> 
> Regards,

-- 
---
[email-masked]

---

From: "John R Levine" <[email-masked]>
Date: 22 Jul 2020 11:38:07 -0400

> ultimate target status was.

If we're going to republish RFCs I would rather address the general 
versioning question rather than just this special case.  But for this 
special case, I still think a better route is to push WGs to deal with 
drafts in an order that doesn't produce huge clusters.

Some of the drafts in C238 were so old that I worry that the world may 
have changed while the drafts snoozed.

Regards,

---

From: tom petch <[email-masked]>
Date: Wed, 22 Jul 2020 16:19:17 +0000

From: Architecture-discuss <[email-masked]> on behalf of John R Levine <[email-masked]>
Sent: 22 July 2020 16:38

<tp>
I have not looked at C238 but I do know of a draft I worked on that is awaiting progress on a Normative Reference which will fail because the world has changed.  This is a cross-WG issue where one WG sets out to provide something useful, others use it and then the something really useful changes in nature and becomes unusable.

This could happen with YANG common types modules.  Some WG have recognised that types needs several iterations to get right and so have held the types back until at least one using module is ready to advance with it which then reduces the risk.  Others just duplicate the definitions they want which can cause other issues.

Tom Petch
Regards,

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 22 Jul 2020 19:09:10 +0200

Thanks, Tom, inline

On Wed, Jul 22, 2020 at 04:19:17PM +0000, tom petch wrote:
> versioning question rather than just this special case.

Maybe use the cluster problem case as a test balloon for generic versioning,
aka: Lets not create a meta clustet, where the initial benefit of solving
the cluster problem has to wait for the last open question on versioning
to be resolved first.

> But for this

I think everybody will tell you that this is not possible in general
than how we deal with it this today. You can not simply re-assign
authors to first help finishing other documents. We are a community
of arbitrary specialization.

> have changed while the drafts snoozed.

Sure, but if this document would not have been pat of a cluster but
been published years ago as standard RFC it could be equally obsolete
now technically. Aka: do not raise requirements of usefulness against
cluster members over those RFCs that are not cluster members.

Heck, i am still trying to find time to go through all the IETF
process to downgrade 1989 IGMPv1 protocol from full standard to eg
historic and upgrade 2002 IGMPv3 to full standard. Aka: if you
wanted to bring in more applicability requirements against clusters
you just worsen the problem.

Cheers
    Toerless

-- 
---
[email-masked]