Subject: [IRTF-Announce] Call for Papers: Workshop on Analyzing IETF Data (AID), 2021

From: IAB Executive Administrative Manager <[email-masked]>
Date: Thu, 26 Aug 2021 10:15:47 -0700

Show me the numbers: Workshop on Analyzing IETF Data (AID), 2021

Web Page: https://www.iab.org/activities/workshops/aid/

The IETF as an international Standards Developing Organization hosts 
diverse data on the history, development, and current activities in the 
development and standardization of Internet protocols and its 
institutions. A large portion of this data is publicly available, yet 
this data is arguably underutilized as a tool to inform the work in the 
IETF and research on topics like Internet governance and trends in ICT 
standard-setting.

This workshop aims to enable engineers and researchers alike to mine the 
IETF's data sources in order to explore trends through the analysis of 
IETF data, such as email archives 
<https://mailarchive.ietf.org/arch/browse/>, I-Ds 
<https://www.ietf.org/standards/ids/>, RFCs 
<https://www.ietf.org/standards/rfcs/>, and the datatracker 
<https://datatracker.ietf.org/>. This work can be used to derive 
insights into the inner workings of the process of standardization, 
participation, and governance[1]. This workshop aims to bring together 
people who have already analyzed IETF data, those who are interested in 
the analysis of IETF data, and those who are interested in the results 
of such analysis as input for improvement of the IETF's work.

We invite the research community, IETF participants, and others with an 
interest in the data collected by the IETF, its protocols, and 
participants, to submit a contribution to the workshop. Furthermore, we 
also welcome participants who are interested in the analysis that could 
be performed based on this data as well as those contributing 
considerations regarding future collection and handling of IETF data.

Possible avenues for explorations include, but are not limited to:

  A. What are patterns for participation in the IETF (what are 
     predictors for a long and productive tenure, when do people stop 
     participating, what is needed to successfully produce RFCs)?
  B. How is the IETF community developing (i.e., affiliations, 
     publications, language, nationality, leadership positions)?
  C. How do affiliations develop in the IETF (i.e., does a change in 
     affiliation translate into a change in behavior, is there a 
     relation between affiliation and leadership positions and/or 
     centrality, what is the affiliation distribution per area and/or 
     WG)?
  D. What social dynamics (gender, nationality, income, occupation, and 
     other social dynamics) are not captured by IETF data and what data 
     and research approaches are needed to develop further insights in 
     the social dynamics of standardization?
  E. How productive and effective is the IETF, with respect to 
     documents, pages, words, letters and in comparison the overall 
     activities e.g. on mailing lists?
  F. How well is the outcome of the IETF used, e.g,. based on references 
     to RFCs in research papers, product manuals, or other sources?
  G. What data would be relevant to collect that is not collected yet or 
     what should be considered with respect to handling of personal data 
     during the data collection and research.
  H. How effective is the IETF's consensus-based decision making 
     process? Is there evidence that documents receive broad and 
     effective reviews? Are experts with relevant expertise engaging 
     with developing standards in a timely manner?

Participation and Submission

People interested in participation are requested to submit short 
position papers (500-1000 words). The paper can cover one or multiple of 
the following points, but this list should not be considered exhaustive:

  1. Research questions and interests in IETF data; indication which 
     question should be answered, the data needed to do so, and how 
     these insights could be used to improve processes and operations;
  2. Description of the IETF data they aim to analyze or the information 
     they would like to see made available to inform their work (such as 
     mailing list archives, or participation data obtained through the 
     datatracker) and their methods for doing so (see footnote 1);
  3. Potential and preliminary findings; and how those insights could 
     either benefit leadership, WG chairs, and authors/participants, 
     and/or society and industry at large;
  4. Potential or preliminary findings and how those add novel insights 
     to ongoing academic debates.

Proposals for data analysis should also contain a brief consideration of 
any related ethics and privacy issues. The basic principles of ethical 
research are outlined in the Belmont Report2 (covering e.g., respect for 
persons, beneficence, and justice) and/or institutional ethics 
guidelines.

The workshop will be invitation-only. The organizers will decide whom to 
invite based on the submissions received. Therefore, please indicate 
your interest by submitting a research proposal by September 29, 2021 to 
[email-masked].

The Program Committee members are Niels ten Oever (chair, University of 
Amsterdam), Colin Perkins (chair, IRTF, University of Glasgow), Corinne 
Cath (chair, Oxford Internet Institute), Mirja Kühlewind (IAB, 
Ericsson), Zhenbin Li (IAB, Huawei), Wes Hardaker (IAB, USC/ISI).

All inputs submitted and considered relevant will be published on the 
workshop web page. Sessions will be organized according to content, and 
not every accepted submission or invited attendee will have an 
opportunity to present as the intent is to foster discussion and not 
simply to have a sequence of presentations.

Position papers from those unable to attend in person are encouraged. A 
workshop report will be published afterwards.

Logistics

   • Submissions Due: 29 September 2021
   • Invitations Issued by: 15 October 2021
   • Workshop Date: November 29 – December 3 2021
   • Location: Online and at the University of Amsterdam (COVID-19 
     permitting).

The workshop will consist of three parts:

  1. opening workshop (Monday)
  2. hackathon (Tuesday – Thursday morning)
  3. closing event (Thursday afternoon)

Feel free to contact the program committee with any further questions 
(including questions related to available data or expected outcomes): 
[email-masked].

-----
[1] Examples of such approaches are: 
https://www.arkko.com/tools/docstats.html, 
http://datactive.github.io/bigbang/, 
https://csperkins.org/research/protocol-standards/2020-12-10-ignacio-iesg-talk/2020-12-10_IESG-50-years-IETF-send.pdf, 
https://sodestream.github.io/impact-of-early-engagement-on-longevity-of-ietf-participation.html

[2] https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_FINAL.pdf