Subject: [Gen-art] Gen-ART review of draft-aboba-sg-experiment-02

From: <[email-masked]>
Date: Fri, 28 Sep 2007 14:02:34 +0300

I have been selected as the General Area Review Team (Gen-ART) 
reviewer for this draft (for background on Gen-ART, please see 
http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html). 

Please resolve these comments along with any other Last Call 
comments you may receive. 

Document: draft-aboba-sg-experiment-02
Reviewer: Pasi Eronen
Review Date: 2007-09-28
IETF LC End Date: 2007-10-08
IESG Telechat date: 2007-10-04

Summary: This draft is ready for publication as an Experimental RFC.

Comments: 

Personal opinion: we could certainly spend a lot of time fine-tuning
the experiment description and adding details, but I agree with 
RFC 3933 when it says that "an experimental attempt to try something 
for a fixed period is typically a better engineering approach than 
extended philosophical discussion without any experience to back it up".

In other words: I don't really know if this is a very good idea or 
not, but I don't see any strong reasons why we should't try this
experiment for a limited time.

One possible addition that might help in evaluating the experiment
when it ends: according to the current text, IESG is required to
inform the community of any Study Groups formed. I think decisions
where a SG was *not* formed would be equally interesting.

Best regards,
Pasi

_______________________________________________
Gen-art mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/gen-art

---

From: Jari Arkko <[email-masked]>
Date: Fri, 28 Sep 2007 14:16:21 +0300

Thanks for your review, Pasi!

> where a SG was *not* formed would be equally interesting.
Yes.

But generally speaking, new work proposals can be classified in
three categories:

1) Too confused, no clear demand, etc.
2) Demand is clear, but work remains to determine what exactly the group
should do
3) Demand and details are all clear.

A proposal from the first category cannot progress. In the
second category we'd consider an SG, and in the third
a WG.

In terms of reporting to the community and evaluating results,
we try to be as clear as possible about the status of the different
efforts whether or not we have an ongoing experiment. But
its clear that understanding whether the SG concept helped
is interesting. Just that it may be hard to compare a proposal
from category 1 with another one from 2 or 3.

Jari

---

From: <[email-masked]>
Date: Fri, 28 Sep 2007 15:23:05 +0300

I guess what I'm interested in is the boundary between categories 1
and 2. Evaluating the usefulness of this SG experiment would benefit
also from knowing what kinds proposals ended up in category 1.

(Although that might be difficult to do in an objective and 
neutral way -- after all, for every work proposal, there's 
probably at least one person who think's it very clear and 
there's plenty of demand :-)

Best regards,
Pasi

> -----Original Message-----
> From: ext Jari Arkko [mailto:[email-masked]] 
> Sent: 28 September, 2007 14:16
> To: Eronen Pasi (Nokia-NRC/Helsinki)
> Cc: [email-masked]; [email-masked]; 
> [email-masked]; [email-masked]
> Subject: Re: Gen-ART review of draft-aboba-sg-experiment-02
> 
> 2) Demand is clear, but work remains to determine what 
> exactly the group should do

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 28 Sep 2007 06:45:12 -0700

I think the question is whether the Experiment will collect sufficient data to determine whether
it would be useful to keep it going, and if so, what changes need to be made.  
 
Currently, the document states that demonstration of interest is a prerequisite, but little else. 
However, interest by itself is probably not enough to allow a SG to have a decent chance of
success. Figuring out when a SG can be profitably formed may turn out to be an important
learning from the experiment, but the question is whether the current proposal will provide
that information. 
 
Are we asking the IESG to identify which RFC 2418 criteria were not met for WG formation?

________________________________

From: [email-masked] [mailto:[email-masked]]
Sent: Fri 9/28/2007 5:23 AM
To: [email-masked]
Cc: [email-masked]; Bernard Aboba; [email-masked]; [email-masked]
Subject: RE: Gen-ART review of draft-aboba-sg-experiment-02

Best regards,
Pasi

---

From: Scott Brim <[email-masked]>
Date: Fri, 28 Sep 2007 11:08:21 -0400

> also from knowing what kinds proposals ended up in category 1.

It would seem there is an iterative process.  For example, demand may
start high, and as the work scope clarifies, go down and then up
again.

What we need is an estimate from the IESG of whether an idea could
turn into something based on the *potential* of both.  For example,
can the scope be clarified, and if the scope clarifies do they guess
that demand will increase.  I don't think you need quantifiable
metrics.  That feels like "physics envy" instead of human process.

Scott

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 28 Sep 2007 10:44:49 -0700

Are there specific text changes that are being recommended? 

-----Original Message-----
From: Scott Brim [mailto:[email-masked]]
Sent: Fri 9/28/2007 8:08 AM
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]; Bernard Aboba
Subject: [Gen-art] RE: Gen-ART review of draft-aboba-sg-experiment-02

Scott

---

From: Jari Arkko <[email-masked]>
Date: Sun, 30 Sep 2007 22:50:02 +0300

Bernard,

Yes.
 
> Are we asking the IESG to identify which RFC 2418 criteria were not

Ideally this should be the case already when decisions about BOF
and WG acceptance are communicated. But I hope you are not
asking to make this formal.

Jari

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 30 Sep 2007 15:25:50 -0700

Jari said:

"Ideally this should be the case already when decisions about BOF
and WG acceptance are communicated. But I hope you are not
asking to make this formal."

I don't think that's necessary.  I'm just wondering if any text
changes are being proposed.