Subject: Tracker etiquette

From: "Elwell, John" <[email-masked]>
Date: Wed, 29 Sep 2010 08:43:56 +0200

In the group I co-chair, SIPREC, we have been using the issue tracker. I am aware of its use in several other RAI WGs (SIPCORE, GEOPRIV, MARTINI), and doubtless a lot more groups use it. There has been a thread on SIPCORE recently on tracker etiquette - the way it is used, particularly in conjunction with the mailing lists. During a SIPREC interim yesterday, we received complaints about practices, and clearly come guidelines are needed. Rather than each WG developing its own guidelines, it might make sense to have some common guidelines (although each WG might want to tailor these to its own needs). This WG chairs list seemed like a good place to kick off the discussion.

The sort of issues that are coming up are:
- When to use the mailing list, versus when to add a comment to the tracker.
- Lack of information in the subject field of tracker-generated mailing list messages to indicate the work item to which the issue relates.
- Lack of the original issue text or previous comments in a tracker-generated mailing list message.
- Difficulty interleaving comment text within the original issue text or previous comment text.

Some of these are motivated by the need for somebody reading emails off-line (e.g., on a plane) to know what an email is talking about.

Another issue is long term archives - mailing lists are archived long-term, but I am not sure we have quite the same guarantees with the tracker.

Another issue I personally experience is slowness of response when using the tracker (I don't know to what extend this affects others), which makes it inconvenient to use the tracker as a substitute for information that could have been provided in the mailing list message and discourages me from using it for simply adding a comment, say. (This isn't the prime focus of this thread, but for me, average response time for navigating from one issue to the next, back to query, and so on, is about 12-13s, compared with about 2-3s for datatracker queries.)

It seems to me the tracker is good for recording an issue and its subsequent changes of state. It is not so good for discussing an issue. Even when an issue is closed, say, there needs to be something on the mailing list that says what the issue is that has been closed, not just its tracker number.

So questions for the list:
- Do people consider this to be a topic worth discussing?
- What other issues do people have?
- Are there any guidelines already in place?
- How to proceed, e.g., do we need a draft, do we want an informal discussion at IETF#79 (e.g., during chairs' lunch)?

I will stress that this is not intended as criticism of the tracker. The aim is just to develop guidelines on use of the tracker, although as a side-effect we might come up with some ideas for future enhancement of the tracker.

John

---

From: "David Harrington" <[email-masked]>
Date: Wed, 29 Sep 2010 09:52:38 -0400

Hi,

I haven't used the issue tracker recently.
I tried a few years ago, and found, as you did,
"It seems to me the tracker is good for recording an issue and its
subsequent changes of state. It is not so good for discussing an
issue. Even when an issue is closed, say, there needs to be something
on the mailing list that says what the issue is that has been closed,
not just its tracker number."

I find that using a web page or wiki is easier for tracking issues
through the process.
A web page can list the issues, and anybody can read the issues list
to see what's happening.

My favorite web page listed each issue with a graphic (stop light)
that indicated open, closing, and closed.
"closing means the chairs have started a consensus call"
This was easy for everybody to understand quickly, especially which
issues were no longer open to discussion.
The issues list can provide a link to the message thread that
identified the issue.
(sometimes an issue splits or combines, and the chairs then need to
create separate threads.)

I found coordinating between the official mailing list discussions and
any issues tracker discussions more work than it was worth.

I did find that I could track chair issues fairly easily, but
hopefully the new datatracker will make that way easier to handle in a
consistent manner.

David Harrington
Director, IETF Transport Area
[email-masked] (preferred for ietf)
[email-masked]
+1 603 828 1401 (cell)

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Elwell, John
> Sent: Wednesday, September 29, 2010 2:44 AM
> To: 'IETF WG Chairs'
> Cc: Henrik Levkowetz
> Subject: Tracker etiquette
> 
> In the group I co-chair, SIPREC, we have been using the issue 
> tracker. I am aware of its use in several other RAI WGs 
> (SIPCORE, GEOPRIV, MARTINI), and doubtless a lot more groups 
> use it. There has been a thread on SIPCORE recently on 
> tracker etiquette - the way it is used, particularly in 
> conjunction with the mailing lists. During a SIPREC interim 
> yesterday, we received complaints about practices, and 
> clearly come guidelines are needed. Rather than each WG 
> developing its own guidelines, it might make sense to have 
> some common guidelines (although each WG might want to tailor 
> these to its own needs). This WG chairs list seemed like a 
> good place to kick off the discussion.
> 
> - Lack of information in the subject field of 
> tracker-generated mailing list messages to indicate the work 
> item to which the issue relates.
> Some of these are motivated by the need for somebody reading 
> emails off-line (e.g., on a plane) to know what an email is 
> talking about.
> 
> Another issue is long term archives - mailing lists are 
> archived long-term, but I am not sure we have quite the same 
> guarantees with the tracker.
> 
> Another issue I personally experience is slowness of response 
> when using the tracker (I don't know to what extend this 
> affects others), which makes it inconvenient to use the 
> tracker as a substitute for information that could have been 
> provided in the mailing list message and discourages me from 
> using it for simply adding a comment, say. (This isn't the 
> prime focus of this thread, but for me, average response time 
> for navigating from one issue to the next, back to query, and 
> so on, is about 12-13s, compared with about 2-3s for 
> datatracker queries.)
> 
> It seems to me the tracker is good for recording an issue and 
> its subsequent changes of state. It is not so good for 
> discussing an issue. Even when an issue is closed, say, there 
> needs to be something on the mailing list that says what the 
> issue is that has been closed, not just its tracker number.
> 
> - How to proceed, e.g., do we need a draft, do we want an 
> informal discussion at IETF#79 (e.g., during chairs' lunch)?
> 
> I will stress that this is not intended as criticism of the 
> tracker. The aim is just to develop guidelines on use of the 
> tracker, although as a side-effect we might come up with some 
> ideas for future enhancement of the tracker.
> 
> John=
>

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 29 Sep 2010 08:13:37 -0700

At 8:43 AM +0200 9/29/10, Elwell, John wrote:
>. . .
>. . .

The first bullet is *definitely* worth discussing. The IETF today relies on WG issues being discussed on the mailing list, while other organizations rely on them being discussed in the tracker. This needs to be made clear for people who come from organizations that use the latter style.

The second bullet is also useful and should be easy to fix once we agree on what it is we want. I'm not so concerned about the third and fourth bullets if we agree on the first.

>- What other issues do people have?

Trying to get WG members to look at the tracker items. IPsecME is tracker-based for our issues, and I don't think Yaron and I got the kind of review we were hoping for on our last few documents. I still think that using the tracker to name and segregate issues is a greater positive than the negative of a bit less interaction from the list, but I would like to hear ideas about how to get more of the "second tier" WG participants to respond to tracker issues.

>- How to proceed, e.g., do we need a draft, do we want an informal discussion at IETF#79 (e.g., during chairs' lunch)?

--Paul Hoffman, Director
--VPN Consortium

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Sep 2010 08:31:44 -0700

Hi, John,

> - Difficulty interleaving comment text within the original issue text or previous comment text.

For discussions, IMO the mailing list is the appropriate place.

IMO, the tracker is a good place only if a draft has a large number of 
pending issues, and only to keep track of which ones are still open vs. 
have been addressed.

For the mailing list, I've led discussions (on mine and other drafts) 
with multiple issues by NOT posting a list of the issues in a single 
mail. If you want per-topic threads, you need to start with the call for 
discussion posts, i.e., by posting a separate message on each issue to 
the list, i.e.:

	(docname) ISSUE #4: (topic)

The tracker could be helpful in supporting this, e.g., if it:

a) sent initial messages to the list with something like the above 
format whenever new messages

b) was able to post followups in a way that looks like a reply thread

The key problem, AFAICT, is that even if the tracker *outputs* messages 
like this, the info flow is unidirectional. The list sees what goes into 
the tracker. Looking at the tracker, however, won't give a full view of 
the exchanges on an issue unless there's a way to subscribe the tracker 
to the mailing list and have it scan for replies and enter them accordingly.

I'm not sure whether that's worth the effort. AFAICT, the tracker may be 
a good place to check the status of changes, but I would trust the 
mailing list as the primary source for discussions about those changes.

Joe

---

From: "David Harrington" <[email-masked]>
Date: Wed, 29 Sep 2010 12:53:06 -0400

There was a discussio of issue tracking at an iesg lunch 2-3 meetings
ago, IIRC.
If you're lucky, some of that got captured in the proceedings.
 
dbh

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Paul Hoffman
> Sent: Wednesday, September 29, 2010 11:14 AM
> To: Elwell, John; 'IETF WG Chairs'
> Cc: Henrik Levkowetz
> Subject: Re: Tracker etiquette
> 
> The first bullet is *definitely* worth discussing. The IETF 
> today relies on WG issues being discussed on the mailing 
> list, while other organizations rely on them being discussed 
> in the tracker. This needs to be made clear for people who 
> come from organizations that use the latter style.
> 
> The second bullet is also useful and should be easy to fix 
> once we agree on what it is we want. I'm not so concerned 
> about the third and fourth bullets if we agree on the first.
> 
> Trying to get WG members to look at the tracker items. 
> IPsecME is tracker-based for our issues, and I don't think 
> Yaron and I got the kind of review we were hoping for on our 
> last few documents. I still think that using the tracker to 
> name and segregate issues is a greater positive than the 
> negative of a bit less interaction from the list, but I would 
> like to hear ideas about how to get more of the "second tier" 
> WG participants to respond to tracker issues.
>

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 30 Sep 2010 15:18:57 -0400

If the tracker is to be of any use at all, it must be archival -- the issues and comments need to be preserved *at least as well* as any associated mailing list's messages.

I assume that the IETF is doing this *already*.  If not, then they shouldn't be operating a tracker.

The fundamental value of a tracker is maintaining a database of open issues so that it is easy to ensure that all issues get taken care of.  It is very easy for a mailing list discussion to "fall off the radar".

In regard to the messages from the tracker to a mailing list being insufficiently informative to be useful, it seems to me that this is a problem of the specific tracker, and not a problem with trackers per se.

Dale

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 1 Oct 2010 08:33:27 +0200

Dale,

Whether or not the tracker as such is archived (I assume is isn't), each tracker change (status change, comment) results in a message being sent to the WG mailing list, so I guess it is archived in that sense. The main difficulty is correlating everything.

For example, one benefit of the tracker is that it keeps record of issues raised perhaps several months ago that have remained dormant, so when such an issue is eventually addressed, a mail with #87 in its subject line will correlate only with a mail from several months back that similarly has #87 in the subject line. Without the previous thread being embedded in the new mail, the reader is left to search back through the mailing list archives to find the older mails (and most likely that requires the person to be on-line, unless old mails have been cached locally).

John

> From: Worley, Dale R (Dale) [mailto:[email-masked]] 
> Sent: 30 September 2010 20:19
> Subject: RE: Tracker etiquette
> 
> If the tracker is to be of any use at all, it must be 
> archival -- the issues and comments need to be preserved *at 
> least as well* as any associated mailing list's messages.
> 
> The fundamental value of a tracker is maintaining a database 
> of open issues so that it is easy to ensure that all issues 
> get taken care of.  It is very easy for a mailing list 
> discussion to "fall off the radar".
> 
> In regard to the messages from the tracker to a mailing list 
> being insufficiently informative to be useful, it seems to me 
> that this is a problem of the specific tracker, and not a 
> problem with trackers per se.
> 
> Dale
>