Subject: Re: Tracker etiquette

From: Yaron Sheffer <[email-masked]>
Date: Fri, 01 Oct 2010 13:21:07 +0200

Whether an email is sent on each Tracker change is a configurable 
option. In the case of ipsecme, we decided early on that these mails 
would overload the mailing list, and we turned them off.

On the other hand, I just found out that you can subscribe to an RSS 
feed for Tracker changes, which may be more convenient for some people.

Thanks,
	Yaron

On 10/01/2010 08:33 AM, Elwell, John wrote:
> Dale,
>
> Whether or not the tracker as such is archived (I assume is isn't), each tracker change (status change, comment) results in a message being sent to the WG mailing list, so I guess it is archived in that sense. The main difficulty is correlating everything.
>
> For example, one benefit of the tracker is that it keeps record of issues raised perhaps several months ago that have remained dormant, so when such an issue is eventually addressed, a mail with #87 in its subject line will correlate only with a mail from several months back that similarly has #87 in the subject line. Without the previous thread being embedded in the new mail, the reader is left to search back through the mailing list archives to find the older mails (and most likely that requires the person to be on-line, unless old mails have been cached locally).
>
> John
>
>> -----Original Message-----
>> From: Worley, Dale R (Dale) [mailto:[email-masked]]
>> Sent: 30 September 2010 20:19
>> To: Elwell, John; 'IETF WG Chairs'
>> Cc: Henrik Levkowetz
>> Subject: RE: Tracker etiquette
>>
>> If the tracker is to be of any use at all, it must be
>> archival -- the issues and comments need to be preserved *at
>> least as well* as any associated mailing list's messages.
>>
>> I assume that the IETF is doing this *already*.  If not, then
>> they shouldn't be operating a tracker.
>>
>> The fundamental value of a tracker is maintaining a database
>> of open issues so that it is easy to ensure that all issues
>> get taken care of.  It is very easy for a mailing list
>> discussion to "fall off the radar".
>>
>> In regard to the messages from the tracker to a mailing list
>> being insufficiently informative to be useful, it seems to me
>> that this is a problem of the specific tracker, and not a
>> problem with trackers per se.
>>
>> Dale

---

From: Ted Lemon <[email-masked]>
Date: Fri, 1 Oct 2010 06:17:01 -0700

> Whether or not the tracker as such is archived (I assume is isn't), each tracker change (status change, comment) results in a message being sent to the WG mailing list, so I guess it is archived in…

The other difficulty is that these messages have a lot of cruft in them.   I tend to resist using the tracker because the noise it generates seems harmful to the flow of the mailing list.   I tend to agree with Dale that it would be better if this were handled as a separate archive.   I also tend to think that we ought to use one or the other system, not both.   I think mailing lists are more in keeping with how the IETF operates.   I'd like to have a tool that allows me (and others) to track issues as they are discussed on the mailing list, rather than being its own self-contained system.

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Fri, 1 Oct 2010 10:43:04 -0400

________________________________________
From: Elwell, John [[email-masked]]

For example, one benefit of the tracker is that it keeps record of issues raised perhaps several months ago that have remained dormant, so when such an issue is eventually addressed, a mail with #87 in its subject line will correlate only with a mail from several months back that similarly has #87 in the subject line. Without the previous thread being embedded in the new mail, the reader is left to search back through the mailing list archives to find the older mails (and most likely that requires the person to be on-line, unless old mails have been cached locally).
________________________________________

I normally use trackers in a different way...  If I receive notification about some change in a tracker issue that interests me, I follow the contained link to the tracker issue page.  If people are using the tracker well, that page contains the issue and its comments, which should be record of all major aspects of the issue.  (Discussions that have not come to conclusions are probably on a mailing list, which one can track down if one is interested.)  If people aren't putting the important facts into the tracker issue, then of course, one will be stuck searching the mailing list archives -- but that is exactly what a well-operated tracker is intended to avoid.

Dale

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 1 Oct 2010 11:50:52 -0400

The tracker is certainly archived, and backed up on 3 separate servers.

Regards
Marshall

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 1 Oct 2010 18:08:45 +0200

Dale,

In my limited experience, it has been very difficult to get people disciplined always to use the tracker when commenting on a tracker issue. Several reasons:
- It is simply easier to reply to an email.
- People sometimes work off-line on emails that have already been downloaded, and access to the tracker to read about an issue, let alone comment on an issue, is infeasible.
- Slowness of response time.
- It is not so easy to interleave your comments.
- The fact that prior to raising a ticket there is invariably at least one email on the list, even after the ticket has been raised people will still continue to respond to the original email thread rather than going via the tracker.
- The previous point is in fact a real pain, because nobody takes the trouble to change the subject line to include the ticket number.

These tend to lead me to using the mailing list for discussion, and using the tracker mainly to record the original issue and state changes. Although the last point above goes against that.

John

> Sent: 01 October 2010 15:43
> Whether or not the tracker as such is archived (I assume is 
> isn't), each tracker change (status change, comment) results 
> in a message being sent to the WG mailing list, so I guess it 
> is archived in that sense. The main difficulty is correlating 
> everything.
> 
> For example, one benefit of the tracker is that it keeps 
> record of issues raised perhaps several months ago that have 
> remained dormant, so when such an issue is eventually 
> addressed, a mail with #87 in its subject line will correlate 
> only with a mail from several months back that similarly has 
> #87 in the subject line. Without the previous thread being 
> embedded in the new mail, the reader is left to search back 
> through the mailing list archives to find the older mails 
> (and most likely that requires the person to be on-line, 
> unless old mails have been cached locally).
> I normally use trackers in a different way...  If I receive 
> notification about some change in a tracker issue that 
> interests me, I follow the contained link to the tracker 
> issue page.  If people are using the tracker well, that page 
> contains the issue and its comments, which should be record 
> of all major aspects of the issue.  (Discussions that have 
> not come to conclusions are probably on a mailing list, which 
> one can track down if one is interested.)  If people aren't 
> putting the important facts into the tracker issue, then of 
> course, one will be stuck searching the mailing list archives 
> -- but that is exactly what a well-operated tracker is 
> intended to avoid.
> 
> Dale
>

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 1 Oct 2010 18:09:41 +0200

Thanks for clarifying - good to know.

John 

> From: Marshall Eubanks [mailto:[email-masked]] 
> Sent: 01 October 2010 16:51
> To: Worley, Dale R (Dale)
> Cc: Elwell, John; 'IETF WG Chairs'; Henrik Levkowetz
> Subject: Re: Tracker etiquette
> 
>

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Fri, 1 Oct 2010 14:44:39 -0400

True, if people aren't going to use the tracker as a tracker, then there's no point in using the tracker.

Dale

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 1 Oct 2010 12:47:09 -0600

So I  have several concerns around trackers but in some ways this archival question is my biggest. 

I regularly have to deal with patents where IETF email list traffic get used as prior art in a patent defense. Given the IETFs IPR policy, how open we are, etc, it is very easy for someone to watch were discussions are going and write patent applications. So if some of the text is only available in the tracker, we need a URL like 

http://trac.tools.ietf.org/wg/martini/trac/ticket/47

to be accessible for a long time to come. 30 years or more. Now obviously we could deal with issues like the trac.tools.ietf.org domain name got changed to something else over time but we need to have a commitment that this text will be available for a long time, or we need to to make sure that 100% of the useful text in the tracker makes it to some other long term archive. 

We also have to look at tracker information when dealing with appeals and even just trying to track down why some decision was made 5 years ago and if we could change it. 

Next we come to the issue of who's tool is this anyways. Are the trackers a IETF owned and supported tool or a tool team thing. I love the tool team for everything they do for us but if IAOC is willing to commit to make sure this data is available over the long term? Even if 10 years from now we stop using trac and move to something different? 

I wonder about backups too - I'm sure backups are made but I'm more concerned about the restores than the backups. If I noticed tomorrow than the tracker databases from the webdav group had been corrupted a year ago, what would I do to get it restored? I'm less concerned about email archive backups because I know that I can make my own backups and that enough other people archive this that even if the IETF looses it, I will be able to find it. 

I also have concerns about if the Note Well statement applies to text entered on the tracker. I think we could (and should) find a way to make it explicit that it does given the trackers often have changes suggested by a non document author than the document editor then copies into the draft.

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 1 Oct 2010 14:08:07 -0600

Uh, to make this more concrete, I just went  to the webdav tracker and it does not seem to be up. Who would I talk to about getting it restored?

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 01 Oct 2010 23:51:04 +0200

On 2010-10-01 22:08 Cullen Jennings said the following:
> 
> Uh, to make this more concrete,

To make *what* more concrete?

> I just went  to the webdav tracker and it does not seem to be up.
> Who would I talk to about getting it restored?

What is the URL of the webdav tracker you're referring to?

	Henrik

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 2 Oct 2010 00:47:54 +0200

On Oct 1, 2010, at 20:47, Cullen Jennings wrote:

> [...] or we need to to make sure that 100% of the useful text in the tracker makes it to some other long term archive. 

To me it seems pretty clear that the only responsible way to operate the tracker is the mode where it sends each and every update to the mailing list.

(I don't buy the argument that the mailing list would be overwhelmed by this.  Yes, it can be a bit noisy when someone opens and then fixes 30 tickets in one day, but the resulting messages can be rather easily ticked off by an uninterested mailing list member.  I also don't buy a legitimate member in the WG even could be uninterested.)

Gruesse, Carsten

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 1 Oct 2010 19:03:40 -0400

> To me it seems pretty clear that the only responsible way to operate the tracker is the mode where it sends each and every update to the mailing list.

I think that for Cullen's purposes both need to be archived.

Regards
Marshall

---

From: Ted Lemon <[email-masked]>
Date: Fri, 1 Oct 2010 16:48:15 -0700

> (I don't buy the argument that the mailing list would be overwhelmed by this.  Yes, it can be a bit noisy when someone opens and then fixes 30 tickets in one day, but the resulting messages can be…

That's not the issue.  The issue is that unlike regular mail, it is a lot of work to evaluate each message to see that it either is or is not of interest, because it's not generated in the usual way and doesn't have the usual tags. E.g. I think the subject line is an issue number.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 2 Oct 2010 10:32:58 +0200

On Oct 2, 2010, at 01:48, Ted Lemon wrote:

> it is a lot of work to evaluate each message to see that it either is or is not of interest, because it's not generated in the usual way and doesn't have the usual tags. E.g. I think the subject line is an issue number.

Ah, now it gets interesting -- there might indeed be potential for improvement.

Let's look at this in specific terms.
An example tracker mail:

http://www.ietf.org/mail-archive/web/core/current/msg00677.html

The Subject: line 

	[core] #23: Virtual server capability?

identifies the ticket number (and the usage of "#" is nicely unique).
It also identifies the subject of the ticket (which was chosen by the submitter to be useful for this).

(In my WGs, this already works very well as is, but:)

It does not identify:
-- the specific deliverable this ticket is on (here: coap)
-- the state transition, if any (here: new ticket).

Something like:

	[core] coap #23 (new): Virtual server capability?

might be more useful in WGs with more deliverables.
(Etiquette might be used to put in the deliverable tag manually for now.)

(You might also note how the ticket submitter excerpted and summarized previous mail to make the ticket useful on its own.  This is good etiquette.  It would be even nicer if there was an easier way to also refer to previous mail.)

Gruesse, Carsten (who should really suffix every wgchairs mail with the disclaimer
"My WGs may be different from your WGs")

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 02 Oct 2010 11:11:32 +0200

Hi Carsten, all,

> (Etiquette might be used to put in the deliverable tag manually for now.)

Implemented.

Best,

	Henrik

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 2 Oct 2010 07:03:02 -0600

> That's not the issue.  The issue is that unlike regular mail, it is a lot of work to evaluate each message to see that it either is or is not of interest, because it's not generated in the usual wa…

One way that would meet both my goals and Teds would be for a WG like martini to have all tracker output go to a a separate email lists, say martini-noise. People like my could subscribe to both, or at least both would be archived, and others could not subscribe to the noise.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 4 Oct 2010 17:18:29 -0600

I'm a bit confused on the IETF level of support for the tracker and how the IAOC see this issue. I'm extremely grateful for all the things Henrik and various tool teams do for folks. And let me insert a big thank you to them here. However, having them responsible for backups and long term archives does not sound like an optimal plan - I could have nightmares about Henrik deciding to open a brew pub as far away from the internet as possible and sending us a post card every year to let us know how happy he was. My understanding from Glen is that AMS has nothing to do with any of the trackers, backups, or their archives. 

Doodle is a tool used by IETF folks all the time and has nothing to do with IETF and we don't have archives of it. That's fine given what we use it for. However, the tracker is being used to make key decisions, is the documented text of some of our conversations and proposals, has people cutting text out of the tracker and assuming that the text is covered under "Note Well" and so on. If 100% of the useful text from the trackers went to an achieved emails list, and it was clear that however stuff got in the tracker was "Note Well" covered, then it would solve these issues regardless of how or who ran the trackers.

---

From: Adam Roach <[email-masked]>
Date: Mon, 04 Oct 2010 20:25:49 -0500

On 10/4/10 18:18, Oct 4, Cullen Jennings wrote:
> If 100% of the useful text from the trackers went to an achieved emails list, and it was clear that however stuff got in the tracker was "Note Well" covered, then it would solve these issues regardless of how or who ran the trackers.

Cullen:

Can you point out any text that has been entered into the tracker but is 
not available on the corresponding working group's mailing list? At 
least, from what I've seen, all of the information that gets into the 
tracker is also sent to the mailing lists. Yeah, it's not convenient, 
but it's there.

Right?

/a

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 05 Oct 2010 14:06:38 +0200

Hi Adam,

> tracker is also sent to the mailing lists. Yeah, it's not convenient, 

Till now, it's been up to the individual WGs how they want to configure
the tracker notifications -- should messages be sent to the mailing list
or not? -- and some WGs have chosen to not have messages sent to the
mailing list.

There should still not be an issue of the tracker information disappearing,
as they are (as Marshall already mentioned) backed up on multiple separate
servers, to which not only I but other key persons have access.  However,
the main point here is (I think) to find a level and method of archiving
which the community is comfortable with, rather than a question of the
tracker databases being securely backed up.

Some possible alternatives:

  a) change the notification settings so that WGs may not choose to turn
     off notifications to the list

  b) as in a), but consistently send notifications to separate lists and
     list archives

  c) don't force notifications to be sent, but ensure that archiving of
     the Trac databases are done also on secretariat servers, to increase
     the certainty that they will be available long-term

For a) and b), I think we also need to reduce the noise of the messages;
I'd suggest we get rid of the table-like beginning of each mail which
lists the attributes of each issue, and keep only the change information.

(I'd hoped that this discussion would come up with a suggestion for also
that (in addition to Carsten's excellent suggestion for improving the
subject line) but with nothing in that direction turning up so far, I
guess I can make an opening suggestion :-)

Regards,

	Henrik

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 5 Oct 2010 10:19:15 -0700

Hi -

> From: "Henrik Levkowetz" <[email-masked]>
> To: "Adam Roach" <[email-masked]>
> Cc: "'IETF WG Chairs'" <[email-masked]>
> Sent: Tuesday, October 05, 2010 5:06 AM
...
> lists the attributes of each issue, and keep only the change information.

The ltru  working group's reaction to tracker notifications was so negative
that I think (a) would have resulted in the working group not using the official
tracker at all, and in the chairs resorting to their own tools to track
the (rather large) number of issues.  However, in that particular case, having
the tracker available made handling appeals much easier than it otherwise
might have been.  Consequently, option (c) would have been my preference.

Randy

---

From: "Elwell, John" <[email-masked]>
Date: Thu, 7 Oct 2010 17:12:50 +0200

Cullen,

In SIPREC, where you raised this question, the tracker is configured to send every change as a message to the SIPREC mailing list, so everything is archived and backed up. I had forgotten that there is an option to turn this off, but by having it turned on we solve the archive and back-up problems, and also have the whole archive for the WG, in one place. The issue then reduces to:
- how to use the tracker so that it serves its prime purpose (keeping track of issues) but does not make life hard for the WG participants in general, or those examining the archives;
- what possible enhancements to the tracker would help in achieving the above.

John

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Cullen Jennings
> Sent: 02 October 2010 14:03
> To: Ted Lemon
> Cc: IETF WG Chairs; Henrik Levkowetz
> > > (I don't buy the argument that the mailing list would be 
> overwhelmed by this.  Yes, it can be a bit noisy when someone 
> opens and then fixes 30 tickets in one day, but the resulting 
> messages can be rather easily ticked off by an uninterested 
> mailing list member.  I also don't buy a legitimate member in 
> the WG even could be uninterested.)
> > 
> > That's not the issue.  The issue is that unlike regular 
> mail, it is a lot of work to evaluate each message to see 
> that it either is or is not of interest, because it's not 
> generated in the usual way and doesn't have the usual tags. 
> E.g. I think the subject line is an issue number.
> > 
> 
> One way that would meet both my goals and Teds would be for a 
> WG like martini to have all tracker output go to a a separate 
> email lists, say martini-noise. People like my could 
> subscribe to both, or at least both would be archived, and 
> others could not subscribe to the noise.

---

From: Yaron Sheffer <[email-masked]>
Date: Thu, 07 Oct 2010 20:20:49 +0200

Backing up the Tracker is a simple and common IT problem. Publishing 
each and every Tracker change to the mailing list creates a usability 
problem for (in our case) more than 1000 people. The tradeoff seems 
quite clear to me.

Thanks,
     Yaron

On 10/07/2010 05:12 PM, Elwell, John wrote:
> Cullen,
>
> In SIPREC, where you raised this question, the tracker is configured 
> to send every change as a message to the SIPREC mailing list, so 
> everything is archived and backed up. I had forgotten that there is an 
> option to turn this off, but by having it turned on we solve the 
> archive and back-up problems, and also have the whole archive for the 
> WG, in one place. The issue then reduces to:
> - how to use the tracker so that it serves its prime purpose (keeping 
> track of issues) but does not make life hard for the WG participants 
> in general, or those examining the archives;
> >> On Oct 1, 2010, at 5:48 PM, "Carsten Bormann"<[email-masked]>  wrote:
> >>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 08 Oct 2010 19:31:13 +0900

> lists the attributes of each issue, and keep only the change information.

I suggest you just put that at the end of the mail, so it's easier to 
ignore, but still there if needed. Another enhancement may be to leave 
out those items that are blank.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]